Friday, 9 April 2021

Cisco SD-WAN - Part III: Overlay Management Protocol

 

Introduction

 

This chapter introduces the operation of the Overlay Management Protocol (OMP). It starts by introducing TLOC Routes which are used for establishing tunnels between vEdges. Next, it explains OMP Routes which in turn are used for advertising client VPN-specific networks reachability information. I am also going to show the data plane encapsulation when data is sent between the hosts in site 10 and site 30. The purpose of the data plane section is to show how the label attribute advertised within OMP routing advertisements is used to identify customer VPN. In order to see inside captured packets, I am using GRE tunnels instead of IPSec. Figure 3-1 illustrates the example topology used in this chapter. The customer VPN 10 is used on both sites. Site 10 subnet is 172.16.10.0/24 and site 30 subnet is 172.16.30.0/24. Interface ge0/­0 in both vEdges is connected to the Public-Internet, and interface ge0/1 is the connected to MPLS transport network where the customer has its dedicated MPLS VPN.



Figure 3-1: SD-WAN Example Topology.



Service VPN Configuration

 

This short section shows how to configure service VPNs in vEdge using CLI. First, we create a VPN and attach an interface to it. Next, we assign an IP address to the interface, enable it and then commit changes. Examples 3-1 and 3-2 show the configuration steps.

 

vEdge-1(config)# vpn 10

vEdge-1(config-vpn-10)# interface ge0/2

vEdge-1(config-interface-ge0/2)# ip address 172.16.10.1/24

vEdge-1(config-interface-ge0/2)# no shutdown

vEdge-1(config-interface-ge0/2)# commit

Example 3-1: VPN 10 Configuration on vEdge-1.

 

 

vEdge-3(config)# vpn 10

vEdge-3(config-vpn-10)# interface ge0/3

vEdge-3(config-interface-ge0/3)# ip address 172.16.30.1/24

vEdge-3(config-interface-ge0/3)# no shutdown

vEdge-3(config-interface-ge0/3)# commit

Example 3-2: VPN 10 Configuration on vEdge-3.

 

 

TLOC Routes

 

Transport Location Identifier (TLOC) route advertisement carries the Overlay Network reachability information. The TLOC entry consists of three main elements and a set of attributes. The first element is the system IP address that is used, as its name describes, to identify the system/device. These addresses are only known by the SD-WAN vEdge devices and the control plane components. The Underlay Network devices are unaware of these IP addresses. The next TLOC element is the color which is used for identifying the transport network. The third element in the TLOC entry describes the tunneling protocol. I am using the Generic Route Encapsulation (GRE) protocol in order to show unencrypted data plane traffic. In real life, the other option,  IPSec is the obvious choice. Each TLOC routes has also a set of attributes, just like we have in BGP.

The Public-IP address attribute describes the routable destination IP address used in the outer IP header (tunnel header). The Private-IP attribute in turn describes the physical IP address of the vEdge interface connected to a specific transport network. In a solution where vEdge is behind the NAT device, the Private-IP is translated to Public-IP which we are using in the tunnel header. When vEdge receives the TLOC routing advertisement, it knows that the advertising vEdge is behind the NAT device if these two addresses are different. The Site-Id attribute is self-explanatory, it identifies the site. The Restrict attribute in our example is set to one, which implies that we only want to build a tunnel (GRE or IPSec) between the vEdge interfaces connected to the same transport network. By default, this attribute is not set, which means that vEdges tries to establish full mesh tunnels between all transport interfaces no matter to which transport network (defined by color) they belong. As an example, without setting the restrict attribute vEdge1 in figure 3-2 tries to build GRE tunnels with vEdge3 from Public-Internet to Public-Internet but also from Public-Internet to MPLS and the same cross color connection over MPLS transport. This will not work since there is no routing between these two transport networks.

vEdges has OMP peering over the DTLS tunnel only with vSmart(s) that in turn distributes the received information to other vEdges. In figure 3-2 vEdge1 sends the TLOC route to vSmart that forwards it to vEdge3. Based on the TLOC routing information it builds two tunnels, one over MPLS and the other one over Public-Internet to vEdge using the GRE encapsulation. It uses the public-IP address as a next-hop address in the tunnel IP header. When the tunnel is up, vEdges start monitoring the tunnel transport capability by using Bi-directional Forwarding Detection (BFD). These tunnels are then used for Service VPN traffic sent between vEdges.

 

Figure 3-2: OMP TLOC Peering and TLOC Route Advertisement.

 

Examples  3-3, 3-4, and 3-5 verify that OMP peering is established only between the vSmart and vEdges.

 

vEdge-1# show omp peers

R -> routes received

I -> routes installed

S -> routes sent

 

                         DOMAIN    OVERLAY   SITE

PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S

--------------------------------------------------------------------------------------

10.100.100.13    vsmart  1         1         100       up       0:00:21:58       2/2/2

Example 3-3: OMP peers of vEdge1.

 

 

vEdge-3# show omp peers

R -> routes received

I -> routes installed

S -> routes sent

 

                         DOMAIN    OVERLAY   SITE

PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S

--------------------------------------------------------------------------------------

10.100.100.13    vsmart  1         1         100       up       0:00:23:01       2/2/2

Example 3-4: OMP peers of vEdge3.

 

 

vsmart# show omp peers

R -> routes received

I -> routes installed

S -> routes sent

 

                         DOMAIN    OVERLAY   SITE

PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S

--------------------------------------------------------------------------------------

10.100.100.101   vedge   1         1         10        up       0:00:23:42       2/0/2

10.100.100.103   vedge   1         1         30        up       0:00:23:40       2/0/2

Example 3-5: OMP peers of vSmart.

 

Example 3-6 shows the TLOC route originated by vEdge and advertised to vSmart.

 

vEdge-1# show omp tlocs advertised detail | exclude not | nomore

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 mpls

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.200.0.101

     public-port       0

     private-ip        10.200.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x80000010

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

            ADVERTISED TO:

peer    10.100.100.13

    Attributes:

     encap-key         0

     public-ip         10.200.0.101

     public-port       0

     private-ip        10.200.0.101

     private-port      0

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     site-id           10

     preference        0

     weight            1

     version           3

     gen-id            0x80000010

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 public-internet

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.100.0.101

     public-port       0

     private-ip        10.100.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x80000010

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

            ADVERTISED TO:

peer    10.100.100.13

    Attributes:

     encap-key         0

     public-ip         10.100.0.101

     public-port       0

     private-ip        10.100.0.101

     private-port      0

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     site-id           10

     preference        0

     weight            1

     version           3

     gen-id            0x80000010

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

Example 3-6: TLOC Route advertised to vSmart by vEdge1.

 

 

Example 3-7 verify that vSmart has received two TLOC routes from vEdge1 which is also has advertised to vEdge3. The status of these routes indicates that vSmart has selected (C = chosen) this TLOC route to be installed (I) into the routing table and the next-hop address defined Public-IP attribute is reachable (R). Note that vSmart installs all valid TLOC routes into RIB but it does not install OMP routes describing Service VPNs into the RIB.

 

 

vsmart# show omp tlocs detail | exclude not | nomore | until tloc\ entries\ for\ 10.100.100.103

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 mpls

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.101

status          C,I,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.200.0.101

     public-port       0

     private-ip        10.200.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

            ADVERTISED TO:

peer    10.100.100.103

    Attributes:

     encap-key         0

     public-ip         10.200.0.101

     public-port       0

     private-ip        10.200.0.101

     private-port      0

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     site-id           10

     preference        0

     weight            1

     version           3

     gen-id            0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 public-internet

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.101

status          C,I,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.100.0.101

     public-port       0

     private-ip        10.100.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

            ADVERTISED TO:

peer    10.100.100.103

    Attributes:

     encap-key         0

     public-ip         10.100.0.101

     public-port       0

     private-ip        10.100.0.101

     private-port      0

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     site-id           10

     preference        0

     weight            1

     version           3

     gen-id            0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

 

---------------------------------------------------

tloc entries for 10.100.100.103

Example 3-7: TLOC Route Entry about vEdge1 TLOCs in vSmart.

  

The example below verifies that vEdge3 has received the TLOC route and that it is also installed into the routing table (status C, I, R). Based on the restrict attribute vEdge3 only builds the GRE tunnels with vEdge only between the TLOCs sharing the same color (transport network identifier). vEdge uses its TLOC specific public-IP address as a source IP and the public IP received via TLOC route as a destination IP in GRE tunnel header when sending data packets.

 

vEdge-3# show omp tlocs ip 10.100.100.101 detail | exclude not\ set | nomore

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 mpls

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.13

status          C,I,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.200.0.101

     public-port       0

     private-ip        10.200.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 public-internet

                 gre

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.13

status          C,I,R

    Attributes:

     attribute-type    installed

     encap-key         0

     public-ip         10.100.0.101

     public-port       0

     private-ip        10.100.0.101

     private-port      0

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     site-id           10

     preference        0

     weight            1

     version           3

    gen-id             0x8000000f

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

Example 3-8: TLOC Route Entry about vEdge1 TLOCs in vEdge3.

 

Example 3-9 shows that vEdge has two GRE tunnels with vEdge1 (10.100.100.101), one over Public-Internet and one over MPLS transport. There are no tunnels between different local-remote colors.


Tunnel Verification

 

 

vEdge-3# show tunnel statistics

tunnel stats gre 10.100.0.103 10.100.0.101 0 0

 system-ip      10.100.100.101

 local-color    public-internet

 remote-color   public-internet

 tunnel-mtu     1471

 tx_pkts        7027

 tx_octets      751722

 rx_pkts        7026

 rx_octets      653168

 tcp-mss-adjust 1391

tunnel stats gre 10.200.1.103 10.200.0.101 0 0

 system-ip      10.100.100.101

 local-color    mpls

 remote-color   mpls

 tunnel-mtu     1471

 tx_pkts        7061

 tx_octets      808604

 rx_pkts        7026

 rx_octets      653129

 tcp-mss-adjust 1391

Example 3-9: GRE tunnel statistics.

 

vEdge has also started the BFD session with vEdge1. Note, in order to install OMP routes about service VPN networks, there has to be an active BFD session to system-ID described in TLOC.

 

vEdge-3# show bfd sessions system-ip 10.100.100.101 | beg 10.

 

10.100.100.101 10  up public-internet public-internet  10.100.0.103  10.100.0.101  gre 

10.100.100.101 10  up mpls            mpls             10.200.1.103  10.200.0.101  gre

Example 3-10: BFD session to vEdge verification on vEdge3.

 

Examples 3-11 and 3-12 shows that we have IP reachability over the transport networks between the Public-IP addresses.

 

vEdge-3# ping vpn 0 10.200.0.101 source 10.200.1.103 count 3

Ping in VPN 0

PING 10.200.0.101 (10.200.0.101) from 10.200.1.103 : 56(84) bytes of data.

64 bytes from 10.200.0.101: icmp_seq=1 ttl=61 time=2.54 ms

64 bytes from 10.200.0.101: icmp_seq=2 ttl=61 time=2.35 ms

64 bytes from 10.200.0.101: icmp_seq=3 ttl=61 time=2.26 ms

 

--- 10.200.0.101 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2003ms

rtt min/avg/max/mdev = 2.267/2.390/2.548/0.123 ms

Example 3-11: BFD session to vEdge verification on vEdge3.

 

 

vEdge-3# ping vpn 0 10.100.0.101 source 10.100.0.103 count 3

Ping in VPN 0

PING 10.100.0.101 (10.100.0.101) from 10.100.0.103 : 56(84) bytes of data.

64 bytes from 10.100.0.101: icmp_seq=1 ttl=64 time=1.40 ms

64 bytes from 10.100.0.101: icmp_seq=2 ttl=64 time=1.29 ms

64 bytes from 10.100.0.101: icmp_seq=3 ttl=64 time=5.46 ms

 

--- 10.100.0.101 ping statistics ---

3 packets transmitted, 3 received, 0% packet loss, time 2001ms

rtt min/avg/max/mdev = 1.294/2.721/5.462/1.938 ms

Example 3-12: BFD session to vEdge verification on vEdge3.

 

 

OMP Routes

 

While TLOC routes are used for tunnel setup between vEdges, OMP routes are used for advertising reachability information related to Service VPN subnets. The OMP route entry includes VPN and the subnet information as well as a set of attributes. In our example, vEdge1 advertises its Service VPN 10 subnet 172.16.10.0/24 to vSmart. The advertisement carries attributes that describe the TLOCs and the label value that is used in data plane encapsulation as VPN identifier. vSmart verifies the reachability of the system-ID of TLOC (based on the TLOC route described in the previous section) before forwarding the advertisement to vEdge3. vSmart doesn’t install the route in its RIB because it is not in the data path i.e. it never has to route packets to Service VPN subnets.

 

Figure 3-3: OMP Route Advertisement.

 

Example 3-13 shows the OMP route about VPN 10 subnet 172.16.10.0/24 from the vEdge perspective. The subnet is reachable through both transport networks. The VPN 10 is identified with label 1003. The Red in the status field means that route is redistributed into OMP (connected routes are redistributed automatically).

  

vEdge-1# show omp routes advertised detail | exclude not | nomore

 

---------------------------------------------------

omp route entries for vpn 10 route 172.16.10.0/24

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

path-id         34

label           1003

status          C,Red,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, mpls, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            RECEIVED FROM:

peer            0.0.0.0

path-id         37

label           1003

status          C,Red,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, public-internet, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            ADVERTISED TO:

peer    10.100.100.13

    Attributes:

     originator       10.100.100.101

     label            1003

     path-id          34

     tloc             10.100.100.101, mpls, gre

     site-id          10

     overlay-id        1

     origin-proto     connected

     origin-metric    0

    Attributes:

     originator       10.100.100.101

     label            1003

     path-id          37

     tloc             10.100.100.101, public-internet, gre

     site-id          10

     overlay-id        1

     origin-proto     connected

     origin-metric    0

Example 3-13: OMP route about VPN 10 subnet 172.16.10.0/24 on vEdge1.

 

Example 3-14 shows that vSmart has received the OMP route. That status shows that route is selected (C) and the system-ID is reachable (R) so the update is valid and vSmart can forward the update to vEdge3.

 

vsmart# show omp routes received | until 172.16.30.0 | exclude not\ set | nomore

 

---------------------------------------------------

omp route entries for vpn 10 route 172.16.10.0/24

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.101

path-id         34

label           1003

status          C,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, mpls, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            RECEIVED FROM:

peer            10.100.100.101

path-id         37

label           1003

status          C,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, public-internet, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

 

---------------------------------------------------

omp route entries for vpn 10 route 172.16.30.0/24

Example 3-14: Received OMP route about VPN 10 subnet 172.16.10.0/24 on vSmart.

 

Example 3-15 shows that vSmart has advertised the OMP routes to vEdge3.

 

vsmart# show omp routes advertised detail | until 172.16.30.0 | exclude not| nomore

 

---------------------------------------------------

omp route entries for vpn 10 route 172.16.10.0/24

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.101

path-id         34

label           1003

status          C,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, mpls, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            RECEIVED FROM:

peer            10.100.100.101

path-id         37

label           1003

status          C,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, public-internet, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            ADVERTISED TO:

peer    10.100.100.103

    Attributes:

     originator       10.100.100.101

     label            1003

     path-id          1

     tloc             10.100.100.101, mpls, gre

     site-id          10

     overlay-id        1

     origin-proto     connected

     origin-metric    0

    Attributes:

     originator       10.100.100.101

     label            1003

     path-id          2

     tloc             10.100.100.101, public-internet, gre

     site-id          10

     overlay-id        1

     origin-proto     connected

     origin-metric    0

 

---------------------------------------------------

omp route entries for vpn 10 route 172.16.30.0/24

Example 3-15: Advertised OMP route about VPN 10 subnet 172.16.10.0/24 on vSmart.

 

Example 3-16 shows that vEdge3 has received an update from vSmart. It also shows that vEdge3 has installed a route into the RIB. In order to install an OMP route to RIB, the TLOC IP address has to be reachable meaning there has to be a valid TLOC route that describes the Public-IP of TLOC IP. Besides, there also has to be an active BFD session with the originating vEdge1.

 

vEdge-3# show omp routes received | until 172.16.30.0 | exclude not\ set | nomore

---------------------------------------------------

omp route entries for vpn 10 route 172.16.10.0/24

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.13

path-id         1

label           1003

status          C,I,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, mpls, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

            RECEIVED FROM:

peer            10.100.100.13

path-id         2

label           1003

status          C,I,R

    Attributes:

     originator       10.100.100.101

     type             installed

     tloc             10.100.100.101, public-internet, gre

     overlay-id        1

     site-id          10

     origin-proto     connected

     origin-metric    0

Example 3-16: Received OMP route about VPN 10 subnet 172.16.10.0/24 on vEdge3.

 

Example 3-17 shows the routing entry about the VPN 10 subnet 172.16.10.0/24 on vEdge3 RIB. Note that the Administrative Distance (AD) is the same 250 as what IOS devices use for NHRP routes. If you are using IOS-XE devices in your SD-WAN solution, then the AD is 251.

 

vEdge-3# show ip routes vpn 10 detail | until 172.16.30.0/24 | nomore

Codes Proto-sub-type:

  IA -> ospf-intra-area, IE -> ospf-inter-area,

  E1 -> ospf-external1, E2 -> ospf-external2,

  N1 -> ospf-nssa-external1, N2 -> ospf-nssa-external2,

  e -> bgp-external, i -> bgp-internal

Codes Status flags:

  F -> fib, S -> selected, I -> inactive,

  B -> blackhole, R -> recursive, L -> import

 

--------------------------------------------

 VPN 10      PREFIX 172.16.10.0/24

--------------------------------------------

 proto           omp

 distance        250

 metric          0

 uptime          0:00:56:59

 tloc-ip         10.100.100.101

 tloc-color      mpls

 tloc-encap      gre

 nexthop-label   1003

 status          F,S

 

--------------------------------------------

 VPN 10      PREFIX 172.16.10.0/24

--------------------------------------------

 proto           omp

 distance        250

 metric          0

 uptime          0:00:56:59

 tloc-ip         10.100.100.101

 tloc-color      public-internet

 tloc-encap      gre

 nexthop-label   1003

 status          F,S

 

--------------------------------------------

 VPN 10      PREFIX 172.16.30.0/24

Example 3-17: RIB Verification in vEdge3.

 

 IP Reachability  Verification
 

Examples 3-18 and 3-19 verifies that we have IP connectivity between subnets in site 10 and 30.

 

vEdge-3# ping vpn 10 172.16.10.10 source 172.16.30.1 count 2

Ping in VPN 10

PING 172.16.10.10 (172.16.10.10) from 172.16.30.1 : 56(84) bytes of data.

64 bytes from 172.16.10.10: icmp_seq=1 ttl=63 time=3.35 ms

64 bytes from 172.16.10.10: icmp_seq=2 ttl=63 time=2.68 ms

Example 3-18: Dataplane Testing by Pinging from vEdge3 to VPN 10 IP address on vEdge1.

 

Srv10> ping 172.16.10.10

 

84 bytes from 172.16.10.10 icmp_seq=1 ttl=62 time=4.143 ms

84 bytes from 172.16.10.10 icmp_seq=2 ttl=62 time=3.159 ms

Example 3-19: Dataplane Testing by Pinging from host 172.16.30.30 to 172.16.10.10.


Data Plane

 

Figure 3-4 shows the encapsulated ICMP messages when sent over transport networks. When the data path goes over the Public-Internet, there is only the tunnel header which in our case is GRE with VPN label. The packet sent over the MPLS transport first has the MPLS Transport label used within the MPLS network for switching packets from ingress PE to egress PE. Then there is a VPN label that identifies the customer VPN in PE devices. Within these MPLS headers, there is a GRE tunnel header.


Figure 3-4: Data Packet Captures.

 

Capture 3-1 shows the encapsulated ICMP message from 172.16.30.30 to 172.16.10.10 over the Public-Internet transport network. The MPLS label is received with the OMP route and it is used as a VPN identifier in remote vEdge.

 

Internet Protocol Version 4, Src: 10.100.0.103, Dst: 10.100.0.101

Generic Routing Encapsulation (MPLS label switched packet)

MultiProtocol Label Switching Header, Label: 1003, Exp: 0, S: 1, TTL: 64

    0000 0000 0011 1110 1011 .... .... .... = MPLS Label: 1003

    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0

    .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1

    .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64

Internet Protocol Version 4, Src: 172.16.30.30, Dst: 172.16.10.10

Internet Control Message Protocol

Capture 3-1: Data Packet Captures from the Public-Internet transport.

 

Capture 3-2 shows the encapsulated ICMP message from 172.16.30.30 to 172.16.10.10 over the MPLS transport network. So basically we are creating a GRE tunnel inside MPLS VPN. This increases overall complexity because in the MPLS transport network we need LDP or Segment Routing for exchanging label binding information for label switching (outer MPLS label) and BGP VPNv4 Unicast for exchanging the customer VRF reachability information (inner/customer MPLS label).

 

MultiProtocol Label Switching Header, Label: 16, Exp: 0, S: 0, TTL: 62

    0000 0000 0000 0001 0000 .... .... .... = MPLS Label: 16

    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0

    .... .... .... .... .... ...0 .... .... = MPLS Bottom Of Label Stack: 0

    .... .... .... .... .... .... 0011 1110 = MPLS TTL: 62

MultiProtocol Label Switching Header, Label: 18, Exp: 0, S: 1, TTL: 62

    0000 0000 0000 0001 0010 .... .... .... = MPLS Label: 18

    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0

    .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1

    .... .... .... .... .... .... 0011 1110 = MPLS TTL: 62

Internet Protocol Version 4, Src: 10.200.1.103, Dst: 10.200.0.101

Generic Routing Encapsulation (MPLS label switched packet)

MultiProtocol Label Switching Header, Label: 1003, Exp: 0, S: 1, TTL: 64

    0000 0000 0011 1110 1011 .... .... .... = MPLS Label: 1003

    .... .... .... .... .... 000. .... .... = MPLS Experimental Bits: 0

    .... .... .... .... .... ...1 .... .... = MPLS Bottom Of Label Stack: 1

    .... .... .... .... .... .... 0100 0000 = MPLS TTL: 64

Internet Protocol Version 4, Src: 172.16.30.30, Dst: 172.16.10.10

Internet Control Message Protocol

Capture 3-2: Data Packet Captures from the MPLS transport.


 

Summary

 

If we compare the control plane operation in the Viptela based SD-WAN solution to the VXLAN Fabric DC solution, we can see a lot of similarities. TLOC routes are used for advertising the tunnel endpoint public IP address while this task in VXLAN fabric is done by control plane protocol used in the underlay network (OSPF, IS-IS, or BGP). The customer subnet and VPN identifier (label) and encapsulation used in the data plane are advertised with OMP routes in Viptela SD-WAN, while in VXLAN fabric the same task is done by MP-BGP L3VPN EVPN update where the customer VPN is identified with Virtual Network Identifier (VNI). The Viptela based SD-WAN solution uses a centralized control plane model where updates are sent to vSamart that forwards valid updates to other vEdges. The same model is also used with VXLAN fabric, Leaf switches send updates only to Spine switches which forwards valid updates to other Leaf switches or Super-Spine switches in case we are using three tiers,  5-stage Clos fabric. So why there is again one new control-plane protocol doing the same job that can be done with existing, standard protocols? The OMP also distributes the encryption keys for tunnel encryption. BGP at the time of writing is not capable of doing that. Besides, OMP messages are sent over DTLS tunnels while BGP update messages are sent as clear text.



Here are the books that I have published. 



3 comments:

  1. do you have show running configuration in your book?

    ReplyDelete
  2. Yes, there are several show commands in each chapetrs.

    ReplyDelete
  3. Hello Tony,I have been reading your articles of SD WAN series.They are meticulously written and are tremendously helpful. Thanks for sharing.If you are looking for network services especially SD-WAN in India ,"Skylark" would be the one stop solution for Data Center Networking,Wi-Fi Solutions,SD-WANs and Campus & Branch Networking. To know more,please visit
    https://www.skylarkinfo.com/enterprise-networking/sd-wan

    ReplyDelete