Introduction
In this article, I will go through the process of DMVPN tunnel establishment between Spoke sites. I am using the same network topology and network setup than in my previous article “DMVPN Part I. Basic Operation and Configuration, March 31, 2017”. Note that prefix for all interfaces/networks is 2001:db8. Figure 1 shows the processes of Spoke-to-Spoke tunnel establishment.
Figure 1.
At starting point, there are mGRE tunnels from both Spoke routers (NHCs) S2R1 and S3R1 to Hub router S1R1 (NHS) but not between each others. Internal BGP peering has been configured between the tunnel interfaces. Note that iBGP peering is done only between Hub and Spoke routers, not between Spoke routers. Both Spoke routers mutually redistribute routes between OSPF and BGP. Hub router S1R1 is BGP Route-Reflector and Spoke routers are RR-Clients. In the starting point, we have not yet sent any traffic between Spoke sites.
Let’s start to send a traffic between Spoke sites.
Step-2 (Data Plane): Router S2R2 starts pinging the loopback address of the router S3R2 (2001:db8:beef::32). The source address is its loopback interface 2001:db8:cafe::22. It has learned the route to the destination via OSPF from the router S2R1.
Step-2a (Data Plane): Router S2R1 receives the ICMP query packet. It has a route to the network destination 2001:db8:beef::/64 with a next hop 2001:db8:0:100::11 (Tunnel interface of S1R1) learned via BGP.
S2R1#show ipv6 route 2001:DB8:beef::/64
Routing entry for 2001:DB8:BEEF::/64
Known via "bgp 64501", distance 200, metric 2, type internal
Redistributing via ospf 1
Backup from "NHRP-IPv6 [250]"
Route count is 1/1, share count 0
Routing paths:
2001:DB8:0:100::11
MPLS label: nolabel
From 2001:DB8:0:100::11
Last updated 06:14:41 ago
Router S2R1 has the NBMA address mapping entry for the address 2001:db8:0:100::11 in its NHRP cache. Router S2R1 encapsulates the packet with GRE IPv6 header with the destination address 2001:db8:11::1 (taken from NHRP cache) and forwards the packet to S1R1 over the Transport Network.
S2R1#sh ipv6 nhrp 2001:db8:0:100::11/128
2001:DB8:0:100::11/128 via 2001:DB8:0:100::11
Tunnel100 created 11:53:16, never expire
Type: static, Flags: used
NBMA address: 2001:DB8:11::1
Step-2b (Data Plane): Router S1R1 receives the ICMP Query packet. The destination address is its NBMA address, therefore it removes the outer header and notices that the real destination in inner header is 2001:db8:beef::32. It has an entry for the destination in the RIB.
S1R1#sh ipv6 route 2001:db8:beef::32
Routing entry for 2001:DB8:BEEF::/64
Known via "bgp 64501", distance 200, metric 2, type internal
Route count is 1/1, share count 0
Routing paths:
2001:DB8:0:100::31
MPLS label: nolabel
From 2001:DB8:0:100::31
Last updated 07:15:05 ago
Router S3R1 has already sent the NHRP registration message to the router S1R1, where it describes the NBMA address of its Tunnel100 interface 2001:db8:0:100:31. So router S1R1 encapsulates the packet with GRE IPv6 header with the destination address 2001:db8:31::1 and forwards the packet to the router S3R1 over the Transport Network.
S1R1#sh ipv6 nhrp 2001:db8:0:100::31/128
2001:DB8:0:100::31/128 via 2001:DB8:0:100::31
Tunnel100 created 11:51:36, expire 00:09:09
Type: dynamic, Flags: registered nhop
NBMA address: 2001:DB8:31::1
Router R3S1 receives the packet and removes the outer GRE IPv6 header and forwards the packet to the router S3R2 based on the inner header destination address 2001:db8:beef::32.
So far so good, the first ICMP query packet has reached its destination. Now it is time to jump back to the router S1R1 and check the Control Plane reaction.
Step-2c (Control Plane): In step-2b, router S1R1 forwards the ICMP query packet out of the same Tunnel100 where it was received (Hair Pinned). For the reaction to this event, S1R1 sends an NHRP redirect message to the router S2R1 to notify it, that there is a better path to the destination.
NHRP: Send Traffic Indication via Tunnel100 vrf global(0x0), <snipped>
src: 2001:DB8:0:100::11, dst: 2001:DB8:CAFE::22
(F) afn: AF_IP6(2), type: IPv6(86DD), hop: 255, ver: 1
shtl: 16(NSAP), sstl: 0(NSAP)
pktsz: 140 extoff: 124
(M) traffic code: redirect(0)
src NBMA: 2001:DB8:11::1
src protocol: 2001:DB8:0:100::11, dst protocol: 2001:DB8:CAFE::22
Contents of nhrp traffic indication packet:
60 00 00 00 00 3C 3A 3E 20 01 0D B8 CA FE 00 00
00 00 00 00 00 00 00 22 20 01 0D B8 BE EF 00 00
00 00 00 00 00 00 00 32 80 00 EE 4B 1E E2 00
Note that the destination address in NHRP redirect message is 2001:db8:cafe::22, which we used as a source of our ICMP Query. The next hop is the Tunnel interface100 address of router S2R1 as can be seen from routing table.
S1R1#sh ipv6 route 2001:db8:cafe::22
Routing entry for 2001:DB8:CAFE::/64
Known via "bgp 64501", distance 200, metric 2, type internal
Route count is 1/1, share count 0
Routing paths:
2001:DB8:0:100::21
MPLS label: nolabel
From 2001:DB8:0:100::21
Last updated 08:11:57 ago
Which is mapped to NBMA address 2001:db8::21::1 (as can be seen from NHRP cache) which is used in the GRE IPv6 header as a destination address.
S1R1#sh ipv6 nhrp 2001:db8:0:100::21/128
2001:DB8:0:100::21/128 via 2001:DB8:0:100::21
Tunnel100 created 12:31:37, expire 00:09:06
Type: dynamic, Flags: registered nhop
NBMA address: 2001:DB8:21::1
Step-2d (Control Plane): For the reaction to the NHRP redirect message, router S2R1 generates the NHRP Resolution Request message to find out the best path to the destination. The request contains both NBMA and protocol addresses of the Spoke router S2R1 as well as the destination protocol address 2001:db8:beef::32 (which is NBMA address it wants to resolve). Router S2R1 sends the packet to the router S1R1 since it is the next hop for the target (target address is the requested address).
NHRP: Send Resolution Request via Tunnel100 vrf global(0x0), packet size: 108
src: 2001:DB8:0:100::21, dst: 2001:DB8:BEEF::32
(F) afn: AF_IP6(2), type: IPv6(86DD), hop: 255, ver: 1
shtl: 16(NSAP), sstl: 0(NSAP)
pktsz: 108 extoff: 88
(M) flags: "router auth src-stable nat ", reqid: 22
src NBMA: 2001:DB8:21::1
src protocol: 2001:DB8:0:100::21, dst protocol: 2001:DB8:BEEF::32
(C-1) code: no error(0)
prefix: 128, mtu: 1452, hd_time: 600
addr_len: 0(NSAP), subaddr_len: 0(NSAP), proto_len: 0, pref: 255
Step-2e (Control Plane): Router S1R1 receives the NHRP resolution request from S2R1. It does the same decapsulation/encapsulation process than in step-2b and forwards the packet to the router S3R1.
Step-2f (Control Plane): Router S3R1 receives the NHRP resolution request message from the router S1R1. For the reaction to the message, it generates NHRP resolution Reply message that contains the information about the NBMA/protocol addresses of the original requester (router S2R1), requested destination host address (2001:db8:beef::32) and the mapping information about NBMA/protocol address for the next hop DMVPN router (S3R1). Router S3R1 sends the NHRP resolution Reply message straight to the router S2R1 since it knows the NBMA address of router S2R1 based on received the NHRP Resolution Request message.
NHRP: Send Resolution Reply via Tunnel100 vrf global(0x0), packet size: 228
src: 2001:DB8:0:100::31, dst: 2001:DB8:0:100::21
(F) afn: AF_IP6(2), type: IPv6(86DD), hop: 255, ver: 1
shtl: 16(NSAP), sstl: 0(NSAP)
pktsz: 228 extoff: 120
(M) flags: "router auth dst-stable unique src-stable nat ", reqid: 22
src NBMA: 2001:DB8:21::1
src protocol: 2001:DB8:0:100::21, dst protocol: 2001:DB8:BEEF::32
(C-1) code: no error(0)
prefix: 64, mtu: 1452, hd_time: 599
addr_len: 16(NSAP), subaddr_len: 0(NSAP), proto_len: 16, pref: 255
client NBMA: 2001:DB8:31::1
client protocol: 2001:DB8:0:100::31
Step-2g (Control Plane): Router S2R1 receives the NHRP Resolution Reply message from the router S3R1. It processes the message and saves the mapping information to the NHRP cache. From the router S2R1 perspective the Spoke-to-Spoke tunnel can now be set up.
NHRP: Receive Resolution Reply via Tunnel100 vrf global(0x0),
<packet size snipped>
(F) afn: AF_IP6(2), type: IPv6(86DD), hop: 255, ver: 1
shtl: 16(NSAP), sstl: 0(NSAP)
pktsz: 228 extoff: 120
(M) flags: "router auth dst-stable unique src-stable nat ", reqid: 22
src NBMA: 2001:DB8:21::1
src protocol: 2001:DB8:0:100::21, dst protocol: 2001:DB8:BEEF::32
(C-1) code: no error(0)
prefix: 64, mtu: 1452, hd_time: 599
addr_len: 16(NSAP), subaddr_len: 0(NSAP), proto_len: 16, pref: 255
client NBMA: 2001:DB8:31::1
client protocol: 2001:DB8:0:100::31
As we can see now from the RIB of the router S2R1, BGP learned route is overwritten with the NHRP shortcut route.
S2R1#show ipv6 route 2001:DB8:beef::/64
Routing entry for 2001:DB8:BEEF::/64
Known via "bgp 64501", distance 200, metric 2, type internal
Redistributing via ospf 1
Backup from "NHRP-IPv6 [250]"
Route count is 1/1, share count 0
Routing paths:
2001:DB8:0:100::11
MPLS label: nolabel
From 2001:DB8:0:100::11
Last updated 06:26:12 ago
2001:DB8:0:100::31, Tunnel100 [Shortcut via "NHRP-IPv6")]
From 2001:DB8:0:100::31
Last updated 00:00:09 ago
Steps 3 – 3g shows the same process from the perspective of the router S3R1. Router S1R1 forwards the ICMP reply packet to the router S2R1 and sends the NHRP Redirect message to the router S3R1. For the reaction to the NHRP redirect message the router S3R1 sends the NHRP Resolution Request to the router S2R1, which in turns replies to the message with the NHRP Resolution Reply message. Router S2R1 sends the NHRP reply message straight to the router S3R1. When router S3R1 receives the message, it saves mapping information to NHRP cache. Now both Spoke routers have all the information needed for the Spoke-to-Spoke tunnel establishment.
Both Spoke routers have a dynamic NBMA/protocol address mapping entry in NHRP cache and they are able to send data over the Spoke-to-Spoke tunnels. An example is taken from the router R2S1.
S2R1#sh ipv6 nhrp
2001:DB8:0:100::11/128 via 2001:DB8:0:100::11
Tunnel100 created 13:04:50, never expire
Type: static, Flags: used
NBMA address: 2001:DB8:11::1
2001:DB8:0:100::31/128 via 2001:DB8:0:100::31
Tunnel100 created 00:00:11, expire 00:09:49
Type: dynamic, Flags: router nhop rib
NBMA address: 2001:DB8:31::1
2001:DB8:BEEF::/64 via 2001:DB8:0:100::31
Tunnel100 created 00:00:10, expire 00:09:49
Type: dynamic, Flags: router rib nho
NBMA address: 2001:DB8:31::1
2001:DB8:CAFE::/64 via 2001:DB8:0:100::21
Tunnel100 created 00:00:11, expire 00:09:49
Type: dynamic, Flags: router unique local
NBMA address: 2001:DB8:21::1
(no-socket)
<snipped>
Step-4: The rest of the ICMP packets are sent via shortest path between Spoke routers.
Dinstar 64 Ports GSM Gateway is a state of art GSM gateway based on the latest VoIP / SIP technology, which enables smooth transit between the mobile network and VoIP networks. It supports up to 64 concurrent calls and LCD, making it a unique choice in the market for users who need a higher-capacity wireless gateway in one single box.
ReplyDelete