DMVPN Part I. Basic Operation and Configuration

Introduction

DMVPN is Cisco proprietary overlay network technology used for building dynamic, transport independent multipoint GRE tunnels over any packet switched networks (Internet, MPLS, 4G, satellite, etc.). 
DMVPN has evolved over time. In the first phase (Phase-1), dynamic tunnels are formed only between Spoke and Hub sites and Spoke-to-Spoke traffic goes through the Hub site. In the next Phase (Phase-2), the on-demand tunnel will be formed between Spoke sites belonging to the same DMVPN domain and traffic does not have to go via the Hub site. However, Phase-2 does not allow on-demand tunnels between Spoke sites belonging to the different DMPN domain, and traffic must go via the Hub that connects these two Domains. The Phase-3 phase (Phase-3) makes it possible to create an on-demand tunnel between two Spoke sites belonging to the DMVPN domain.

This article is about DMVPN Phase 3.

DMVPN Phases

Short description of protocols, components, and terminology of DMVPN.

•       NBMA address: Outer IP header address. Transport network routes packet based on NBMA address. Also, terms underlay or transport addresses can be used.

•       Protocol address: Logical Tunnel interface address. Used in the overlay network, for example, iBGP peering uses these addresses. 

•       Next Hop Server (NHS): Router on the Hub site. Stores the NBMA/protocol address mapping information received via NHRP registration messages from Next Hop Clients (NHC) to the NHRP cache.

•       Next Hop Client (NHC): Router on the Spoke site. Registers its NBMA/protocol address mapping to NHS with NHRP registration messages.

•       Multipoint GRE (mGRE):  DMVPN uses mGRE tunneling mode for building dynamic Point-to-multipoint tunnels between Hub-and-Spoke as well as between Spoke-to-Spoke sites. From the Hub router perspective, tunnels to Spoke sites are dynamic (mapping between Spoke routers NBMA and protocol addresses is dynamic). From the Spoke routers perspective tunnels to Hub routers are static (mapping between Hub routers NBMA and protocol addresses is static). Spoke-to-Spoke tunnels are dynamic and are established only when needed.

•       Next Hop Resolution Protocol (NHRP): NHRP is a Control Plane protocol of DMVPN. Spoke routers sends their address mapping information (NBMA and protocol address) to Hub routers (Next Hop Server - NHS) by using NHRP registration messages. This way Hub routers gets the address mapping information dynamically. If there is a need for the Spoke-to-Spoke tunnel, the Hub router will instruct the Spoke router by sending "NHRP redirect" message to ask tunnel end-point NBMA-protocol address mapping information with "NHRP request" message. 

•       IPsec (Optional): Since DMVPN overlay network might use untrusted transport network as an underlay, we need to protect data by authenticating data origin and verify data confidently and integrity. We use  DMVPN with IPsec Transport Mode in our examples

•       Front-door VRF (Optional): Virtual router used for WAN connection. Even though FRVRF is not a mandatory part of DMVPN design, it gives a couple of benefits. First, there is no route leaking between FRVRF and default VRF where tunnel interface stands. This prevents recursive routing, where router learns tunnel destination address via tunnel interface, causing tunnel flapping. Second, because there is no route leaking between default VRF and FRVRF, customer internal networks are inaccessible from untrusted underlay transport network. Third, FRVRF helps with Direct Internet Access (DIA) design.

Figure 1 shows the interaction between DMVPN Building blocks and protocols. In our example, the NHRP runs on the Control Plane of default/global VRF. Multipoint GRE uses NHRP for ARP-like function; for resolving and reporting the mapping information about NBMA-/protocol addresses. Since tunnel interfaces are VRF aware, they can use addresses from any vrf as a source. Data packets sent and received to/from tunnel goes through the encryption - decryption - authentication process.

Figure 1: DMVPN protocol interaction.

DMVPN OPERATION

Figure 2 shows the example topology.  There are three Sites (letter S in the name of router represents the site number) where we have two Spoke routers S2R1 and S3R1 (NHCs) and the Hub router S1R1 (NHS). Each router has a front-door VFR for the WAN connection, with a static default route pointing to the SP router of the Transport Network. Tunnel Interfaces belongs to the default VRF. Mapping information about protocol-/ NBMA address of NHS are statically defined in both NHC and stored in their NHRP cache as a type "static". There are no entries in the router S1R1 NHRP cache since registration process is still in progress. Control Plane operation for establishing Spoke-to-Hub tunnel is proactive, which means that tunnels are set up before any data is sent. Tunnel setup between Hub and Spoke routers is dynamic process from the Hub perspective but static from the Spoke perspective. Hub and Spoke tunnels under normal situation are always up.

Control Plane operation for establishing Spoke-to-Spoke tunnel is reactive, meaning that Spoke routers set up tunnels dynamically when needed and bring them down after they are not needed anymore.

In example network, the first 32 bits of the network prefix are 2001:db8.

Figure 2: Example topology.

DMVPN SPOKE TO HUB TUNNELS

a.     Both NHCs generates NHRP registration messages. The message includes information about their source and destination protocol addresses and source NBMA address.

b.    NHCs encapsulates NHRP registration messages with GRE header, where the source address is the front-door VRF WAN interface address and the destination is the statically configured NBMA address of S1R1 (NHS).

c.    Transport network routers forward NHRP messages over the Transport Network to the router S1R1.

d.    NHS S1R1 receives messages. Based on the next header field in IPv6 header, it recognizes that the messages are GRE encapsulated. It removes the outer header and notices that the original packet is destined to tunnel interface. It processes the NHRP registration messages and saves the received mapping information to the NHRP cache (Type Dynamic/Flag Registered).

e.     Now also NHS has a necessary information about protocol-/NBMA address mapping of NHCs. Tunnels between Spoke to Hub are up and running.

Figure 3: NHRP registration process of NHC

DMVPN SPOKE TO SPOKE TUNNELS

a.   Router S2R1 wants to communicate with tunnel address of the router S3R1. It has an entry in the RIB, where the network 2001:db8:0:100::/64 is connected via Tunnel 100. Therefore, it sends a packet out to the tunnel interface.

b.   Router S1R1 receives the packet from the Tunnel 100. Since the destination address belongs to the router S3R1, router S1R1 forwards the packet out of the Tunnel interface to the router S3R1.

c.   NHC S1R1 Notices that data packet is forwarded out of the same tunnel interface where it was received (Hair pinned). Therefore, it sends an NHRP redirect message to the router S2R1 to inform it that there is a better route to the destination.

Figure 4: Setting up the Spoke to Spoke tunnel – NHRP Redirect.

d.   For the reaction to the NHRP redirect message, Spoke router S2R1 sends out the "NHRP resolution Request" message. The message includes the NBMA address 2001:db8:0:21::1 of S2R1, src protocol address 2001:db8:0:100::21 of S2R1 and target protocol address 2001:db8:0:100::31 (destination network/host).

e.   Hub S1R1 receives the NHRP resolution request from the router S2R1 via Tunnel 100. It then forwards the NHRP resolution request to the router S3R1 out of the tunnel 100.

f.    Spoke S3R1 receives the NHRP resolution request. For the reaction it sends an NHRP Resolution Reply directly to the router S2R1 over the transport network using NBMA address learned from NHRP resolution request message. The NHRP resolution reply messages include NBMA address as well as protocol address of S3R1. 

Figure 5 Setting up the Spoke to Spoke tunnels – Resolution Reply

g.   The Spoke S2R1 receives an NHRP resolution reply from the spoke router S3R1. It adds the information received via message to its NHRP cache. Since mapping information has been learned dynamically through the Resolution Request/Reply process, the mapping entry is marked as “Dynamic”.

h.   We have so far skipped learning process from the router S3R1 perspective. When the router  S3R1 got the first data packet from the router S2R1, it replies to it. This starts the learning process, where the Hub router S1R1 forwards data sent by the router S3R1 to the router S2R1 out of the Tunnel 100. Since the router S1R1 sends the message out of the same interface where it was received, the router S1R1 sends an NHRP redirect message to the router S3R1, which, as a reaction to the message, sends an NHRP resolution request to the router S2R1 out of the Tunnel 100. The router S1R1 forwards the message to S2R1, which in turn sends the NHRP Resolution Reply message straight to the Spoke router S3R1. Now also Spoke router S3R1 has an entry about protocol-/NMBA address mappings of S2R1.

Now we have all the tunnels up and running.

Figure 6: Spoke-to-Spoke Tunnels.

Note! Dynamically learned mapping entries (Spoke to Spoke) in NHRP cache are removed after period based on NHRP holdtime. Recommended time is 600 seconds (10 minutes). If the tunnel is still being used within 2 minutes before timer expiration, new NHRP request will be sent and the timer is started again.  Other NHRP related timer is NHRP timeout, which takes care that NHRP registration messages are sent from Spoke to Hub to maintain connectivity. The default time is holdtime divided by three. 

PROTECTING DATA WITH IPSEC

Because DMVPN can be run over any Packet Switched Transport Network including the Internet, it is extremely important to protect data sent over tunnels. We use DMVPN with IPSec Transport Mode in our example, where the original packet plus GRE flags are encrypted and the ESP header, GRE flags and original packet are authenticated. The GRE address header is neither encrypted nor authenticated.

Components:

•     IKEv2 Keyring: We use Pre-shared keys, where keys are defined locally and are never transported over the network. There could be a different key for each peer, but we use the same for all peers. Key association of remote peer is based on its address.

•     IKEv2 Profile: Here we define authentication mode (pre-shared) for connection request received from the remote peer as well as connection requests sent to the remote peer. We also define the keyring where keys are stored. We also associate IKEv2 Profile to correct VRF, in our case to front-door VRF.

•     IPSec Transform Set: Define the Encapsulation Security Protocol (ESP) algorithms for both data encryption and authentication.

•     IPSec Profile: We have defined that peer authentication uses pre-shared keys from the keyring. We have also defined encryption and authentication algorithm. Now we map these together. The last thing to do is associate IPsec profile to the tunnel interface.

Figure 7: IPsec Tunnel Protection.

ROUTING BETWEEN SITES

Now all the Tunnels are up and Data Plane is protected with IPsec. Then it is time to set up routing between sites. Site 2 has internal router S2R2 which has network 2001:db8:0:cafe::/64 connected to its loopback interface. Site 3 has internal router S3R2 which has network 2001:db8:0:beef::/64 connected to its loopback interface. OSPFv3 is running on LAN as IGP protocol.

BGP peering is established between Spoke and Hub routers but not between Spoke routers. BGP peering is iBGP and S1R1 is Route-Reflector while Spoke routers S2R1 and S3R1 are Route-Reflector-Clients. Both Spoke routers redistribute BGP learned routes to OSPF. OSPF learned routes are advertised by using network clauses under BGP process.

Figure 8: Routing over DMVPN tunnels.

COMPLEXITY OF DMVPN

Figure 9 illustrates the Complexity of DMVPN. To be able to form dynamic mGRE tunnels we have Next Hop Resolution Protocol (NHRP), which is used for resolving address mapping of NBMA and transport addresses. NHRP can also modify routes in RIB by overwriting routes learned from other routing sources with NHRP shortcut routes.

We also have BGP process, which uses addresses learned via NHRP process for internal BGP peering. Both NHRP and BGP messages are GRE encapsulated with a source address of front-door VRF interface and with an NBMA destination address of remote site learned with NHRP. So if we have a problem with NHRP it affects to BGP peering.

We also use OSPF inside the LAN. Routes learned via iBGP are redistributed to OSPF. For securing data sent over tunnel interfaces, we use IPsec Transport mode for encryption and authentication, meaning that also both BGP messages and NHRP messages are encrypted and authenticated.

All above protocols are running on default/global VRF. Then we have front-door VRF. There is no route leaking between default VRF and front-door VRF, so it looks like they are an independent component, that does not have any relationship between each other. However, that’s not true, mGRE tunnels in global VRF use both Layer 2 and Layer 3 addresses from the WAN interface of front-door VRF. Also, IKEv2 Profile is associated to front-door VRF to protect Data Plane traffic. 

Figure 9: DMVPN complexity.

DMVPN BASIC CONFIGURATIONS

Step-1: Configure front-door VRF.

•     Create front-door VRF (named VRVRF-INET) and enable afi ipv6. Attach WAN interface Gi2 to front-door VRF and assign the IPv6 address to the interface.

•     Configure static default route towards transport network, remember we do not want to exchange routing information with a provider (transport independence).

vrf definition VRVRF-INET

!

 address-family ipv6

!

interface GigabitEthernet2

 vrf forwarding VRVRF-INET

 ipv6 address 2001:DB8:21::1/64

!

ipv6 route vrf VRVRF-INET ::/0 2001:DB8:21::100

Step-2: Configure Tunnel interface and NHRP (all routers)

•     Configure Tunnel 100 and assign local IPv6 addresses (protocol address). Note that Link-Local address manual configuration is mandatory if the device has more than one DMVPN tunnels.

•     Set the largest IP packet size (in our example MTU 1400) that can be sent over the link without fragmentation (TCP and IP headers are included). Note, since fragmentation is not supported in IPv6, also path MTU is needed.

•     Set the largest amount of data in bytes (in our example TCP MSS 1360) that the router can receive in a single TCP datagram. Routers send this information in TCP SYN header during the tcp three-way handshake process.

•     (Optional) Define tunnel key, it has to be the same for both Hub and Spoke routers.

•     Associate Tunnel to vrf front-door VRF and set tunnel mode to mGRE IPv6.

•     Enable NHRP on the interface by defining NHRP network-id.

interface Tunnel100

 ipv6 address FE80:100::31 link-local

 ipv6 address 2001:DB8:0:100::31/64

 ipv6 mtu 1400

 ipv6 tcp adjust-mss 1360

 ipv6 nhrp network-id 100

 tunnel source GigabitEthernet2

 tunnel mode gre multipoint ipv6

 tunnel key 100

 tunnel path-mtu-discovery

 tunnel vrf VRVRF-INET

Step-3: Configure NHS NHRP options

•        Under the Tunnel 100 interface, enable Multicast support for Multicast traffic and routing protocols that use Multicast. Note! This is enabled by default with IOS XE Denali 16.3.

•        Instruct router to send an NHRP Redirect message if traffic received from one Spoke is forwarded out of the same interface where it was received.

Interface Tunnel100

 ipv6 nhrp map multicast dynamic

 ipv6 nhrp redirect

Step-4: Configure NHC NHRP options

•        Map NHS protocol address (tunnel interface address) to NBMA and enable Multicast.

•        Enable NHRP shortcut. When enabled NHRP check every egress packet and check if there is NHRP cache entry for the destination. If the cache entry is found, the packet is switched based on the cache information. Note! This is enabled by default with IOS XE Denali 16.3.

Interface Tunnel100

 ipv6 nhrp nhs 2001:DB8:0:100::11 nbma 2001:DB8:11::1 multicast

 ipv6 nhrp shortcut

DMVPN IPSEC CONFIGURATIONS

Step-1: Configure IKEv2 Keyring

•        Configure keyring named KEYRING-INET.

•        We use same pre-shared key “Cisco123” for all peers.

crypto ikev2 keyring KEYRING-INET

 peer ALL-DMVPN-HOSTS

  address ::/0

  pre-shared-key Cisco123

Step-2: Configure IKEv2 profile

•        Configure IKEv2 crypto profile IKE-PROFILE-INET.

•        Configure vrf association with front-door VRF.

•        Configure both local and remote peer to use pre-shared authentication.

•        All peers’ (::/0) uses a pre-shared key from IKEv2 keyring defined in step-1.

crypto ikev2 profile IKE-PROFILE-INET

 match fvrf VRVRF-INET

 match identity remote address ::/0

 authentication local pre-share

 authentication remote pre-share

 keyring local KEYRING-INET

Step-3: Configure IPsec transform set

•        Configure IPsec transform-set DMVPN-AES256

•        Set encryption and authentication algorithms.

•        Set tunnel mode to transport.

crypto ipsec transform-set DMVPN-AES256 esp-aes 256 esp-sha-hmac

 mode transport

Step-4: Configure IPsec profile

•        Configure IPsec profile DMVPN-IPSEC-INET

•        Combine IKEv2 profile defined in Step-2 and IPsec profile defined in Step-3 to IPsec profile.

crypto ipsec profile DMVPN-IPSEC-INET

 set transform-set DMVPN-AES256

 set ikev2-profile IKE-PROFILE-INET

Step-5: Secure data sent over Tunnel interface

•        Attach IPsec profile to Tunnel Interface.

Int tunn 100

 tunnel protection ipsec profile DMVPN-IPSEC-INET

DMVPN TUNNEL VERIFICATION

Step-1: Verify Tunnel status on Spoke S2R1

S2R1 is Spoke router and it has Static Tunnel to Hub router and Dynamic tunnel to Spoke S3R1 (We have generated traffic between Spoke routers by pinging between S2R2 and S3R2 loopback interfaces).

S2R1#sh dmvpn | beg Interface

Interface: Tunnel100, IPv6 NHRP Details

Type:Spoke, Total NBMA Peers (v4/v6): 2

    1.Peer NBMA Address: 2001:DB8:11::1

        Tunnel IPv6 Address: 2001:DB8:0:100::11

        IPv6 Target Network: 2001:DB8:0:100::11/128

        # Ent: 1, Status: UP, UpDn Time: 02:09:04, Cache Attrib: S

    2.Peer NBMA Address: 2001:DB8:31::1

        Tunnel IPv6 Address: 2001:DB8:0:100::31

        IPv6 Target Network: 2001:DB8:0:100::31/128

        # Ent: 1, Status: UP, UpDn Time: 00:00:47, Cache Attrib: D

Step-2: Verify Tunnel status on Hub S1R1

From Hub point of view, both Tunnels to Spoke routers are Dynamic.

S1R1#sh dmvpn | beg Interface

Interface: Tunnel100, IPv6 NHRP Details

Type:Hub, Total NBMA Peers (v4/v6): 2

    1.Peer NBMA Address: 2001:DB8:21::1

        Tunnel IPv6 Address: 2001:DB8:0:100::21

        IPv6 Target Network: 2001:DB8:0:100::21/128

        # Ent: 1, Status: UP, UpDn Time: 02:39:14, Cache Attrib: D

    2.Peer NBMA Address: 2001:DB8:31::1

        Tunnel IPv6 Address: 2001:DB8:0:100::31

        IPv6 Target Network: 2001:DB8:0:100::31/128

        # Ent: 1, Status: UP, UpDn Time: 02:38:51, Cache Attrib: D

Step-3: Verify NHRP cache

Here we can see the mapping information in S1R1 NHRP cache. There are two entries, one from the S2R1 and the other one from the S3R1. Each entry has mapping information about protocol-/NBMA addresses as well as Type/Flag, which is Dynamic/registered.

S1R1#show ipv6 nhrp brief

<snipped>

Intf     NextHop Address                                    NBMA Address

         Target Network                              T/Flag

-------- ------------------------------------------- ------ ----------------

Tu100    2001:DB8:0:100::21                                 2001:DB8:21::1

         2001:DB8:0:100::21/128                      D/r   

Tu100    2001:DB8:0:100::31                                 2001:DB8:31::1

         2001:DB8:0:100::31/128                      D/r

Step-4: Verify NHRP cache: option 2

Here we can see little bit detailed information, such as expiration time of NHRP cache entry.

S1R1#sh ipv6 nhrp 2001:DB8:0:100::21/128 detail         

2001:DB8:0:100::21/128 via 2001:DB8:0:100::21

   Tunnel100 created 02:52:47, expire 00:07:56

   Type: dynamic, Flags: registered nhop

   NBMA address: 2001:DB8:21::1

   Preference: 255

IPSEC VERIFICATION

Step-1: Verify IKEv2 profile

Here we can see that we use a pre-shared key from Keyring KEYRING-INET for both local and remote peers. Lifetime is default 86400seconds (24h).

S2R1#show crypto ikev2 profile | i IKE|Loc|Rem|Key| Life     

IKEv2 profile: IKE-PROFILE-INET

  Local address/interface: none

 Local identity: none

 Remote identity: none

 Local authentication method: pre-share

 Remote authentication method(s): pre-share

 Keyring: KEYRING-INET

 Lifetime: 86400 seconds

Step-2: Verify IPsec transport-set

We have transport-set DMVPN-AES256.

S2R1#show crypto ipsec transform-set

<default Transformset removed>

Transform set DMVPN-AES256: { esp-256-aes esp-sha-hmac  }

   will negotiate = { Transport,  },

Step-3: Verify IPsec profile

IKE profile and transform-set defined in steps 1 and 2 are attached to IPsec profile. This profile is attached to the Tunnel interface.

S2R1#show crypto ipsec profile

IPSEC profile DMVPN-IPSEC-INET

        IKEv2 Profile: IKE-PROFILE-INET

        Security association lifetime: 4608000 kilobytes/3600 seconds

        Responder-Only (Y/N): N

        PFS (Y/N): N

        Mixed-mode : Disabled

        Transform sets={

                DMVPN-AES256:  { esp-256-aes esp-sha-hmac  } ,

        }

<default profile removed>

Note! Output from command "show crypto ipsec sa" also give usefull informataion.

Step-4: Verify Tunnel protection

Here we can see that traffic sent between Spoke-to-Hub and Spoke-to-Spoke Tunnels is encrypted.

S2R1#show dmvpn detail | beg Crypto

Crypto Session Details:

--------------------------------------------------------------------------------

Interface: Tunnel100

Session: [0x7F27D791CB78]

  Session ID: 2 

  IKEv2 SA: local 2001:DB8:21::1/500

          remote 2001:DB8:11::1/500 Active

          Capabilities:(none) connid:1 lifetime:21:14:59

  Crypto Session Status: UP-ACTIVE     

  fvrf: (none), Phase1_id: 2001:DB8:11::1

  IPSEC FLOW: permit 47 host 2001:DB8:21::1 host 2001:DB8:11::1

        Active SAs: 2, origin: crypto map

        Inbound:  #pkts dec'ed 54 drop 0 life (KB/Sec) 4607998/2979

        Outbound: #pkts enc'ed 59 drop 0 life (KB/Sec) 4607999/2979

   Outbound SPI : 0x   209DB, transform : esp-256-aes esp-sha-hmac

    Socket State: Open

Interface: Tunnel100

Session: [0x7F27D791C9F8]

  Session ID: 7 

  IKEv2 SA: local 2001:DB8:21::1/500

          remote 2001:DB8:31::1/500 Active

          Capabilities:(none) connid:2 lifetime:23:59:42

  Crypto Session Status: UP-ACTIVE    

  fvrf: (none), Phase1_id: 2001:DB8:31::1

  IPSEC FLOW: permit 47 host 2001:DB8:21::1 host 2001:DB8:31::1

        Active SAs: 2, origin: crypto map

        Inbound:  #pkts dec'ed 5 drop 0 life (KB/Sec) 4607999/3581

        Outbound: #pkts enc'ed 4 drop 0 life (KB/Sec) 4607999/3581

   Outbound SPI : 0x302B3735, transform : esp-256-aes esp-sha-hmac

    Socket State: Open

 

BGP CONFIGURATION

Step-1: BGP and OSPF configuration on Spoke routers

BGP peering is done between Spoke and Hub routers tunnel interfaces. Note that BGP does not distribute an iBGP-learned route to IGP without additional BGP redistribute-internal command under address-family IPv6 configuration. BGP redistributed routes from RIB, that are learned via OPSF. The only Spoke specific routing configurations is related to prefix-list OSPF-routes, which defines site internal networks.

Spoke S2R1

router bgp 64501

 bgp log-neighbor-changes

 neighbor HUB peer-group

 neighbor HUB remote-as 64501

 neighbor HUB timers 20 60

 neighbor 2001:DB8:0:100::11 peer-group HUB

 !

 address-family ipv6

  redistribute ospf 1 route-map OSPF-to-BGP

  bgp redistribute-internal

  neighbor HUB send-community

  neighbor HUB soft-reconfiguration inbound

  neighbor 2001:DB8:0:100::11 activate

 exit-address-family

!

ipv6 router ospf 1

 redistribute bgp 64501 metric 10

!

ipv6 prefix-list OSPF-routes seq 10 permit 2001:DB8:CAFE::/64

!

route-map OSPF-to-BGP permit 5

 match ipv6 address prefix-list OSPF-routes

Step-2: BGP configuration on HUB routers

BGP uses dynamic peering, therefore adding a new Spoke site does not require configuration in Hub site. Note that Hub router is a BGP Route-Reflector. Note! for additional security we could also use password protection between peers.

S1R1#sh run | sec bgp

router bgp 64501

 bgp log-neighbor-changes

 bgp listen range 2001:DB8:0:100::/64 peer-group SPOKE-ROUTERS

 bgp listen limit 3

 neighbor SPOKE-ROUTERS peer-group

 neighbor SPOKE-ROUTERS remote-as 64501

 neighbor SPOKE-ROUTERS timers 20 60

 !

 address-family ipv6

  neighbor SPOKE-ROUTERS activate

  neighbor SPOKE-ROUTERS send-community

  neighbor SPOKE-ROUTERS route-reflector-client

  neighbor SPOKE-ROUTERS next-hop-self all

  neighbor SPOKE-ROUTERS soft-reconfiguration inbound

 exit-address-family

ROUTING VERIFICATION

Step-1: Generate traffic from spoke S2R2 to 2001:db8:beef::32

We need to generate traffic between Spoke sites start Spoke to Spoke tunnel establishment process. We use ping from S2R2 to S3R2.

S2R2#ping ipv6 2001:db8:beef::32 source 2001:db8:cafe::22

<snipped>

Step-2: Check BGP table for 2001:db8:beef:.32

As now can be seen from the router S3R1, route to network 2001:db8:beef::/64 has been learned from S1R1 via BGP.

S2R1#sh bgp ipv6 unicast 2001:db8:beef::32/64

BGP routing table entry for 2001:DB8:BEEF::/64, version 13

Paths: (1 available, best #1, table default)

  Not advertised to any peer

  Refresh Epoch 1

  Local, (received & used)

    2001:DB8:0:100::11 from 2001:DB8:0:100::11 (10.255.0.4)

      Origin incomplete, metric 2, localpref 100, valid, internal, best

      Originator: 10.255.0.7, Cluster list: 10.255.0.4

      rx pathid: 0, tx pathid: 0x0

Step-3:Check RIB for 2001:db8:beef:.32

But the shortcut route via better path (straight to Spoke S3R1) has been installed to RIB instead of BGP learned route.

S2R1#sh ipv6 route 2001:db8:beef::32

Routing entry for 2001:DB8:BEEF::/64

  Known via "bgp 64501", distance 200, metric 2, type internal

  Redistributing via ospf 1

  Backup from "NHRP-IPv6 [250]"

  Route count is 1/1, share count 0

  Routing paths:

    2001:DB8:0:100::11

      MPLS label: nolabel

      From 2001:DB8:0:100::11

      Last updated 00:59:54 ago

    2001:DB8:0:100::31, Tunnel100 [Shortcut via "NHRP-IPv6")]

      From 2001:DB8:0:100::31

      Last updated 00:01:59 ago

Step-4: DMVPN Tunnel verification

We can also see that new DMVPN tunnel has been set up between Spoke routers. For our target network 2001:db8:beef::/64 has NHRP cache attributes D  and T2. T2 means that the next-hop to the target network learned from the BGP is overwritten with an NHRP shortcut route.

S2R1#sh dmvpn

Legend: Attrb --> S - Static, D - Dynamic, I - Incomplete

        N - NATed, L - Local, X - No Socket

        T1 - Route Installed, T2 - Nexthop-override

        C - CTS Capable, I2 - Temporary

        # Ent --> Number of NHRP entries with same NBMA peer

        NHS Status: E --> Expecting Replies, R --> Responding, W --> Waiting

        UpDn Time --> Up or Down Time for a Tunnel

======================================================================

Interface: Tunnel100, IPv6 NHRP Details

Type:Spoke, Total NBMA Peers (v4/v6): 2

    1.Peer NBMA Address: 2001:DB8:11::1

        Tunnel IPv6 Address: 2001:DB8:0:100::11

        IPv6 Target Network: 2001:DB8:0:100::11/128

        # Ent: 1, Status: UP, UpDn Time: 04:59:59, Cache Attrib: S

    2.Peer NBMA Address: 2001:DB8:31::1

        Tunnel IPv6 Address: 2001:DB8:0:100::31

        IPv6 Target Network: 2001:DB8:0:100::31/128

        # Ent: 2, Status: UP, UpDn Time: 00:04:07, Cache Attrib: DT1

    3.Peer NBMA Address: 2001:DB8:31::1

        Tunnel IPv6 Address: 2001:DB8:0:100::31

        IPv6 Target Network: 2001:DB8:BEEF::/64

        # Ent: 0, Status: UP, UpDn Time: 00:04:07, Cache Attrib: DT2

Step-5: NHRP cache verification

There are three flags attached to entry 2001:db8:beef::/64

router flag: This flag indicates that mapping entry is received from the remote router that advertises network to DMVPN domain.

rib flag: This means that network has a route in a Routing Information Table, but it is overwritten by NHRP and routing decision is made based on shortcut route (entry is marked “H” code).

nho flag: This flag means that NHRP entry route is used instead of other route learned from other routing protocol.

S2R1#sh ipv6 nhrp 2001:DB8:BEEF::/64

2001:DB8:BEEF::/64 via 2001:DB8:0:100::31

   Tunnel100 created 00:09:35, expire 00:00:24

   Type: dynamic, Flags: router rib nho

   NBMA address: 2001:DB8:31::1

Here we can see the “H” code on S2R1 RIB

S2R1#sh ipv6 route     

<snipped>

H   2001:DB8:0:100::31/128 [250/255]

     via 2001:DB8:0:100::31, Tunnel100

There are a lot more to consider when implementing DMVPN in a production environment, such as Hub redundancy, dual homing in Spoke sites, security not forgetting implementation strategy. But I will come back to these in my future posts.

Reference:

Cisco Intelligent WAN (iWAN) – ISBN-10: 1-58714-463-8

Brad Edgeworth, Jean Marc Barozet, David Prall, Anthony Lockhart, Nir Ben-Dvora

Edited February 3.3.2018 | Toni Pasanen CCIE#28158