Introduction
In this DMVPN article, I am going to implement Direct Internet Access (DIA) in the Remote Site. In initial situation, both edge routers in the Central site advertises the default route over the DMVPN tunnels to the router S2R1 with iBGP. Router S1R1 advertises the default route with a better Local Preference value so that one is installed in the routing table of router S2R1.
Figure 1. Default route from the Central site over the DMVPN WAN.
Next, we change the architecture from Centralized Internet Access to Direct Internet Access (DIA). To be able to do that, we need to do the following things:
1. Install the static default route pointing to the Internet connection to the global RIB:
The static route is installed in the rib only if two out of three conditions are met:
- The NBMA IP address 10.12.112.12 of Central site Internet WAN interface is reachable over the Internet
- The line-protocol of S2R1 Tunnel interface12 is up.
- Some public IP- address on the Internet is reachable, e.g. a public DNS server.
2. Network Address Translation - NAT:
Internal users’ traffic to the Internet uses the IP address of router S2R1 interface Gi3 as a source IP addresses.
3. Routing return traffic based on global RIB – Policy Based Routing (PBR):
Traffic from default-VRF to the Internet passes via INET vrf, but the INET vrf does not know how return packets should be routed since it does not have any entries in routing table related to default-VRF networks. That is why we need to use a Policy-Based Routing (PBR).
4. Traffic filtering with Zone-Based Fire Wall
As the last step, we are using a stateful firewall for protecting the remote site from untrusted Internet. I am going to use Zone-Based FW included in IOS.
So let's start.
So let's start.
1. Install the static default route pointing to the Internet to the global RIB:
Step 1 - IP SLA reachability test:
- Configure IP SLA 122 which tests the reachability of the IP address 10.12.112.12.
- Configure IP SLA 123 which tests the reachability of the IP address 10.9.2.11 (emulates the public, always reachable IP address). In real life implementation, the anycast address of F root DNS can be used. Address are described in section "Step-4: Master tracking object"
- Set the interval of ICMP packets to 10 seconds.
- Set the upper threshold to 1000ms. This means that if we do not get the response within 1000ms, the router could perform predefined functions like trigger an SNMP trap or send a syslog message.
- Set the timeout value to 3000ms. This means that if we do not get any response within 3000ms the IP SLA is violated and IP SLA state is changed from up to down.
- Assign the IP SLA to vrf INET and set it’s start time “now” and lifetime “forever”.
Figure 2. IP SLA for reliability.
ip sla 122
icmp-echo 10.12.112.12 source-interface GigabitEthernet3
vrf INET
threshold 1000
timeout 3000
ip sla schedule 122 life forever start-time now
!
ip sla 123
icmp-echo 10.9.2.11 source-interface GigabitEthernet3
vrf INET
threshold 1000
timeout 3000
ip sla schedule 123 life forever start-time now
Step 2 - Track the line protocol of the interface Tunnel12:
- Configure tracking object 121 (first child object) to track the line-protocol state of interface Tunnel12. Note that command “if-state nhrp” is required because it activates the tunnel health-monitoring; if the NHRP Next Hop Server is reachable, the line protocol of the tunnel is up, other vices it is down. The reason for tracking the interface Tunnel12 is that it goes over the Internet.
Figure 3. Tracking the state of the line protocol of the interface Tunnel12.
interface Tunnel12
if-state nhrp
!
track 121 interface Tunnel12 line-protocol
Step-3: Tracking Objects for IP SLA and master object to track child objects
- Configure tracking object 122 (second child object) for track the status of IP SLA 122.
- Configure tracking object 123 (third child object) for track the status of IP SLA 123.
- Configure the master tracking object 12 that monitors the status of child objects 121, 122 and 123.
- Set the master tracking objects percentage-based threshold values to 40 and 60. This means that if 40% out of tracked child objects are down, the state of the master object changes from up to down. When 60% out of tracked child objects are up, the state of the master object changes from down to up. In our case, if one out of three of child objects goes down, we still have 66% up. If two out of three child objects goes down, then we only have 33% up and the master tracking object change from UP to DOWN.
- Configure 5 seconds delay from changing the state of the master object from DOWN to UP, so the other Control Plane protocols have some additional recovery time.
Figure 4. Tracking objects
track 12 list threshold percentage
object 121
object 122
object 123
threshold percentage down 40 up 60
delay up 15
track 121 interface Tunnel12 line-protocol
track 122 ip sla 122 reachability
track 123 ip sla 123 reachability
!
ip sla 122
icmp-echo 10.12.112.12 source-interface GigabitEthernet3
vrf INET
threshold 1000
timeout 3000
ip sla schedule 122 life forever start-time now
!
ip sla 123
icmp-echo 10.9.2.11 source-interface GigabitEthernet3
vrf INET
threshold 1000
timeout 3000
ip sla schedule 123 life forever start-time now
Step-4: Master tracking object.
Because the router S2R1 interface Gi3 gets the IP address and the next hop address from the DHCP, the next-hop tracking is not an option. As a workaround, we first configure the static host route, which we route to the interface Gi3 with the next hop learned from the DHCP. Then we configure the static default route, which we also route to the interface Gi3, but with the next hop of static host route and with tracking of object 12.
For the host route, we need some IP host address, which internal users does not use and which is reachable 24/7/365. Probably the most reliable servers on the Internet are root DNS servers and they are only used by SP DNS servers, not by users. There are 13 DNS root servers (from A to M) at the time of writing, so which one we should use? Since I am living in Finland in the city of Helsinki, I first check what we have available locally. There are three DNS root servers in Helsinki:
F root DNS server: operator ISC, Anycast IPv4 address: 192.5.5.241, servers in 140 sites
I root DNS server: operator Netnod, Anycast address: 192.36.148.17, servers in 59 sites
ISC has most sites, so I choose the F root DNS (anycast IP 192.5.5.241) and we will use that in our host route.
Figure 5. Default route to RIB.
ip route 192.5.5.241 255.255.255.255 GigabitEthernet3 dhcp 10
ip route 0.0.0.0 0.0.0.0 GigabitEthernet3 192.5.5.241 10 track 12
Default static route verification
We can see that the track state from default static route is up
S2R1#sh ip route track
ip route 0.0.0.0 0.0.0.0 GigabitEthernet3 192.5.5.241 10 track 12 state is [up]
We can also see that the static route is installed into the RIB instead of BGP learned default route.
S2R1#sh ip route
<snipped>
Gateway of last resort is 192.5.5.241 to network 0.0.0.0
S* 0.0.0.0/0 [10/0] via 192.5.5.241, GigabitEthernet3
Also, all tracking objects are UP.
S2R1#show track 12
Track 12
List threshold percentage
Threshold Percentage is Up (100%)
2 changes, last change 00:30:56
object 121 Up (33%)
object 122 Up (33%)
object 123 Up (33%)
Threshold percentage down 40% up 60%
Delay up 15 secs
Tracked by:
Static IP Routing 0
Testing default static route process
Now I am going to test this set up by shutting down the Tunnel12 Interface of the Central site router S1R2 as well as the Loopback 9211 (10.9.2.11) of router R112. After some period, the line protocol of router S2R1 interface Tunnel12 should go down. For the reaction to this event, S2R1 should change the tracking object 121 from UP to DOWN. Also, the IP SLA 123 should change from UP to DOWN, which in turn change the tracking object 123 state to DOWN. These two events should change the state of the master tracking object 12 from UP to DOWN.
*Mar 22 22:43:44.067: %LINEPROTO-5-UPDOWN: Line protocol on Interface Tunnel12, changed state to down
*Mar 22 22:43:44.069: %TRACK-6-STATE: 121 interface Tu12 line-protocol Up -> Down
*Mar 22 22:46:21.708: %TRACK-6-STATE: 123 ip sla 123 reachability Up -> Down
*Mar 22 22:46:21.899: %TRACK-6-STATE: 12 list threshold percentage Up -> Down
As we can now see, the threshold percentage is 33% which is under the threshold limit 40%. For the reaction to this event, the master tracking object 12 goes DOWN.
S2R1#sh track 12
Track 12
List threshold percentage
Threshold Percentage is Down (33%)
3 changes, last change 00:01:17
object 121 Down (0%)
object 122 Up (33%)
object 123 Down (0%)
Threshold percentage down 40% up 60%
Delay up 15 secs
Tracked by:
Static IP Routing 0
Now the default static route is removed from the RIB and BGP route learned over the DMVPN tunnel over MPLS network is used.
S2R1#sh ip route
<snipped>
B* 0.0.0.0/0 [19/1] via 10.11.21.11, 00:02:19
Now I am going to bring up both interface Tunnel12 on router S1R2 and the loopback interface 9211 on Internet router R112. We can see that the static route comes back to RIB.
*Mar 22 22:50:19.747: %BGP-5-ADJCHANGE: neighbor 10.12.21.12 Up
*Mar 22 22:50:21.730: %TRACK-6-STATE: 123 ip sla 123 reachability Down -> Up
*Mar 22 22:50:30.019: %TRACK-6-STATE: 12 list threshold percentage Down -> Up
*Mar 22 22:50:30.020: RT: updating static 0.0.0.0/0 (0x0) :
via 10.21.112.112 Gi3 0 1048578
*Mar 22 22:50:30.020: RT: closer admin distance for 0.0.0.0, flushing 1 routes
*Mar 22 22:50:30.020: RT: add 0.0.0.0/0 via 10.21.112.112, static metric [10/0]
*Mar 22 22:50:30.020: RT: default path is now 0.0.0.0 via 10.21.112.112
Network Address Translation
For the next step, we will configure the Network Address Translation (NAT) for the local user’s traffic to the Internet. We are going to use vrf INET WAN Interface Gi3 address as a source address for the traffic from the 172.16.21.0/24 to the Internet.
Figure 6. NAT
Step-1: Define the NAT policy.
We are going to use NAT for traffic from the Remote site LAN 172.16.21.0/24 to any destination on Internet sent over the Interface Gi3.
- Configure the access-list, which matches to the traffic sent from the source network 172.16.21.0/24 to any destination.
- Configure the route-map to match access-list and the Interface G3.
ip access-list extended ACL-FOR-NAT
permit ip 172.16.21.0 0.0.0.255 any
!
route-map RM-FOR-INET permit 10
match ip address ACL-FOR-NAT
match interface Gi3
Step-2: Define "NAT Inside" and "NAT Outside" interfaces and enable NAT for traffic defined in Step-1. Note that in our example we are going to use the interface Loopback 21 of the router S2R1 as an Inside interface.
interface Loopback21
ip address 172.16.21.21 255.255.255.0
ip nat inside
!
interface GigabitEthernet3
vrf forwarding INET
ip address dhcp
ip nat outside
!
ip nat inside source route-map RM-FOR-INET interface GigabitEthernet 3 overload
Traffic steering from the Internet back to the remote site
Traffic from LAN to the Internet is now ok, but what about the return traffic? INET vrf has no idea how to route return packet back to network 172.16.21.0/24 which is routed in global RIB. We are going to define a Policy-Based Routing (PBR) for return traffic. When return packets with the destination address 172.16.21.21 arrives in Interface Gi3, router S2R1 make a routing decision based on the default vrf RIB and packet is routed to the destination.
Figure 7.
Step-1: Define networks that are routed based on global RIB
- Configure an access-list for site internal networks. We use the same extended access-list ACL-FOR-NAT that we made in our previous example.
- Define the route-map that match to the extended access-list ACL-FOR-NAT. Use “set global” command under the route-map statement to route traffic based on the global routing table.
ip access-list extended ACL-FOR-NAT
permit ip 172.16.21.0 0.0.0.255 any
!
route-map RM-FROM-INET2LAN permit 10
match ip address ACL-FOR-NAT
set global
Step-2: Enable PBR on the Internet Interface. Note that we are also going to implement Zone-Based Firewall (ZBFW) on router S2R1. Since the ZBFW is a stateful firewall, it might generate a TCP reset for a session that is expired or for traffic that is denied. Packets that goes through the router is policy routed by default, but for locally generated traffic by ZBFW we need to enable PBR also globally.
interface GigabitEthernet3
vrf forwarding INET
ip address dhcp
ip nat outside
ip policy route-map RM-FROM-INET2LAN
!
ip local policy route-map RM-FROM-INET2LAN
Now we are going to test if we are able to telnet to the Internet router.
S2R1#telnet 10.9.2.11 /source-interface loopback 21
Trying 10.9.2.11 ... Open
User Access Verification
Password: cisco
Internet>
As we can see, Telnet works just fine and we can see from the NAT table that also addresses translation works.
S2R1#sh ip nat translations
Pro Inside global Inside local Outside local Outside global
tcp 10.21.112.21:4501 172.16.21.21:26810 10.9.2.11:23 10.9.2.11:23
Total number of translations: 1
Zone-Based Fire Wall
Now we are going to configure Zone-Based Firewall on router S2R1 to protect a Remote site from the untrusted Internet.
Step-1: Security zones “Default” and “Outside”.
- Create security zones “Default”. The default zone is a system-level zone where all unassigned interfaces are automatically attached. Despite the zone name “default zone” it does not exist by default and we have to configure it. The use of Default zone makes it possible to add a PfRv3 (SD-WAN technology) to the network to give more intelligence for the link selections process. PfRv3 uses dynamic mGRE auto-tunneling between Border routers and those auto-tunnels cannot be assigned to any security zone.
- Create security zone “OUTSIDE” and assign interface Gi3 to it.
Figure 8.
zone security default
description ** Default Zone for all unassigned Interfaces **
zone security OUTSIDE
description ** Internet Interface **
!
interface GigabitEthernet3
vrf forwarding INET
ip address dhcp
ip nat outside
zone-member security OUTSIDE
ip policy route-map RM-FROM-INET2LAN
Step-2: Rules for the internal users' Internet traffic.
- Configure an inspection class-map CM-IN2OUT that allows TCP, UDP and ICMP packets. Note that we need to use “match-any” option to match one of the allowed protocol.
- Configure an inspection Policy-map PM-IN2OUT and assign the inspection class-map CM-IN2OUT to it. ZBFW keeps session/state information for all protocols traffic defined in this class-map CM-IN2OUT and allows return packets.
class-map type inspect match-any CM-IN2OUT
match protocol tcp
match protocol udp
match protocol icmp
!
policy-map type inspect PM-IN2OUT
class type inspect CM-IN2OUT
inspect
Step-3: Zone pairs
- Define the zone pair between zone Default and zone OUTSIDE
zone-pair security DEFAULT-TO-OUTSIDE source default destination OUTSIDE
service-policy type inspect PM-IN2OUT
!
interface GigabitEthernet3
vrf forwarding INET
ip address dhcp
ip nat outside
zone-member security OUTSIDE
ip policy route-map RM-FROM-INET2LAN
Step-4: Define the access-lists for the traffic to and from the self-zone.
In our example, there are several Control Plane protocols running on the router S2R1. We have GRE for Tunneling, NHRP for NHS resolution and NHC registration used for multipoint GRE, iBGP for routing exchange between sites, ISAKMP and ESP for securing connection over tunnels, DHCP for address assignment to INET WAN interface and last ICMP for IP SLAs. All of these are Control Plane protocols but some of them are encapsulated by other Control Plane protocols, so those does not need FW rule. These protocols are NHRP and BGP, which are sent “inside” the tunnel encapsulated with GRE header. The Protocols that needs its own rule are ISAKMP, ICMP, DHCP, ESP and GRE. ISAKMP and ICMP need a stateful inspection (inspect) while DHCP, ESP and GRE do not (pass).
Figure 9.
Step-4.1: Access-list for the traffic from the Internet to router
- Define the access-list ACL-ROUTER-IN for the traffic to the router from the OUTSIDE zone.
ip access-list extended ACL-ROUTER-IN
permit udp any any eq non500-isakmp
permit udp any any eq isakmp
permit icmp any any echo
permit icmp any any echo-reply
permit icmp any any ttl-exceeded
permit icmp any any port-unreachable
permit udp any any range 33434 33463 ttl eq 1
Step-4.2: Access-list for traffic from the router to Internet
- Define the access-list ACL-ROUTER-OUT for the traffic originated by the router from the self-zone to the Internet.
ip access-list extended ACL-ROUTER-OUT
permit udp any any eq non500-isakmp
permit udp any any eq isakmp
permit icmp any any
Step-4.3: Access-list for DHCP packets from the Internet to the router
- Define the access-list ACL-DHCP-IN to allow the router to get its IP address via DHCP from the SP. This traffic
ip access-list extended ACL-DHCP-IN
permit udp any eq bootps any eq bootpc
Step-4.4: Access-list for DHCP packets from the router to the Internet
- Define the access-list ACL-DHCP-OUT to allow the router to request IP address from the DHCP server of SP to WAN interface.
ip access-list extended ACL-DHCP-OUT
permit udp any eq bootpc any eq bootps
Step-4.5: Access-list for ESP packets
- Define access-lists ACL-ESP-IN and ACL-ESP-OUT to allow the router to establish IPsec for DMVPN. Explicit inbound and outbound rule is required.
ip access-list extended ACL-ESP-IN
permit esp any any
!
ip access-list extended ACL-ESP-OUT
permit esp any any
Step-4.6: Access-list for Tunnel packets
- Define the access-list ACL-GRE-IN to allow the router to establish GRE tunnels. Only inbound access-list is needed.
ip access-list extended ACL-GRE-IN
permit gre any any
Step-5: Define the class-maps and assign access-lists to them.
Step-5.1: Class-map for stateful traffic from the Internet to the router
- Define the class-map CM-INSPECT-ROUTER-IN and assign the access-list ACL-ROUTER-IN to the class-map.
class-map type inspect match-any CM-INSPECT-ROUTER-IN
match access-group name ACL-ROUTER-IN
Step-5.2: Class-map for stateful traffic from the router to the Internet.
- Define the class-map CM-INSPECT-ROUTER-OUT and assign the access-list ACL-ROUTER-OUT to the class-map.
class-map type inspect match-any CM-INSPECT-ROUTER-OUT
match access-group name ACL-ROUTER-OUT
Step-5.3: Class-map for stateless traffic from the Internet to the router
- Define the class-map CM-PASS-ROUTER-IN and assign the access-lists ACL-ESP-IN, ACL-DHCP-IN and ACL-GRE-IN to the class-map.
class-map type inspect match-any CM-PASS-ROUTER-IN
match access-group name ACL-ESP-IN
match access-group name ACL-DHCP-IN
match access-group name ACL-GRE-IN
Step-5.4: Define the class-map for stateless traffic from the router to the Internet
- Define the class-map CM-PASS-ROUTER-OUT and assign the access-lists ACL-ESP-OUT and ACL-DHCP-OUT to the class-map.
class-map type inspect match-any CM-PASS-ROUTER-OUT
match access-group name ACL-ESP-OUT
match access-group name ACL-DHCP-OUT
Step-6: Define the Policy-maps and assign class-maps to them.
Step-6.1: Policy-map for traffic from the Internet to the router
- Define the policy-map PM-IN-POLICY.
- Assign class-map CM-INSPECT-ROUTER-IN with “inspect” action to the policy-map.
- Assign class-map CM-PASS-ROUTER-IN with “pass” action to the policy-map.
- Drop any other traffic
policy-map type inspect PM-IN-POLICY
class type inspect CM-INSPECT-ROUTER-IN
inspect
class type inspect CM-PASS-ROUTER-IN
pass
class class-default
drop
Step-6.2: Policy-map for traffic from the router to the Internet
- Define the policy-map PM-OUT-POLICY
- Assign class-map CM-INSPECT-ROUTER-OUT with “inspect” action to the policy-map.
- Assign class-map CM-PASS-ROUTER-OUT with “pass” action to the policy-map.
- Drop any other traffic
policy-map type inspect PM-OUT-POLICY
class type inspect CM-INSPECT-ROUTER-OUT
inspect
class type inspect CM-PASS-ROUTER-OUT
pass
class class-default
drop
Step-7: Zone-Pairs
Step-7.1: Define the zone pair for traffic from the OUTSIDE zone to the Self zone and apply the policy-map PM-IN-POLICY to it.
zone-pair security ZP-TO-ROUTER source OUTSIDE destination self
service-policy type inspect PM-IN-POLICY
Step-7.2: Define the zone pair for traffic Self-zone to OUTSIDE zone and apply the policy-map PM-OUT-POLICY.
zone-pair security FROM-ROUTER source self destination OUTSIDE
service-policy type inspect PM-OUT-POLICY
Verification
S2R1#sh policy-map type inspect zone-pair
Zone-pair: DEFAULT-TO-OUTSIDE
Service-policy inspect : PM-IN2OUT
Class-map: CM-IN2OUT (match-any)
Match: protocol tcp
Match: protocol udp
Match: protocol icmp
Inspect
Session creations since subsystem startup or last reset 0
Current session counts (estab/half-open/terminating) [0:0:0]
Maxever session counts (estab/half-open/terminating) [0:0:0]
Last session created never
Last statistic reset never
Last session creation rate 0
Last half-open session total 0
Class-map: class-default (match-any)
Match: any
Drop (default action)
0 packets, 0 bytes
Zone-pair: FROM-ROUTER
Service-policy inspect : PM-OUT-POLICY
Class-map: CM-INSPECT-ROUTER-OUT (match-any)
Match: access-group name ACL-ROUTER-OUT
Inspect
Packet inspection statistics [process switch:fast switch]
icmp packets: [0:4]
Session creations since subsystem startup or last reset 2
Current session counts (estab/half-open/terminating) [0:0:0]
Maxever session counts (estab/half-open/terminating) [2:0:0]
Last session created 00:00:17
Last statistic reset never
Last session creation rate 2
Last half-open session total 0
Class-map: CM-PASS-ROUTER-OUT (match-any)
Match: access-group name ACL-ESP-OUT
Match: access-group name ACL-DHCP-OUT
Pass
0 packets, 0 bytes
Class-map: class-default (match-any)
Match: any
Drop
0 packets, 0 bytes
Zone-pair: ZP-TO-ROUTER
Service-policy inspect : PM-IN-POLICY
Class-map: CM-INSPECT-ROUTER-IN (match-any)
Match: access-group name ACL-ROUTER-IN
Inspect
Session creations since subsystem startup or last reset 0
Current session counts (estab/half-open/terminating) [0:0:0]
Maxever session counts (estab/half-open/terminating) [0:0:0]
Last session created never
Last statistic reset never
Last session creation rate 0
Last half-open session total 0
Class-map: CM-PASS-ROUTER-IN (match-any)
Match: access-group name ACL-ESP-IN
Match: access-group name ACL-DHCP-IN
Match: access-group name ACL-GRE-IN
Pass
0 packets, 0 bytes
Class-map: class-default (match-any)
Match: any
Drop
0 packets, 0 bytes
S2R1#
If we take a ping from the Internet router R112 to the router S2R1 WAN interface Gi3, it succeeds, but telnet doesn't.
Internet#ping 10.21.112.21
Type escape sequence to abort.
Sending 5, 100-byte ICMP Echos to 10.21.112.21, timeout is 2 seconds:
!!!!!
Success rate is 100 percent (5/5), round-trip min/avg/max = 2/2/3 ms
Internet#telnet 10.21.112.21
Trying 10.21.112.21 ...
% Connection timed out; remote host not responding
Internet#
We can see that there are drops (telnet attempts).
S2R1#sh policy-map type inspect zone-pair ZP-TO-ROUTER | sec class-default
Class-map: class-default (match-any)
Match: any
Drop
8 packets, 464 bytes
We can see that also the icmp counter under CM-IN SPECT-ROUTER-OUT increases because of IP SLA generated icmp packets.
S2R1#sh policy-map type inspect zone-pair FROM-ROUTER
Zone-pair: FROM-ROUTER
Service-policy inspect : PM-OUT-POLICY
Class-map: CM-INSPECT-ROUTER-OUT (match-any)
Match: access-group name ACL-ROUTER-OUT
Inspect
Packet inspection statistics [process switch:fast switch]
icmp packets: [0:62]
One last thing to do. If both DMVPN tunnels are down, the traffic from the Remote site destined to corporate network 172.16.0.0/22 will be routed based on static default route to Internet direction. We can prevent this by setting the static null0 route for network 172.16.0.0/22 with AD 254 (higher than any other AD). If tunnels are down, all packets to network 172.16.0.0/22 will silently be dropped.
S2R1(config)#ip route 172.16.0.0 255.255.252.0 null0 254
Conclusion
Figure 10 shows the complexity of Zone-Based Firewall implementation.
Figure 10.
References:
Cisco Intelligent WAN (iWAN) – ISBN-10: 1-58714-463-8
Brad Edgeworth, Jean Marc Barozet, David Prall, Anthony Lockhart, Nir Ben-Dvora
No comments:
Post a Comment
Note: only a member of this blog may post a comment.