Introduction
In the previous chapter, you learned how to route east-west traffic through the Network Virtual Appliance (NVA) using subnet-specific route tables with User Defined Routes (UDR). This chapter introduces how to route north-south traffic between the Internet and your Azure Virtual Network through the NVA.
Figure 3-1 depicts our VNet setup, which includes DMZ and Web Tier zones. The NVA, vm-nva-fw, is connected to subnet snet-north (10.0.2.0/24) in the DMZ via a vNIC with Direct IP (DIP) 10.0.2.4. We've also assigned a public IP address, 51.12.90.63, to this vNIC. The second vNIC is connected to subnet snet-west (10.0.0.0/24) in the Web Tier, with DIP 10.0.0.5. We have enabled IP Forwarding in both vNICs and Linux kernel. We are using Network Security Groups (NSGs) for filtering north-south traffic.
Our web server, vm-west, has a vNIC with DIP 10.0.0.4 that is connected to the subnet snet-west in the Web Tier. We have associated the route table to the subnet with the UDR, which forwards traffic to destination IP 141.192.166.81 (remote host) to NVA. To publish the web server to the internet, we've used the public IP of NVA.
On the NVA, we have configured a Destination NAT rule which rewrites the destination IP address to 10.0.0.4 to packets with the source IP address 141.192.166.81 and protocol ICMP. To simulate an http connection, we're using ICMP requests from a remote host.
Figure 3-1: Example Diagram and.
Packet Walk
Figure 3-1 also illustrates the data path and related VFP operations between a remote host and our web server, vm-west. Because there is no http service running on vm-west, we emulate the http connection with ICMP messages. Instead of a detailed explanation of a VFP header transposition and Unified Flow Table entry creation process, this section focuses on the packet processing actions.
- Remote host 141.192.166.81 starts pinging the web server vm-west using the public IP address 51.12.90.63 as a destination. The ICMP Request messages are routed to Azure, and Azure infra routes traffic to NVA’s vNIC on DMZ.
- The first packet arrives at NVA. As an SLB/NAT layer action, the public IP address 51.12.90.63 of NVA is translated to its DIP: 10.0.2.4.
- Next, the packet is matched against the inbound policy rule in NSG associated with the vNIC. Because we have allowed ICMP from the remote host to 10.0.2.4, the packet is forwarded to NVA.
- The source IP address is 141.192.166.81, and the protocol ICMP matches NVA’s NAT rule. The NAT engine rewrites the destination public IP 51.12.90.63 to vm-west’s DIP 10.0.0.4. Then, the Linux routing process finds the next-hop IP address for the destination. The subnet 10.0.0.0/24 is connected to its interface eth2 via vNIC vm-nva-fw-west.
- The destination IP address 10.0.0.4 is within the same VNet as NVA’s egress vNIC, so the default Intra-VNet Outbound policy rule allows traffic. Note that if you are using a third-party FW NVA, there is no need for NVA-based NSGs. In our example, NVA does only routing, not policy filtering. That is why we are using NSGs.
- Next, the VFP routing process checks where the packet should be sent. Based on the Customer Address (CA) to Provider Address (PA) mapping entry (10.0.0.4 > Host B) found on the local host cache, the routing process knows that this is an Intra-VNet flow, meaning there is no need for NAT.
- Before sending the packet toward the destination, the VNet layer encapsulates the packet with tunnel headers, where the destination IP address is the PA of Host B, and the Virtual Network Identifier (VNI) is the VNet Id used with vnet-nwkt. Azure infra routes packets based on the destination IP address on the tunnel header.
- The packet arrives at Host B. Based on the CA and the VNI, the packet is forwarded to vm-west. The VNet layer decapsulates the packet.
- The Inbound policy allows the ICMP packet from the remote host, and the packet is forwarded to vm-west.
- Vm-west replies to the ICMP Request message by sending an ICMP Reply message to remote host 141.192.166.81.
- The Outbound policy allows Internet egress traffic by default.
- We have associated a route table with the subnet snet-west, in which UDR explicitly routes packets to the destination 141.192.166.81 using next-hop 10.0.0.5. Next, the VFP routing process checks the CA-PA mapping cache and finds mapping entry 10.0.0.5 > Host A. If we haven’t associated the route table with the subnet, the packet is routed to the Internet edge router without encapsulation.
- The VNet layer encapsulates the packet with tunnel headers, and then the packet is sent toward NVA.
- The packet arrives at Host A. Based on the CA and the VNI, the packet is forwarded to vm-nva-fw. The VNet layer action decapsulates the packet.
- The source IP address of the packet is within the VNet CIDR (10.0.0.0/16), so it matches the default Inbound Intra-VNet policy rule. The packet is sent to NVA, which routes the packet based on the default route table.
- NSG's OutBound policy rule allows internet traffic by default.
- Finally, the SLB/NAT layer translates the source IP address to a public IP address before sending the packet to the remote host.
Deployment
Linux NVA
Linux uses an Iptable kernel module for Network Address Translation (NAT). You can check if the Iptable process is running by using the command lsmod | grep iptables. In our example Linux, it is running by default. If it is not running, you can start the process by using the command sudo systemctl start iptables.
azureuser@vm-nva-fw:~$ lsmod|grep iptable
iptable_security 16384 1
ip_tables 32768 1 iptable_security
x_tables 53248 5 xt_conntrack,iptable_security,xt_tcpudp,xt_owner,ip_tables
Example 3-1: IP Table verification on Linux IP.
Besides Iptables, you need to enable IP Forwarding on Linux. You can start the process with the command sudo sh -c "echo 1 > /proc/sys/net/ipv4/ip_forward". Use the command sudo sysctl -a |grep ip_forward for IP Forwarding status verification.
azureuser@vm-nva-fw:~$ sudo sysctl -a |grep ip_forward
net.ipv4.ip_forward = 0
net.ipv4.ip_forward_update_priority = 1
net.ipv4.ip_forward_use_pmtu = 0
Example 3-2: Verify the State of the “IP-Forward” on Linux IP.
After enabling IP forwarding and Itables modules, we can use the command sudo iptables -t nat -L -n to list the current NAT entries programmed on Iptable. At this phase, our NAT table is still empty.
azureuser@vm-nva-fw:~$ sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Example 3-3: NAT entries in IptableTable.
Next, we create a Destination NAT rule (Prerouting) which checks the source IP address and protocol. If the source IP is 141.192.166.81 and the protocol is ICMP, the NAT process rewrites the destination IP address with the new IP address 10.0.0.4. We could also define a Source NAT rule (Postrouting) to change the original source IP address to 10.0.0.5. This would make it appear that the packet originated from vm-nva-fw, eliminating the need for UDR. However, for demonstration purposes, we will use only Destination NAT and rely on UDR.
sudo iptables -t nat -A PREROUTING -s 141.192.166.81 -p icmp -j DNAT --to-destination 10.0.0.4
Example 3-4: NAT entries in IptableTable.
Example 3-5 verifies that our DNAT configuration is installed into Iptable.
azureuser@vm-nva-fw:~$ sudo iptables -t nat -L -n
Chain PREROUTING (policy ACCEPT)
target prot opt source destination
DNAT icmp -- 141.192.166.81 0.0.0.0/0 to:10.0.0.4
Chain INPUT (policy ACCEPT)
target prot opt source destination
Chain OUTPUT (policy ACCEPT)
target prot opt source destination
Chain POSTROUTING (policy ACCEPT)
target prot opt source destination
Example 3-5: NAT entries in Iptable After Adding a New DNAT Rule.
IP Forwarding on vNIC
We have enabled IP Forwarding on vNIC vm-nva-fw-west in the previous chapter. Now we only have to turn it on on vNIC vm-nva-fw239. Example 3-6 shows how you can do it on Azure CLI.
az network nic update `
--name vm-nva-fw239 `
--resource-group rg-nwkt-demo `
--ip-forwarding true
Example 3-6: Enable IP Forwarding on vNIC.
Use the Azure CLI command “az network route-table route create” to add a routing entry to the route table. You need to specify the name of the route, the resource group, and the route table where you are deploying it. Besides, define the prefix, next-hop type, and next-hop IP address.
az network route-table route create `
--name rt-west-vm-nva-fw `
--resource-group rg-nwkt-demo `
--route-table-name rt-west `
--address-prefix 141.192.166.81/32 `
--next-hop-type VirtualAppliance `
--next-hop-ip-address 10.0.0.5
Example 3-7: Create UDR to Route Table.
Once you have associated subnet snet-west with the route table rt-west, vNICs vm-west747 (vm-west) and vm-nva-nic-west (vm-nva-fw) start using the route. To confirm which routes are effective in vNIC, use the Azure CLI command "az network nic show-effective routes".
az network nic show-effective-route-table `
>> -g rg-nwkt-demo `
>> -n vm-west747 `
>> -o table
Source State Address Prefix Next Hop Type Next Hop IP
-------- ------- ------------------ ---------------- -------------
Default Active 10.0.0.0/16 VnetLocal
Default Active 0.0.0.0/0 Internet
**snipped for brevity**
User Active 141.192.166.81/32 VirtualAppliance 10.0.0.5
Example 3-8: vNIC Effective Routes Verification.
Data Plane verification
The example below shows that we can ping from the Remote host to IP address 51.12.90.63. However, it doesn’t verify from whom we get the replay.
C:\Users\remote>ping -S 141.192.166.81 51.12.90.63 -n 2
Pinging 51.12.90.63 with 32 bytes of data:
Reply from 51.12.90.63: bytes=32 time=69ms TTL=45
Reply from 51.12.90.63: bytes=32 time=76ms TTL=45
Ping statistics for 51.12.90.63:
Packets: Sent = 2, Received = 2, Lost = 0 (0% loss),
Approximate round trip times in milli-seconds:
Minimum = 69ms, Maximum = 76ms, Average = 72ms
Example 3-9: Ping from Remote Host to Web Server.
Example 3-10 shows that the ICMP Request packet, sent by the Remote host, enters NVA from interface eth0. The public destination IP address 51.12.90.63 belongs to NVA. When forwarding the packet to VM, the VFP NAT engine does public IP to Direct IP translation (51.12.90.63 > 10.0.2.4). The example below also verifies that ICMP Reply packets are forwarded to the Internet out of the same interface. Note that Linux doesn’t see the Source NAT operation of the VFP when the ICMP Reply messages are forwarded toward the Remote host.
azureuser@vm-nva-fw:~$ sudo tcpdump -i eth0 icmp -n -t
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 141.192.166.81 > 10.0.2.4: ICMP echo request, id 1, seq 168, length 40
IP 10.0.2.4 > 141.192.166.81: ICMP echo reply, id 1, seq 168, length 40
IP 141.192.166.81 > 10.0.2.4: ICMP echo request, id 1, seq 169, length 40
IP 10.0.2.4 > 141.192.166.81: ICMP echo reply, id 1, seq 169, length 40
Example 3-10: Tcpdump from the Interface Eth0 on NVA vm-nva-fw.
Example 3-11 proves that our Destination NAT works because the packet is forwarded to the web server vm-west (10.0.0.4). ICMP Replies are received from the same interface, which verifies that we have a symmetric routing.
azureuser@vm-nva-fw:~$ sudo tcpdump -i eth2 icmp -n -t
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth2, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 141.192.166.81 > 10.0.0.4: ICMP echo request, id 1, seq 170, length 40
IP 10.0.0.4 > 141.192.166.81: ICMP echo reply, id 1, seq 170, length 40
IP 141.192.166.81 > 10.0.0.4: ICMP echo request, id 1, seq 171, length 40
IP 10.0.0.4 > 141.192.166.81: ICMP echo reply, id 1, seq 171, length 40
Example 3-11: Tcpdump from the Interface Eth2 on NVA vm-nva-fw.
Example 3-12 shows the tcpdump taken from vm-west. It shows that it a) receives the ICMP Request and b) answers with ICMP Reply.
azureuser@vm-west:~$ sudo tcpdump -i eth0 icmp -n -t
tcpdump: verbose output suppressed, use -v or -vv for full protocol decode
listening on eth0, link-type EN10MB (Ethernet), capture size 262144 bytes
IP 141.192.166.81 > 10.0.0.4: ICMP echo request, id 1, seq 168, length 40
IP 10.0.0.4 > 141.192.166.81: ICMP echo reply, id 1, seq 168, length 40
IP 141.192.166.81 > 10.0.0.4: ICMP echo request, id 1, seq 169, length 40
IP 10.0.0.4 > 141.192.166.81: ICMP echo reply, id 1, seq 169, length 40
Example 3-12: Tcpdump from the Interface Eth0 on Web Server vm-west.
References
[1] High availability ports overview
https://learn.microsoft.com/en-us/azure/load-balancer/load-balancer-ha-ports-overview, February 12, 2023
[2] Deploy highly available NVAs
https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/dmz/nva-ha
[3] Dissociate a public IP address from an Azure VM
https://learn.microsoft.com/en-us/azure/virtual-network/ip-services/remove-public-ip-address-vm, March 8, 2023
[4] Deploy highly available NVAs
https://learn.microsoft.com/en-us/azure/architecture/reference-architectures/dmz/nva-ha
[5] Gateway Load Balancer
https://learn.microsoft.com/en-us/azure/load-balancer/gateway-overview, February 12, 2023.
[6] Best practices to consider before deploying a network virtual appliance
https://azure.microsoft.com/es-es/blog/best-practices-to-consider-before-deploying-a-network-virtual-appliance/
[7] Virtual Appliance Scenario
https://learn.microsoft.com/en-us/azure/virtual-network/virtual-network-scenario-udr-gw-nva, March 23, 2023
[8] Virtual Appliance Scenario
https://access.redhat.com/documentation/en-us/red_hat_enterprise_linux/4/html/security_guide/s1-firewall-ipt-fwd
[9] How to redirect traffic to another IP using iptables
https://serverok.in/how-to-redirect-traffic-to-another-ip-using-iptables
[10] Forward a TCP port to another IP or port using NAT with Iptables
https://jensd.be/343/linux/forward-a-tcp-port-to-another-ip-or-port-using-nat-with-iptables
