Thursday, 2 February 2023

Azure Networking Fundamentals: VNET Peering

Comment: Here is a part of the introduction section of the eight chapter of my Azure Networking Fundamentals book. I will also publish other chapters' introduction sections soon so you can see if the book is for you. The book is available at Leanpub and Amazon (links on the right pane).

This chapter introduces an Azure VNet Peering solution. VNet peering creates bidirectional IP connections between peered VNets. VNet peering links can be established within and across Azure regions and between VNets under the different Azure subscriptions or tenants. The unencrypted data path over peer links stays within Azure's private infrastructure. Consider a software-level solution (or use VGW) if your security policy requires data path encryption. There is no bandwidth limitation in VNet Peering like in VGW, where BW is based on SKU. From the VM perspective, VNet peering gives seamless network performance (bandwidth, latency, delay, and jitter) for Inter-VNet and Intra-VNet traffic. Unlike the VGW solution, VNet peering is a non-transitive solution, the routing information learned from one VNet peer is not advertised to another VNet peer. However, we can permit peered VNets (Spokes) to use local VGW (Hub) and route Spoke-to-Spoke data by using a subnet-specific route table (chapter 9 explains the concept in detail). Note that by deploying a VNet peering, we create a bidirectional, always-on IP data path between VNets. However, we can prevent traffic from crossing the link if needed without deleting the peering. Azure uses Virtual Network Service Tags for VNet peering traffic policy.

Figure 8-1 shows our example topology. We create a VNet Peering between vnet-spoke-2 and vnet-nsg-rt-swedencentral. Besides the Inter-VNet connection, our solution allows vnet-spoke-2 to use vnet-nsg-rt-swedencentral as a transit VNet to other peered VNets (which we don’t have in this example). We also permit IP connection to/from vnet-spoke-2 to vnet-spoke-1 and on-prem location by authorizing vnet-spoke-2 to use vgw-nwkt as a transit gateway.

Figure 8-1: VNet Peering Example Diagram.

Sunday, 29 January 2023

Azure Networking Fundamentals: Site-to-Site VPN

Comment: Here is a part of the introduction section of the fifth chapter of my Azure Networking Fundamentals book. I will also publish other chapters' introduction sections soon so you can see if the book is for you. The book is available at Leanpub and Amazon (links on the right pane).

A Hybrid Cloud is a model where we split application-specific workloads across the public and private clouds. This chapter introduces Azure's hybrid cloud solution using Site-to-Site (S2S) Active-Standby VPN connection between Azure and on-prem DC. Azure S2S A/S VPN service includes five Azure resources. The first one, Virtual Network Gateway (VGW), also called VPN Gateway, consists of two VMs, one in active mode and the other in standby mode. These VMs are our VPN connection termination points on the Azure side, which encrypt and decrypt data traffic. The active VM has a public IP address associated with its Internet side. If the active VM fails, the standby VM takes the active role, and the public IP is associated with it. Active and standby VMs are attached to the special subnet called Gateway Subnet. The name of the gateway subnet has to be GatewaySubnet. The Local Gateway (LGW) resource represents the VPN termination point on the on-prem location. Our example LGW is located behind the NAT device. The inside local IP address of LGW is the private IP, which the NAT device translates to public IP Because of this, we set our VGW in ResponderOnly mode. The last resource is the Connection resource. It defines the tunnel type and its termination points. In our example, we are using Site-to-Site (IPSec) tunnels, which are terminated to our VGW and LGW.

Figure 5-1: Active-Standby Site-to-Site VPN Overview.

Thursday, 26 January 2023

Azure Networking Fundamentals: Internet Access with VM-Specific Public IP

Comment: Here is a part of the introduction section of the Third chapter of my Azure Networking Fundamentals book. I will also publish other chapters' introduction sections soon so you can see if the book is for you. The book is available at Leanpub and Amazon (links on the right pane).

In chapter two, we created a VM vm-Bastion and associated a Public IP address to its attached NIC vm-bastion154. The Public IP addresses associated with VM’s NIC are called Instance Level Public IP (ILPIP). Then we added a security rule to the existing NSG vm-Bastion-nsg, which allows an inbound SSH connection from the external host. Besides, we created VMs vm-front-1 and vm-Back-1 without public IP address association. However, these two VMs have an egress Internet connection because Azure assigns Outbound Access IP (OPIP) addresses for VMs for which we haven’t allocated an ILPIP (vm-Front-1: and vm-Back-1- The Azure portal does not list these IP addresses in the Azure portal VM view. Note that neither user-defined nor Azure-allocated Public IP addresses are not configured as NIC addresses. Instead, Azure adds them as a One-to-One entry to the NAT table (chapter 15 introduces a NAT service in detail). Figure 3-1 shows how the source IP address of vm-Bastion is changed from to when traffic is forwarded to the Internet. The source IP address of the Internet traffic from vm-Front-1 and vm-Back-1 will also be translated in the same way. The traffic policy varies based on the IP address assignment mechanism. The main difference is that external hosts can initiate connection only with VMs with an ILPIP. Besides, these VMs are allowed to use TCP/UDP/ICMP, while VMs with the Azure assigned public IP address can only use TCP or UDP but not ICMP. 

Figure 3-1: Overview of the Azure Internet Access.

Tuesday, 24 January 2023

Azure Networking Fundamentals: Network Security Group (NSG)

Comment: Here is a part of the introduction section of the second chapter of my Azure Networking Fundamentals book. I will also publish other chapters' introduction sections soon so you can see if the book is for you. The book is available at Leanpub and Amazon (links on the right pane). 

This chapter introduces three NSG scenarios. The first example explains the NSG-NIC association. In this section, we create a VM that acts as a Bastion host*). Instead of using the Azure Bastion service, we deploy a custom-made vm-Bastion to snet-dmz and allow SSH connection from the external network. The second example describes the NSG-Subnet association. In this section, we launch vm-Front-1 in the front-end subnet. Then we deploy an NSG that allows SSH connection from the Bastion host IP address. The last part of the chapter introduces an Application Security Group (ASG), which we are using to form a logical VM group. We can then use the ASG as a destination in the security rule in NSG. There are two ASGs in figure 2-1. We can create a logical group of VMs by associating them with the same Application Security Group (ASG). The ASG can then be used as a source or destination in NSG security rules. In our example, we have two ASGs, asg-Back (associated with VMs and asg-Back#2 (associated with VMs The first ASG (asg-Back) is used as a destination in the security rule on the NSG nsg-Back that allows ICMP from VM vm-Front-1. The second ASG (asg-Back#2) is used as a destination in the security rule on the same NSG nsg-Back that allows ICMP from VM vm-Bastion. Examples 1-7 and 1-8 show how we can get information about Virtual Networks using Azure AZ PowerShell.

*) Azure Bastion is a managed service for allowing SSH and RDP connections to VMs without a public IP address. Azure Bastion has a fixed price per hour and outbound data traffic-based charge.                            

Figure 2-1: Network Security Group (NSG) – Example Scenarios.

Wednesday, 11 January 2023

Azure Host-Based Networking: vNIC Interface Architecture - Synthetic Interface and Virtual Function

Before moving to the Virtual Filtering Platform (VFP) and Accelerated Network (AccelNet) section, let’s look at the guest OS vNIC interface architecture. When we create a VM, Azure automatically attaches a virtual NIC (vNIC) to it. Each vNIC has a synthetic interface, a VMbus device, using a netvsc driver. If the Accelerated Networking (AccelNet) is disabled on a VM, all traffic flows pass over the synthetic interface to the software switch. Azure hosts servers have Mellanox/NVIDIA Single Root I/O Virtualization (SR-IOV) hardware NIC, which offers virtual instances, Virtual Function (VF), to virtual machines. When we enable AccelNet on a VM, the mlx driver is installed to vNIC. The mlx driver version depends on an SR-IOV type. The mlx driver on a vNIC initializes a new interface that connects the vNIC to an embedded switch on a hardware SR-IOV. This VF interface is then associated with the netvsc interface. Both interfaces use the same MAC address, but the IP address is only associated with the synthetic interface. When AccelNet is enabled, VM’s vNIC forwards VM data flows over the VF interface via the synthetic interface. This architecture allows In-Service Software Updates (ISSU) for SR-IOV NIC drivers. 

Note! Exception traffic, a data flow with no flow entries on a UFT/GFT, is forwarded through VFP in order to create flow-action entries to UFT/GFT.

Figure 1-1: Azure Host-Based SDN Building Blocks.

Sunday, 8 January 2023

Azure Host-Based Networking: VFP and AccelNet Introduction

Software-Defined Networking (SDN) is an architecture where the network’s control plane is decoupled from the data plane to centralized controllers. These intelligent, programmable controllers manage network components as a single system, having a global view of the whole network. Microsoft’s Azure uses a host-based SDN solution, where network virtualization and most of its services (Firewalls, Load balancers, Gateways) run as software on the host. The physical switching infrastructure, in turn, offers a resilient, high-speed underlay transport network between hosts.

Figure 1-1 shows an overview of Azure’s SDN architecture. Virtual Filtering Platform (VFP) is Microsoft’s cloud-scale software switch operating as a virtual forwarding extension within a Hyper-V basic vSwitch. The forwarding logic of the VFP uses a layered policy model based on policy rules on Match-Action Table (MAT). VFP works on a data plane, while complex control plane operations are handed over to centralized control systems. VFP layers, such as VNET, NAT, ACL, and Metering, have dedicated controllers that programs policy rules to MAT using southbound APIs.

Software switches switching processes are CPU intensive. To reduce the burden of CPU cycles, VFP offloads data forwarding logic to hardware NIC after processing the first packet of the flow and creating the flow entry to MAT. The Header Transposition (HT) engine programs flow and their forwarding actions, like source IP address rewrite, into a Unified Flow Table (UFT), which has flow entries for all active flows of every VM running on a host. Flows and policies on UFT are loaded into a Generic Flow Table (GFT) on the hardware NIC’s Field Programmable Gate Array (FPGA) unit and subsequent packets take a fast path over a hardware NIC. Besides GFT, a hardware NIC has Single Root I/O Virtualization (SR-IOV) NIC. It offers vNIC-specific, secure access between VM and hardware NIC. From the VM perspective, the SR-IOV NIC appears as a PCI device using a Virtual Function (VF) driver. The guest OS connection to VFP over VMBus uses a synthetic interface with Network Virtual Service Client (NetVSC) driver. NetVSC and VF interfaces are bonded and use the same MAC address. However, the IP address is attached to the NetVSC interface. A vNIC exposes only the synthetic interface to the TCP/IP stack of the guest OS. This solution makes it possible to switch flows from the fast (VF) path to the slow path (NetVSC) during a hardware NIC service operation or failure event without disturbing active connections.

VFP software switch and FPGA/SR-IOV hardware NIC together forms Microsoft’s host-based-SDN architecture called Accelerated Network (AccelNet). This post series introduces the solution in detail.

Figure 1-1: Azure Host-Based SDN Building Blocks.


[1] Daniel Firestone et al., “VFP: A Virtual Switch Platform for Host SDN in the Public Cloud”, 2017

[2] Daniel Firestone et al., “Azure Accelerated Networking: SmartNICs in the Public Cloud”, 2018

Tuesday, 3 January 2023

Azure Host-Based SDN: Part 1 - VFP Introduction

Azure Virtual Filtering Platform (VFP) is Microsoft’s cloud-scale virtual switch operating as a virtual forwarding extension within a Hyper-V basic vSwitch. Figure 1-1 illustrates an overview of VFP building blocks and relationships with basic vSwitch. Let’s start the examination from the VM vm-nwkt-1 perspective. Its vNIC vm-cafe154 has a synthetic interface eth0 using a NetVSC driver (Network Virtual Service Client). The Hyper-V vSwitch on the Parent Partition is a Network Virtual Service Provider (NetVSP) with VM-facing vPorts. Vm-cafe154 is connected to vPort4 over the logical inter-partition communication channel VMBus. VFP sits in the data path between VM-facing vPorts and default vPort associated with physical NIC. VFP uses port-specific Layers for filtering traffic to and from VMs. A VFP Layer is a Match Action Table (MAT) having a set of policy Rules. Rules consist of Conditions and Actions and are divided into Groups. Each layer is programmed by independent, centralized Controllers without cross-controller dependencies.

Let’s take a concrete example of Layer/Group/Rule object relationship and management by examining the Network Security Group (NSG) in the ACL Layer. Each NSG has a default group for Infrastructure rules, which allows Intra-VNet traffic, outbound Internet connection, and load balancer communication (health check, etc.). We can’t delete, add or modify rules in this group. The second group has User Defined rules, which we can use to allow/deny traffic flows based on our security policy. An NSG Rule consists of Conditions and Actions. Condition defines the match policy using 5-tuple of src-dst IP/Protocol/src-dst Ports. A Condition is associated with an Action for matching data flows. In our example, we have an Inbound Infrastructure Rule with Condition/Action that allows connection initiation from VMs within the VNet. ACL layer control component is Security Controller. We use the Security Controller's Northbound API when we create or modify an NSG with Windows PowerShell or Azure GUI. Security Controllers, in turn, use a Southbound API to program our intent to VFP via Host Agent.

The next post explains how VFP handles outgoing/incoming data streams and creates Unified Flow Tables (UFT) from them using the Header Transposition solution.

Figure 1-1: Virtual Filtering Platform Overview (click to enlarge).