Here is the Table of Contents of my AWS Networking Fundamentals book. I have added the figures which illustrate the example scenarios in each chapter. The book is available at Leanpub.com. It is still in progress, and there will be additional chapters soon.
Back-End EC2 instances like Application and Database servers are most often launched on a Private subnet. As a recap, a Private subnet is a subnet that doesn’t have a route to the Internet Gateway in its Route table. Besides, EC2 instances in the Private subnet don’t have Elastic-IP address association. These two facts mean that EC2 instances on the Private subnet don’t have Internet access. However, these EC2 instances might still need occasional Internet access to get firmware upgrades from the external source. We can use a NAT Gateway (NGW) for allowing IPv4 Internet traffic from Private subnets to the Internet. When we launch an NGW, we also need to allocate an Elastic-IP address (EIP) and associate it with the NGW. This association works the same way as the EIP-to-EC2 association. It creates a static NAT entry to IGW that translates NGW’s local subnet address to its associated EIP. The NGW, in turn, is responsible for translating the source IP address from the ingress traffic originated from the Private subnet to its local subnet IP address. As an example, EC2 instance NWKT-EC2-Back-End sends packets towards the Internet to NGW. When the NGW receives these packets, it rewrites the source IP address 10.10.1.172 with its Public subnet IP address 10.10.0.195 and forwards packets to the Internet gateway. IGW translates the source IP address 10.10.0.195 to EIP 18.132.96.95 (EIP associated with NGW). That means that the source IP of data is rewritten twice, first by NGW and then by IGW.
Figure 4-1 illustrates our example NAT GW design and its configuration steps. As a pretask, we launch an EC2 instance on the Private subnet 10.10.1.0/24 (1). We also modify the existing Security Group (SG) to allow an Inbound/Outbound ICMP traffic within VPC CIDR 10.10.0.0/16 (2). We also allow an SSH session initiation from the 10.10.0.218/24. I’m using the same SG for both EC2 instances to keep things simple. Besides, both EC2 uses the same Key Pair. Chapter 3 shows how to launch an EC2 instance and how we modify the SGs, and that is why we go ahead straight to the NGW configuration.
When we have done pre-tasks, we launch an NGW on the Public subnet (3). Then we allocate an EIP and associate it with NGW (4). Next, we add a default route towards NGW on the Private subnet Route Table (5).
The last three steps are related to connectivity testing. First, verify Intra-VPC IP connectivity using ICMP (6). Then we test the Internet connectivity (7). As the last step, we can confirm that no route exists back to NWKT-EC-Backe-End from the IGW. We are using an AWS Path Analyzer for that (8).
Note! Our example doesn’t follow good design principles. AWS Availability Zones (AZ) are restricted failure domains, which means that failure in one AZ doesn’t affect the operation of other AZ. Now, if our NGW on AZ eu-west-2c fails, Internet traffic from the Private subnet on eu-west2a fails. The proper design is to launch NGW on the AZ where unidirectional egress Internet access is needed.
Figure 3-20: EC2 Instance, Elastic IP, and Security Group.
This chapter explains what components/services and configurations we need to allow Internet traffic to and from an EC2 instance. VPCs themselves are closed entities. If we need an Internet connection, we need to use an AWS Internet Gateway (IGW) service. The IGW is running on a Blackfoot Edge Device in the AWS domain. It performs Data-Plane VPC encapsulation and decapsulation, as well as IP address translation. We also need public, Internet routable IP addresses. In our example, we allocate an AWS Elastic-IP (EIP) address. Then we associate it with EC2 Instance. By doing it, we don’t add the EIP to the EC2 instance itself. Instead, we create a static one-to-one NAT entry into the VPC associated IGW. The subnet Route Table includes only a VPC’s CIDR range local route. That is why we need to add a routing entry to the Subnet RT, default or more specific, towards IGW. Note that a subnet within an AWS VPC is not a Broadcast domain (VPC doesn’t even support Broadcasts). Rather, we can think of it as a logical place for EC2 instances having uniform connection requirements, like reachability from the Internet. As a next step, we define the security policy. Each Subnet has a Network Access Control List (NACL), which is a stateless Data-Plane filter. The Stateless definition means that to allow bi-directional traffic flow, we have to permit flow-specific Request/Reply data separately. For simplicity, we are going to use the Subnet Default NACL. The Security Group (SG), in turn, is a stateful EC2 instance-specific Data-Plane filter. The Stateful means that filter permits flow-based ingress and egress traffic. Our example security policy is based on the SG. We will allow an SSH connection from the external host 91.152.204.245 to EC2 instance NWKT-EC-Fron-End. In addition, we allow all ICMP traffic from the EC2 instance to the same external host. As the last part, this chapter introduces the Reachability Analyzer service, which we can use for troubleshooting connections. Figure 3-1 illustrates what we are going to build in this chapter.
Figure 3-1: Setting Up an Internet Connection for Public
Subnet of AWS VPC.
At this phase, we have attached subnets to their respective Availability Zones. Next, we will create subnet-specific route tables for both Public and Private subnets.
Figure 1-25: VPC Subnets: Select VPC.
When we have created a new VPC, we can start adding subnets to it. We are going to create two subnets. Subnet 10.10.0.0/24 is a Public Subnet in Availability Zone eu-west2c, where we later add an Internet GW. Subnet 10.10.0.0/24 is a Private Subnet in Availability Zone eu-west2a that will use a NAT GW for uni-directional Internet access.
Figure 1-18: VPC Route Table: Routes.
The focus of this section is to show how we can create a VPC using AWS CloudFormation service. Figure 1-12 shows our example AWS CloudFormation Templates. Its first section, AWSTemplateFormatVersion, specifies the template language format. At the time of writing, 2010-09-09 is the latest and only valid version. We can use the Description section to describe our template. Note that it must follow the AWSTemplateFormatVersion Section. AWSTemplateFormation-Version and Description are optional sections. The Resources section specifies the actual AWS resources and their properties. Each AWS resource is identified with a logical name. I have given the logical name NwktVPC for our example VPC. We can use resource-specific logical names for defining dependencies between resources. For example, when adding the AWS::EC2::Subnet resource to our template, we assign the VpcId value by calling it from the AWS::EC2::VPC resource using !REF intrinsinc function. I will explain the process in the Subnet section later. The resources and their properties are defined under logical names. The Resources section is the only required section in AWS CloudFormation-Template. AWS CloudFormation Templates are identified by using Stack Names in AWS Cloud Formation. Our example Stack Name is MyNetworkStack.
Figure 1-12: AWS CloudFormation: VPC.
We can verify our VPC configuration by using AWS CLI. Example 1-1 shows the output for command aws ec2 describe-vpc in JSON format. This command lists all our VPC resources with their properties. The first one is the newest VPC NVKT-VPC-01, and the second one is the default VPC which I have named DFLT-VPC. The first VPC gets ordinal zero [0], and the second VPC gets number one [1]. Note that ordinal numbers are not shown in the output. VPC properties describe the VPC-specific CIDR Block, DHCP Options, VPC Identifier, Owner Id, CIDR Block Association, and Tags.
aws ec2 describe-vpcs
{
"Vpcs": [
{
"CidrBlock": "10.10.0.0/16",
"DhcpOptionsId": "dopt-09217361",
"State": "available",
"VpcId": "vpc-04ef72cc79a73f82e",
"OwnerId": "123456654321",
"InstanceTenancy": "default",
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-0379c0e3e854f43ff",
"CidrBlock": "10.10.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": false,
"Tags":
[
{
"Key": "Name",
"Value": "NVKT-VPC-01"
}
]
},
{
"CidrBlock": "172.31.0.0/16",
"DhcpOptionsId": "dopt-09217361",
"State": "available",
"VpcId": "vpc-cfbac1a7",
"OwnerId": "123456654321",
"InstanceTenancy": "default",
"CidrBlockAssociationSet": [
{
"AssociationId": "vpc-cidr-assoc-89d487e1",
"CidrBlock": "172.31.0.0/16",
"CidrBlockState": {
"State": "associated"
}
}
],
"IsDefault": true,
"Tags":
[
{
"Key": "Name",
"Value":
"DFLT-VPC"
}
]
}
]
Example 1-1: AWS CLI: Retrieve VPC Information.
We can use filters for retrieving information only from some specific resources. The command aws ec2 describe-vpcs --filters Name=tag:Name,Values=NVKT-VPC-01 shows VPCs where we have attached the Key/Value pair Name/NVKT-VPC-01.
aws ec2 describe-vpcs --filters
Name=tag:Name,Values=NVKT-VPC-01
{
"Vpcs": [
{
"CidrBlock":
"10.10.0.0/16",
"DhcpOptionsId":
"dopt-09217361",
"State":
"available",
"VpcId":
"vpc-04ef72cc79a73f82e",
"OwnerId":
"123456654321",
"InstanceTenancy":
"default",
"CidrBlockAssociationSet": [
{
"AssociationId":
"vpc-cidr-assoc-0379c0e3e854f43ff",
"CidrBlock":
"10.10.0.0/16",
"CidrBlockState":
{
"State":
"associated"
}
}
],
"IsDefault": false,
"Tags": [
{
"Key":
"Name",
"Value":
"NVKT-VPC-01"
}
]
}
]
}
Example 1-2: AWS CLI: Retrieve VPC Information.
We can also query resource-specific information using the command aws ec2 describe-vpcs --query "Vpcs[0]". The zero within square brackets after the resource Vpcs identifies the ordinal number of a resource. In our example, VPC NVKT-VPC-01 is the first one, and it has an ordinal number zero.
aws ec2 describe-vpcs --query
"Vpcs[0]"
{
"CidrBlock":
"10.10.0.0/16",
"DhcpOptionsId":
"dopt-09217361",
"State": "available",
"VpcId":
"vpc-04ef72cc79a73f82e",
"OwnerId": "123456654321",
"InstanceTenancy":
"default",
"CidrBlockAssociationSet": [
{
"AssociationId":
"vpc-cidr-assoc-0379c0e3e854f43ff",
"CidrBlock":
"10.10.0.0/16",
"CidrBlockState": {
"State":
"associated"
}
}
],
"IsDefault": false,
"Tags": [
{
"Key": "Name",
"Value":
"NVKT-VPC-01"
}
]
}
Example 1-3: AWS CLI: Retrieve VPC Information.
If we want to see only some specific resource properties, we can add the properties after the resource, separated by a dot. Example 1-4 shows how we can see the CIDR Block Association for VPC NVKT-VPC-01 (ordinal zero).
aws ec2 describe-vpcs --query
"Vpcs[0].CidrBlockAssociationSet"
[
{
"AssociationId":
"vpc-cidr-assoc-0379c0e3e854f43ff",
"CidrBlock":
"10.10.0.0/16",
"CidrBlockState": {
"State":
"associated"
}
}
]
Example 1-4: AWS CLI: Retrieve CIDR (Properties) Association
to VPC (Resource).
We can change the output representation from the JSON to table by using the option --output table. The table output is a good choice when we create documentation about VPCs. Note that you can use this option with all other commands too.
aws ec2 describe-vpcs --query
"Vpcs[0].CidrBlockAssociationSet" --output table
------------------------------------------------------
| DescribeVpcs |
+-----------------------------------+----------------+
| AssociationId |
CidrBlock |
+-----------------------------------+----------------+
| vpc-cidr-assoc-0379c0e3e854f43ff | 10.10.0.0/16
|
+-----------------------------------+----------------+
|| CidrBlockState ||
|+------------------+-------------------------------+|
|| State
| associated ||
|+------------------+-------------------------------+|
Example 1-5: AWS CLI: Retrieve CIDR Association to VPC – Table
Output.
The first thing to do when we create a VPC is to log in to the AWS console. Then we select the AWS Region where we want to launch our VPC. We are going to use VPC Region Europe (London) eu-west-2. As the last step, we give the name to VPC and associate a CIDR block 10.10.0.0/16 to it.
Figure 1-3: Virtual Private Cloud (VPC) – Example VPC.
AWS Virtual Private Cloud (VPC) is a virtual network for Amazon Elastic Cloud Compute instances (EC2) within AWS Region. AWS Regions, in turn, belongs to the global AWS Cloud environment. Each AWS Region consists of three or more physical data centers, Availability Zones (AZ). At the time of writing, Seoul and Tokyo have four, and Northern Virginia has six AZs. All other regions have three AZs. VPC spans over regional AZs but not between AWS Regions. In other words, VPCs are region-specific virtual networks.
A VPC has to have a CIDR (Classless Interdomain Routing) IP block attached to it. The VPC CIDR defines the IP range that we can use when creating subnets to VPC. CIDR range is VPC specific and can overlap with other VPC’s CIRD range. If there should be VPC-to-VPC inter-connection, VPC CIDR IP ranges have to be unique per VPC.
We can allocate subnets for EC2 instances from the VPC’s CIDR range. Subnets are AZ-specific, and they can’t be span from one AZ to another. Subnets are classified either as Public Subnets or Private Subnets. Public Subnet has a route to Internet GW (Internet Gateway) in its Routing Table (RT). EC2 instances launched in a Public Subnet have to have a public IPv4 address in order to have an Internet connection. Note that IPv6 addresses are always assigned from the public address space. EC2 launched in a Private Subnet doesn’t need a public IPv4 address, they can have an Internet connection through the NAT GW. To allow Internet connection to EC2 instances in Private Subnet, we need to add a route to NAT GW into the Private Subnet Routing Table. We can allow a stateful egress-only Internet connection for EC2 instances with IPv6 addresses in Private Subnet by using Egress-Only Internet GW. This way EC2 instance has an Internet connection but hosts on the internet can’t initiate a connection to EC2. IP connectivity between EC2 instances within VPC is established between private IP address even if one of the EC2s is attached to Public Subnet and has a Public IP address. VPC has a main Routing Table that is used with subnets which we don’t define subnet-specific RT.
Each VPC also has a default Network Access Control List (NACL). The default NACL is bind to all subnets in VPC by default. NACL is stateless by nature, traffic to and from the subnet has to be allowed in both inbound and outbound directions. The default NACL allows all ingress/egress traffic.
Figure 1-1 illustrates our example VPC and its relationship to AWS Availability Zones, AWS Regions, and AWS Account. When we create VPC, we first have to log on to our AWS account. Next, we select an AWS Region, in our case Europe (London) eu-west-2. Then we choose Availability Zones for subnets. In our case, network 10.10.0.0/24 is a Public Subnet in the AZ eu-west-2c, and network 10.10.1.0/24 is a Private Subnet in the AZ eu-west-2a. As the last step, we create subnet-specific Routing Tables where we can later add subnet-specific routes.
Figure 1-1: Virtual Private Cloud (VPC) Basic Building
Blocks.
Every Overlay Network solution requires IP
reachability between edge devices via Underlay Network. This section explains
the basic routing solution in Underlay Network from Campus Fabric, SD-WAN, and
Datacenter Fabric perspectives. Figure 7-1 illustrates the IP reachability
requirements for Campus Fabric, SD-WAN, and Datacenter Fabric.
Figure 7-1: IP Reachability Requirements.
This chapter introduces Data-Plane operation and explains
how the data packets from EP3 (IP 172.16.30.3) in Datacenter Fabric are
forwarded via SD-WAN to EP1 (IP 172.16.100.10) in Campus Fabric. (1) EndPoint3
sends the ICMP Request packet to its gateway switch Leaf-11. Leaf-11 makes routing
decisions based on the VRF NWKT routing table. Before forwarding the packet, Leaf-11
adds a VXLAN header where it uses L3VNI 10077. It also sets the outer IP header
where it uses the Border-Leaf-13 tunnel interface’s IP address 192.168.50.13 as
a destination. Spine-1 routes the packet to Border-Leaf-13 based on the outer
IP address. Border-Leaf-13 notices that the destination IP address of the received
IP packet belongs to its’s NVE1 tunnel interface. It removes the outer IP
header and based UDP destination port it notices that this is VXLAN
encapsulated packet. It knows that L3VNI 10077 belongs to VRF NWKT. It strips
off the VXLAN header and routes the packet to vEdge-2. The ingress interface
towards DC in vEdge-2 belongs to VPN 10. vEdge-2 consults its routing table.
Based on it, vEdge-2 constructs tunnel headers and sends ICMP Request to
vEdge-1 via Public-Internet using MPLS Label 1003 as a VPN identifier. Routers
in Internet routes packet based on the outer destination IP address. When
vEdge-1 receives the packet, it notices that the destination IP address is its’
Public IP address. It first removes the outer IP header. Then it checks the
tunnel header. Based on the Label value 1003, it knows that packet belongs to
VPN 10. It consults the VPN 10 RIB and routes the packet to Border-PxTR-13. The
ingress interface on Border-PxTR-13 belongs to VRF 100_NWKT that belongs to
LISP Instance 100. It checks the Instance 100 specific LISP mapping in order to
know how it should route the packet. The LISP mapping Database does not contain
the information because this is the first packet to destination 172.16.100.10.
Border-PxTR-13 sends a LISP Map-Request message to MapSrv-22, which replies
with a LISP Map-Reply message, where it describes the RLOC of Edge-xTR-11 that
has registered the IP address 172.16.100.10. I have excluded the
Map-Request/Reply processes from figure 6-1 to keep the figure simple.
Border-Leaf-13 encapsulates the ICMP Request packet with a tunnel header. It
sets the Instance-Id 100 on the VXLAN header and adds the outer IP header where
it uses the Edge-xTR-11’s IP address 192.168.0.13 as a destination address.
Core-1 routes the packet to Edge-xTR-11 based on the outer IP header
destination address. Edge-xTR-11 processes the ingress IP packet because the
destination IP address belongs to it. Based on the destination UDP port 4789,
it knows that the following header is a VXLAN header. Edge-xTR-11 knows that
the LISP Instance-Id 100 is bind to BD 100. Because Edge-xTR-11 has an L3
interface in BD 100, it resolves the MAC address for the IP address
172.16.100.10 from the ARP table and the egress interface for the MAC from the
MAC address table. EP1 processes the ICMP Request packet and sends the ICMP
Reply to EP3.
Figure
6-1:
End-to-End Data-Plane Operation.
This chapter introduces how Border-PxTR-13 registers
the external IP prefix 172.16.30.0/24 received as a BGP update from vEdge-1 to
MapSrv-22 using LISP Map-register messages. Chapter 2 explains the LISP
RLOC-to-EID mapping process in detail so this chapter just briefly recaps the
operation. Figure 5-1 illustrates the overall process. vEdge-1 sends a BGP
Update message where it describes the NLRI for prefix 172.16.30.0/24.
Border-PxTR-13 first imports the information into the LISP processes. Next, it
sends a LISP Map-Register message to MapSrv-22. In addition to IP prefix
information, the Map-Register message carries Locator Record information that
describes the destination IP address used in the outer IP header (tunnel
header) when devices route IP packets towards the advertised subnet.
Figure 5-1: Overall Control-Plane Operation: OMP to LISP
We have seen in previous chapters how the IP address
172.16.100.10 assigned to EP1 is advertised within the LISP domain and advertised
as an aggregate route all the way down to Leaf-11 in the BGP EVPN domain. This
chapter first explains how the EP3 ‘s IP address 172.16.30.3 is first
advertised by Leaf-11 as BGP EVPN MAC Advertisement Route (Route-Type 2) via
Spine-1 to Border-Leaf-13. Next, you will learn how Border-Leaf-13 advertises
the aggregate route 172.16.30.0/24 to SD-WAN edge device vEdge-2. The last
section briefly shows how the routing information is propagated over the SD-WAN.
The BGP EVPN NLRI MAC Advertisement Route carries to MPLS Labels which identifies L2VN (10000) and L3VN (10077). In our example, VLAN 10 is part of the VRF NWKT
and it is attached to L2VN 10000. L3VNI for VRF NWKT is 10077.
Figure 4-1: Overall Control-Plane Operation: BGP EVPN to OMP to LISP.
Figure 3-1 shows the Overlay Network Control-Plane
interaction. Edge-xTR-11 registers reachability information (IP address and
location) of EP1 to MapSrv-22. MapSrv-22 stores the information into the Mapping
database and then installs it to RIB. Then MapSrv-22 exports the information to
the BGP process and sends BGP Update using VPNv4 address format to
Border-PxTR-13. Border-PxTR-13 imports NLRI into BRIB and RIB. Then it sends
BGP Update to the local SD-WAN edge device vEdge-1. After importing the
received information into BRIB and RIB, vEdge-1 exports the information to the
OMP process and advertises it over a DTLS connection to vSmart (SD-WAN
Control-Plane node). vSmart, in turn, advertises information to remote SD-WAN
device vEdge-2. After importing the received information into the RIB, vEdge-2
exports the information to the BGP process and sends BGP Update to
Border-Leaf-13. Border-Leaf-13 installs the information into BRIB and RIB.
Next, Border-Leaf-13 sends BGP Update message using EVPN route type 5 (IP
Prefix Route) to its iBGP peer Spine-1 (BGP Route-Reflector) using
auto-generated Route-Target 65030:10077. Spine-1 forwards the BGP Update to
Leaf-1, which imports the information into L3VNI used with VRF NWKT and
installs the route into the VRF NWKT RIB.
Figure 3-1: Overall Control-Plane Operation.
Figure 2-1 shows
the Control Plane operation when host EP1 using IP 172.16.100.10/32 joins the
network. In the previous chapter, we saw how Edge-xTR-11 learned the IP address
and registered it to MapSrv-22 by using the LISP Map-register Message where the
Instance-Id 100 represents Virtual Network-Id (VN-Id). MapSrv-22, in turn,
advertised the NLRI to Border-PxTR-13 as BGP VPNv4 Update message where
extended community RT 1:100 (=VN-Id) is used as a kind of VPN identifier (BGP
VPNv4 route import/export policy is based on RT value). This chapter first
explains how Border-PxTR-13 sends BGP IPv4 Update message to local SD-WAN
device vEdge-1. The eBGP peering between Border-PxTR-13 and vEdge1 is VRF based
and BGP updates over it don’t carry any VN-Id. vEdge-1 imports the routing
information from BRIB to RIB. Then it advertises the routing information by
using OMP (Overlay Management Protocol) to the SD-WAN centralized Control Plane
vSmart over the DTLS tunnel using System-IP as an originator-Id and VPN label
1003 as a VN-Id. vSmart forwards this update to SD-WAN device vEdge-2 located
in Datacenter. vEdge-2 installs the route into the routing table. After that,
it exports the routing information from the RIB to the BGP process and sends
the BGP IPv4 Update message to Border-Leaf-13 over VRF NWKT eBGP peering
without VN-Id.
Figure 2-1: Overall Control-Plane Operation.
