VXLAN Part X: Recovery issue when BGP EVPN peering uses the same loopback interface as a source than VXLAN NVE1 interface

Now you can also download my VXLAN book from the Leanpub.com 

"Virtual Extensible LAN VXLAN - A Practical guide to VXLAN Solution Part 1. (373 pages)

Does it really matter if the NVE1 interface of a VTEP switch and BGP EVPN use the same Loopback interface IP address as a source or should there be a dedicated Loopback interface for BGP EVPN? In this post, I am trying to give an answer by showing the difference in BGP EVPN convergence process for both of these design options.

  Figure 10-1: VXLAN BGP EVPN Example Topology and IP addressing

VXLAN Part IX: VXLAN BGP EVPN - vPC

This post describes how the Multi-Chassis Link Aggregation Group (MC-LAG) technology using virtual PortChannel (vPC) works in a VXLAN BGP EVPN fabric. I will first go through the vPC configuration with a short explanation and then I’ll show the Control- and Data Plane operation from VXLAN BGP EVPN perspective by using various show commands and packet capture. I am also going to explain the “Advertising VIP/PIP” options using the external connection. Example topology is shown in Figure 9-1. Complete configurations of vPC peer switches Leaf-102 and Leaf-103 (Leaf-101 and Spine-11 configuration are the same than in the previous post) can be found from the Appendix 1 at the end of the post.

Figure 9-1: VXLAN BGP EVPN vPC Example Topology and IP addressing

VXLAN Part VIII: VXLAN BGP EVPN – External Connection

This post shows how to connect an external network to our existing VXLAN fabric. From the two models, Border Leaf and Border Spine, I am going to use Border Leaf model since I do not want to install additional services to the Spine switches, which already hosts both Multicast Rendezvous Point (RP) and BGP Route Reflector (BGP RR). We could, of course, implement Border to Spine switches without having any performance issue, but then the Spine switches become VTEP switches, which means that they will do a VXLAN encapsulation and decapsulation. Keep it in mind that if we scale out the Spine layer by adding a new Spine switch, we also need to scale out the external connection. With the Border Leaf solutions, we get a dedicated border zone.

I am using full-mesh BGP model instead of a U-shaped model for a couple of reasons, it is the most resilient option, there will be no black holing in event of one link failure and there is no need for iBGP peering between Border Leaf switches.

Figure 8-1 shows the topology which we are going to build.

Figure 8-1: VXLAN Fabric external connection basic setup.

VXLAN Part VII: VXLAN BGP EVPN –Control Plane operation

Now you can also download my VXLAN book from the Leanpub.com 

"Virtual Extensible LAN VXLAN - A Practical guide to VXLAN Solution Part 1. (373 pages)

In my previous post “VXLAN Part VI: VXLAN BGP EVPN – Basic Configurations”, I have shown how to configure the VXLAN BGP EVPN on Nexus 9000v. This post is about BGP EVPN Control Plane operation.

Figure 7-1 represents the logical structure of the example VXLAN fabric. BGP peering is established between the VTEP Leaf switches and the Spine-11 switch, which is BGP Route Reflector (not shown in figure 7-1). Both VTEP Leaf switches have a local VRF context TENANT77 that has VNI 10077 (L3VNI) attached to it and used for routing between the hosts in different vlan/vn-segment. Hosts Café and Beef are connected to vlan 10 (192.168.11.0/24), which in turns is attached to vn-segment 10000 (L2VNI). Hosts Abba and Babe are connected to vlan 20 (192.168.12.0/24), which in turns is attached to vn-segment 20000 (L2VNI). We are using auto-generated RD/RT values and ARP-suppression in both L2VNIs. Physical topology and the configurations of the switches is presented in Appendix 1 at the end of the document. For simplicity, I have used only one uplink in each VTEP switches.

 Figure 7-1: VXLAN Fabric logical structure

VXLAN Part VI: VXLAN BGP EVPN – Basic Configurations

In my previous post “VXLAN Part V: Flood and Learn”, I have shown, how VXLAN works without Control Plane protocol. In this post, I am going to show how to configure BGP EVPN on VXLAN fabric.

In Figure 1, you can see the high-level overview of our example VXLAN fabric design. We have one vrf context (=tenant) TENANT77 spread over the two VTEPs. We also have two VLANs; VLAN 10 (attached to L2VNI 10000) and VLAN 20 (attached to L2VNI 20000). On each VTEPs there are two connected hosts (Cafe and Abba on VTEP-101, Beef, and Babe on VTEP-102). The cross VLAN flows between the hosts in different VTEPs is routed over the L3VNI 10077. The reason why I start with the configurations is that I want to use show commands as well as Wireshark captures while explaining the theory in my next post.

Note! I am using Cisco VIRL with Nexus 9000v (nxos.7.0.3.I7.1.bin).

Figure 1: VXLAN BGP EVPN

Updated: February 21.4.2018 | Toni Pasanen

VXLAN Part V: Flood and Learn

In this chapter, I am going to show how the VXLAN Flood & Learn mac learning process works. I am going to ping from Host-1 to Host-2 and then walk through the Flood and Learn process starting from ARP request. I am using the same Lab that was used in VXLAN Part-IV. Configurations can be found from the VXLAN Part-1 and Part IV.

Figure 1: VXLAN Flood & Learn topology

VXLAN Part IV: The Underlay Network – Multidestination Traffic: PIM BiDir

My Last post, VXLAN Part III, introduces VXLAN Fabric L2VNI service with Anycast-RP PIM (RFC4610 and RFC 7761). In this chapter, I will show how the PIM BiDir (RFC5015) with Phantom-RP can be used for the same purpose. I will use configurations, show commands and Wireshark captures to explain the theory part.

Figure 1: Example VIRL topology

VXLAN Part III: The Underlay Network – Multidestination Traffic: Anycast-RP with PIM

The role of the Underlay Network, related to BUM traffic in the VXLAN fabric, is to transport ARP, ND, DHCP and other Layer 2 BUM (Broadcast, Unknown Unicast, and Multicast) traffic between the hosts connected to different VTEPs. For the Layer 3 Multicast traffic between hosts, there should be an overlay Multicast routing design. This chapter shows how an Anycast-RP with PIM can be used in a VXLAN fabric. In figure 1, we can see our example topology used in this chapter. There are two Spine switches, which shares the same Anycast-RP IP address and belongs to the same “Anycast-RP set” group (Loopback 238). In addition to that, there is an another loopback interface, which must be unique in each Spine (Loopback 511 and 512). These addresses are used as an Anycast-RP group member Id. Both addresses, shared and unique, needs to be reachable for all switches. Complete configuration can be found from the Appendix 1 at the end of the document.

Note! I am using Cisco VIRL with nxos.7.0.3.I7.1

Figure 1: Example topology with Anycast-RP - IP addresses.

VXLAN Part II. The Underlay network – Unicast Routing

Introduction

VXLAN is MAC-over-IP / UDP tunneling mechanism that allows the Layer2 segments to be stretched over the Layer3 network (Underlay/Transport). In this chapter, I will show one possible design of the Underlay network. I will also show basic configurations and monitor commands. At the end of this article, you can find a mindmap for memory builder.

Our example network consists of four Cisco Nexus 9000 switches. The edge switches Leaf-101 and Leaf-102 works as a VTEP (VXLAN Tunnel Endpoint) devices. VTEPs are responsible for encapsulation of Ethernet frames received from directly connected hosts with VXLAN header as well as removing VXLAN header from the packet received from another VTEP switch. Spine-11 and Spine-12 are the core switches. These switches are not aware of hosts/VMs behind the VTEP Leaf switches, Spine switches only route packet between VTEP switches.

Figure-1: Example topology

VXLAN Part I. Why do we need VXLAN?

Introduction

This section examines the challenges that virtualization of servers causes for Datacenter networks with traditional three-layer architecture and how VXLAN can respond to these challenges. At the end of this article, you can find a mindmap for memory builder.

Challenges for existing Datacenter networks

Figure 1-1 shows a hypothetical 3-tier Cloud Service Provider DC network consisting of the following components.

• Access layer (L2): Twenty of the 48-port switches. Access - Distribution links 2 x 10Gbps MEC (Multichassis EtherChannel).
• Distribution layer (L2 / L3): Two distribution switches, which together form a virtualized switch. The default gateway for server segments is in distribution switches. Distribution - Core Links are L3.
• Core Layer (L3): Two Core switches

Figure 1-1: The hypothetical Cloud SP Datacenter network.

Rapid per VLAN Spanning Tree protocol (Rapid PVST+): Synchronization

Introduction

This article describes the Rapid per VLAN Spanning Tree Protocol (Rapid PVST +) synchronization process based on Proposal / Agreement messages. The purpose of the synchronization is to form a new loop-free layer 2 topology as a response to changes in stable STP domains. The synchronization process is started by a switch that loses its connection to a current STP root switch and it does not have an STP Alternate port.

The article also describes the topology change process that occurs when the STP non-Edge port status changes from Discarding/Learning state to Forwarding state.

The idea for writing this article started when I studied the new Cisco VXLAN-based Campus Fabric model and I wanted to compare its complexity with the complexity of the traditional Campus network.

Figure 1-1: Spanning Tree topology

Border Gateway Protocol – Finite State Machine (BGP-FSM)

Introduction

The BGP-FSM model presents different states during BGP neighbor negotiation: Idle, Connect, Active, OpenSent, OpenConfirm and Established. It also describes the events that launch changes from one state to another. This article is based on Based on RFC 4271 – A Border Gateway Protocol 4 (BGP-4).

There are 28 events, which are grouped into four categories.

• Administrative Events: Events that either start or stop the process.
• Timer Events: Expiration of HoldTimer, KeepaliveTimer or ConnectRetryTimer trigger this event.
• TCP connection based events: These events describe the different state of the TCP three-way handshake.
• BGP message based events: Receiving or sending BGP messages: OPEN, KEEPALIVE, UPDATE and NOTIFICATION launch this event.

A Complete list can be found in Appendix A.

Figure 1-1 shows the map of different states and related event numbers. Timers and counters are excluded from both the text part and picture to keep things as clear as possible.

Figure 1-1: Overview of BGP-FSM.