AI for Network Engineers: Backpropagation Algorithm

 Introduction 

This chapter introduces the training model of a neural network based on the Backpropagation algorithm. The goal is to provide a clear and solid understanding of the process without delving deeply into the mathematical formulas, while still explaining the fundamental operations of the involved functions. The chapter also briefly explains why, and in which phases the training job generates traffic to the network, and why lossless packet transport is required. The Backpropagation algorithm is composed of two phases: the Forward pass (computation phase) and the Backward pass (adjustment and communication phase).

In the Forward pass, neurons in the first hidden layer calculate the weighted sum of input parameters received from the input layer, which is then passed to the neuron's activation function. Note that neurons in the input layer are not computational units; they simply pass the input variables to the connected neurons in the first hidden layer. The output from the activation function of a neuron is then used as input for the connected neurons in the next layer. The result of the activation function in the output layer represents the model's prediction, which is compared to the expected value (ground truth) using the error function. The output of the error function indicates the accuracy of the current training iteration. If the result is sufficiently close to the expected value (error function close to zero), the training is complete. Otherwise, it triggers the Backward pass process.

As the first step in the backward pass, the backpropagation algorithm calculates the derivative of the error function, providing the output error (gradient) of the model. Next, the algorithm computes the error term (gradient) for the neuron(s) in the output layer by multiplying the derivative of each neuron’s activation function by the model's error term. Then, the algorithm moves to the preceding layer and calculates the error term (gradient) for its neuron(s). This error term is now calculated using the error term of the connected neuron(s) in the next layer, the derivative of each neuron’s activation function, and the value of the weight parameter associated with the connection to the next layer.

After calculating the error terms, the algorithm determines the weight adjustment values for all neurons simultaneously. This computation is based on the input values, the adjustment values, and a user-defined learning rate. Finally, the backpropagation algorithm refines all weight values by adding the adjustment values to the initial weights. Once the backward pass is complete, the backpropagation algorithm starts a new iteration of the forward pass, gradually improving the model's predictions until they closely match the expected values, at which point the training is complete.

Figure 2-1: Backpropagation Overview.

The First Iteration - Forward Pass

Training a model often requires multiple iterations of forward and backward passes. In the forward pass, neurons in the first hidden layer calculate the weighted sum of input values, each multiplied by its associated weight parameter. These neurons then apply an activation function to the result. Neurons in subsequent layers use the activation output from previous layers as input for their own weighted sum calculations. This process continues through all the layers until reaching the output layer, where the activation function produces the model's prediction.

After the forward pass, the backpropagation algorithm calculates the error by comparing the model's output with the expected value, providing a measure of accuracy. If the model's output is close to the expected value, training is complete. Otherwise, the backpropagation algorithm initiates the backward pass to adjust the weights and reduce the error in subsequent iterations.

 

Neuron-a Forward Pass Calculations

Weighted Sum

In Figure 2-2, we have an imaginary training dataset with three inputs and a bias term. Input values and their respective initial weight values are listed below: 

• x1 = 0.2 , initial weight wa1 = 0.1

• x2 = 0.1, initial weight wa2 = 0.2

• x3 = 0.4 , initial weight wa3 = 0.3

• ba0 = 1.0 , initial weight wa0 = 0.6

From the model training perspective, the input values are constant, unchageable values, while weight values are variables which will be refined during the backward pass process.

The standard way to write the weighted sum formula is: 

Where:

• n = 3 represents the number of input values (x1, x2, x3).

• Each input xi  is multiplied by its respective weight wi, and the sum of these products is added to the bias term b.

In this case, the equation can be explicitly stated as:

Which with our parameters gives:

Activation Function

Neuron-a uses the previously calculated weighted sum as input for the activation function. We are using the ReLU function (Rectified Linear Unit), which is more popular than the hyperbolic tangent and sigmoid functions due to its simplicity and lower computational cost.

The standard way to write the ReLU function is:

Where:

f(a) represents the activation function.

z  is the weighted sum of inputs.

The ReLU function returns the z if z > 0. Otherwise, it returns 0 if z ≤ 0.

In our example, the weighted sum za is 0.76, so the ReLU function returns:

Figure 2-2: Activation Function for Neuron-a.

 

Neuron-b Forward Pass Calculations

Weighted Sum

Besides the bias term value of 1.0,  Neuron-b uses the result provided by the activation function of neuron-a as an input to weighted sum calculation. Input values and their respective initial weight values are listed below: 

This gives us:

Activation Function

Just like Neuron-a, Neuron-b uses the previously calculated weighted sum as input for the activation function. Because the zb = 0.804 is greater than zero, the ReLU activation function f(b) returns:

Neuron-b is in the output layer, so its activation function result y_b represents the prediction of the model. 

 

Figure 2-3: Activation Function for Neuron-b.

Error Function

To keep things simple, we have used only one training example. However, in real-life scenarios, there will always be more than one training example. For instance, a training dataset might contain 10 images of cats and 10 images of dogs, each having 28x28 pixels. Each image represents a training example, giving us a total of 20 training examples. The purpose of the error function is to provide a single error metric for all training examples. In this case, we are using the Mean Squared Error (MSE).

We can calculate the MSE using the formula below where the expected value y is 1.0 and the model’s  prediction for the training example yb = 0.804. This gives an error metric of 0.019, which can be interpreted as an indicator of the model's accuracy.

The result of the error function is not sufficiently close to the desired value, which is why this result triggers the backward pass process.

Figure 2-4: Calculating the Error Function for Training Examples.

 

Backward Pass

In the forward pass, we calculate the model’s accuracy using several functions. First, Neuron-a computes the weighted sum Σ(za ) by multiplying the input values and the bias term with their respective weights. The output, za, is then passed through the activation function f(a), producing ya. Neuron-b, in turn, takes ya and the bias term to calculate the weighted sum Σ(zb ). The activation function f(b) then uses zb to compute the model’s output, yb. Finally, the error function f(e) calculates the model’s accuracy based on the output.

So, dependencies between function can be seen as:

The backpropagation algorithm combines these five functions to create a new error function, enew(x), using function composition and the chain rule. The following expression shows how the error function relates to the weight parameter w1 used by Neuron-a:

This can be expressed using the composition operator (∘) between functions:

Next, we use a method called gradient descent to gradually adjust the initial weight values, refining them to bring the model's output closer to the expected result. To do this, we compute the derivative of the composite function using the chain rule, where we take the derivatives of:

1. The error function (e) with respect to the activation function (b).

2. The activation function b with respect to the weighted sum (zb). 

3. The weighted sum (zb) with respect to the activation function (a).

4. The activation function (a) with respect to weighted sum (za(w1)). 

In Leibniz’s notation, this looks like:

Figure 2-5 illustrates the components of the backpropagation algorithm, along with their relationships and dependencies.

Figure 2-5: The Backward Pass Overview.

Partial Derivative for Error Function – Output Error (Gradient)

As a recap, and for illustrating that the prediction of the first iteration fails, Figure 2-6 includes the computation for the error function (MSE = 0.019). 

As a first step, we calculate the partial derivative of the error function. In this case, the partial derivative describes the rate of change of the error function when the input variable yb changes. The derivative is called partial when one of its input values is held constant (i.e., not adjusted by the algorithm). In our example, the expected value y is constant input. The result of the partial derivative of the error function describes how the predicted output should change yb to minimize the model’s error.

We use the following formula for computing the derivative of the error function:

Figure 2-6: The Backward Pass – Derivative of the Error Function.

The following explanation is meant for readers interested in why there is a minus sign in front of the function.

When calculating the derivative, we use the Power Rule. The Power Rule states that if we have a function f(x) = xn , then its derivative is f’(x) = n ⋅ xn-1. In our case, this applies to the error function:

Using the Power Rule, the derivative becomes:

Next, we apply the chain rule by multiplying this result by the derivative of the inner function (y − yb), with respect to yb . Since y is treated as a constant (because it represents our target value, which doesn't change during optimization), the derivative of (y − yb) with respect to yb  is simply −1, as the derivative of − yb  with respect to yb  is −1, and the derivative of y (a constant) is 0.

Therefore, the final derivative of the error function with respect to yb  is:

Partial Derivative for the Activation Function

After computing the output error, we calculate the derivative of the activation function f(b) with respect to zb . Neuron b uses the ReLU activation function, which states that if the input to the function is greater than 0, the derivative is 1; otherwise, it is 0. In our case, the result of the activation function f(b)=0.804, so the derivative is 1.

Error Term for Neurons (Gradient)

The error term (Gradient) for neuron-b is calculated by multiplying the output error, the partial derivative of the error function,  by the derivative of the neuron's activation function. This means that now we propagate the model's error backward using it as a base value for finetuning the model accuracy (i.e., refining new weight values). This is why the term backward pass fits perfectly for the process.

Figure 2-7: The Backward Pass – Error Term (Gradient) for Neuron-b.

After computing the error term for Neuron-b, the backward pass moves to the preceding layer, the hidden layer, and calculates the error term for Neuron-a. The algorithm computes the derivative for the activation function f(a) = 1, as it did with the Neuron-b. Next, it multiplies the result by Neuron-b's error term (-0.196) and the connected weight parameter , wb1 =0.4. The result -0.0784 is the error term for Neuron-a.

Figure 2-8: The Backward Pass – Error Term (Gradient) for Neuron-a.

Weight Adjustment Value

After computing error terms for all neurons in every layer, the algorithm simultaneously calculates the weight adjustment value for every weight. The process is simple, the error term is multiplied with an input value connected to weight and with learning rate (η). The learning rate balances convergence speed and training stability. We have set it to -0.6 for the first iteration. The learning rate is a hyper-parameter, meaning it is set by the user rather than learned by the model during training. It affects the behavior of the backpropagation algorithm by controlling the size of the weight updates. It is also possible to adjust the learning rate during training—using a higher learning rate at the start to allow faster convergence and lowering it later to avoid overshooting the optimal result. 

 

Weight adjustment value for weight wb1 and wa1 respectively:

Note! It is not recommended to use a negative learning rate. I use it here because we get a good enough output for the second forward pass iteration.

Figure 2-9: The Backward Pass – Weight Adjustment Value for Neurons.

Refine Weights

As the last step, the backpropagation algorithm computes new values for every weight parameter in the model by simply summing the initial weight value and weight adjustment value.

New values for weight  parameters wb1 and wa1 respectively:

Figure 2-10: The Backward Pass – Compute New Weight Values.

The Second Iteration - Forward Pass

After updating all the weight values (wa0, wa1, wa2, and wa3 ), the backpropagation process begins the second iteration of the forward pass. As shown in Figure 2-11, the model output yb = 0.9982 is very close to the expected value y = 1.0. The new MSE = 0.0017, is much better than 0.019 computed in the first iteration.

Figure 2-11: The Second Iteration of the Forward Pass.

Network Impact

Figure 2-12 shows a hypothetical example of Data Parallelization, where our training data set is split into two batches, A and B, which are processed by GPU-A and GPU-B, respectively. The training model is the same on both GPUs: Fully-Connected, with two hidden layers of four neurons each, and one output neuron in the output layer.

After computing a model prediction during the forward pass, the backpropagation algorithm begins the backward pass by calculating the gradient (output error) for the error function. Once computed, the gradients are synchronized between the GPUs. The algorithm then averages the gradients, and the process moves to the preceding layer. Neurons in the preceding layer calculate their gradient by multiplying the weighted sum of their connected neurons’ averaged gradients and connected weight with the local activation function’s partial derivative. These neuron-based gradients are then synchronized over connections. Before the process moves to the preceding layer, gradients are averaged. The backpropagation algorithm executes the same process through all layers. 

If packet loss occurs during the synchronization, it can ruin the entire training process, which would need to be restarted unless snapshots were taken. The cost of losing even a single packet could be enormous, especially if training has been ongoing for several days or weeks. Why is a single packet so important? If the synchronization between the gradients of two parallel neurons fails due to packet loss, the algorithm cannot compute the average, and the neurons in the preceding layer cannot calculate their gradient. Besides, if the connection, whether the synchronization happens over NVLink, InfiniBand, Ethernet (RoCE or RoCEv2), or wireless connection, causes a delay, the completeness of the training slows down. This causes GPU under-utilization which is not efficient from the business perspective.

Figure 2-12: Backward Pass – Gradient Synchronization and Averaging.

To be conntinued...

AI for Network Engineers: Chapter 2 - Backpropagation Algorithm: Introduction

This chapter introduces the training model of a neural network based on the Backpropagation algorithm. The goal is to provide a clear and solid understanding of the process without delving deeply into the mathematical formulas, while still explaining the fundamental operations of the involved functions. The chapter also briefly explains why, and in which phases the training job generates traffic to the network, and why lossless packet transport is required. The Backpropagation algorithm is composed of two phases: the Forward pass (computation phase) and the Backward pass (adjustment and communication phase).

In the Forward pass, neurons in the first hidden layer calculate the weighted sum of input parameters received from the input layer, which is then passed to the neuron's activation function. Note that neurons in the input layer are not computational units; they simply pass the input variables to the connected neurons in the first hidden layer. The output from the activation function of a neuron is then used as input for the connected neurons in the next layer, whether it is another hidden layer or the output layer. The result of the activation function in the output layer represents the model's prediction, which is compared to the expected value (ground truth) using the error function. The output of the error function indicates the accuracy of the current training iteration. If the result is sufficiently close to the expected value, the training is complete. Otherwise, it triggers the Backward pass process.

In the Backward pass process, the Backpropagation algorithm first calculates the derivative of the error function. This derivative is then used to compute the error term for each neuron in the model. Neurons use their calculated error terms to determine how much and in which direction the current weight values must be fine-tuned. Depending on the model and the parallelization strategy, GPUs in multi-GPU clusters synchronize information during the Backpropagation process. This process affects network utilization.

Our feedforward neural network, shown in Figure 2-1, has one hidden layer and one output layer. If we wanted our example to be a deep neural network, we would need to add additional layers, as the definition of "deep" requires two or more hidden layers. For simplicity, the input layer is not shown in the figure.

We have three input parameters connected to neuron-a in the hidden layer as follows:

• Input X1 = 0.2 > neuron-a via weight Wa1 = 0.1

• Input X2 = 0.1 > neuron-a via weight Wa2 = 0.2

• Input X3 = 0.4 > neuron-a via weight Wa3 = 0.3

• Bias ba0 = 1.0 > neuron-a via weight Wa0 = 0.6

The bias term helps ensure that the neuron is active, meaning its output value is not zero.

The input parameters are treated as constant values, while the weight values are variables that will be adjusted during the Backward pass if the training result does not meet expectations. The initial weight values are our best guess for achieving the desired training outcome. The result of the weighted sum calculation is passed to the activation function, which provides the input for neuron-b in the output layer. We use the ReLU (Rectified Linear Unit) activation function in both layers due to its simplicity. There are other activation functions, such as hyperbolic tangent (tanh), sigmoid, and softmax, but those are outside the scope of this chapter.

The input values and weights for neuron-b are:

• Neuron-a activation function output f(af) > neuron-b via weight Wb1

• Bias ba0 = 1.0 > neuron-b via weight Wa0 = 0.5

The output, Ŷ, from neuron-b represents our feedforward neural network's prediction. This value is used along with the expected result, y, as input for the error function. In this example, we use the Mean Squared Error (MSE) error function. As we will see, the result of the first training iteration does not match our expected value, leading us to initiate the Backward pass process.

In the first step of the Backward pass, the Backpropagation algorithm calculates the derivative of the error function (MSE’). Neurons-a and b use this result as input to compute their respective error terms by multiplying MSE’ with the result of the activation function and the weight value associated with the connection to the next neuron. Note that for neuron-b, there is no next layer—just the error function—so the weight parameter is excluded from the error term calculation of neuron-b. Next, the error term value is multiplied by an input value and learning rate, and this adjustment value is added to the current weight.

After completing the Backward pass, the Backpropagation algorithm starts a new iteration of the Forward pass, gradually improving the model's prediction until it closely matches the expected value, at which point the training is complete.

Figure 2-1: Backpropagation Algorithm.

Artificial Intelligence for Network Engineers: Introduction

Several books on artificial intelligence (AI) and deep learning (DL) have been published over the past decade. However, I have yet to find a book that explains deep learning from a networking perspective while providing a solid introduction to DL. My goal is to fill this gap by writing a book titled AI for Network Engineers (note that the title name may change during the writing process). Writing about such a complex subject will take time, but I hope to complete and release it within a year.

Part I: Deep Learning and Deep Neural Networks

The first part of the book covers the theory behind Deep Learning. It begins by explaining the construct of a single artificial neuron and its functionality. Then, it explores various Deep Neural Network models, such as Feedforward Neural Networks (FNN), Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN). Next, the first part discusses data and model parallelization strategies such as Data, Pipeline, and Tensor Parallelism, explaining how input data and/or model sizes that exceed the memory capacity of GPUs within a single server can be distributed across multiple GPU servers.

Part II: AI Data Center Networking - Lossless Ethernet

After a brief introduction of RoCEv2, the second part continues from part one by explaining how parallelization strategies affect network utilization. It then discusses the Data Center Quantized Congestion Notification (DCQCN) scheme for RoCEv2, introducing key concepts such as Explicit Congestion Notification (ECN) and Priority-based Flow Control (PFC). In addition to ECN and PFC, this section covers other congestion-avoidance methods, such as packet spraying and deep buffers. The second part also delves into AI data center design choices focusing on the East-West backend network. It introduces Rail, Top-of-Rack (ToR), and Rail-Optimized designs.

Figure 1: Book Introduction.

AI for Network Engineers: Chapter 1 - Deep Learning Basics

Content

Introduction 
Artificial Neuron 
  Weighted Sum for Pre-Activation Value 
  ReLU Activation Function for Post-Activation 
  Bias Term
  S-Shaped Functions – TANH and SIGMOID
Network Impact
Summary
References

Introduction

Artificial Intelligence (AI) is a broad term for solutions that aim to mimic the functions of the human brain. Machine Learning (ML), in turn, is a subset of AI, suitable for tasks like simple pattern recognition and prediction. Deep Learning (DL), the focus of this section, is a subset of ML that leverages algorithms to extract meaningful patterns from data. Unlike ML, DL does not necessarily require human intervention, such as providing structured, labeled datasets (e.g., 1,000 bird images labeled as “bird” and 1,000 cat images labeled as “cat”). 

DL utilizes layered, hierarchical Deep Neural Networks (DNNs), where hidden and output layers consist of computational units, artificial neurons, which individually process input data. The nodes in the input layer pass the input data to the first hidden layer without performing any computations, which is why they are not considered neurons or computational units. Each neuron calculates a pre-activation value (z) based on the input received from the previous layer and then applies an activation function to this value, producing a post-activation output (ŷ) value. There are various DNN models, such as Feed-Forward Neural Networks (FNN), Convolutional Neural Networks (CNN), and Recurrent Neural Networks (RNN), each designed for different use cases. For example, FNNs are suitable for simple, structured tasks like handwritten digit recognition using the MNIST dataset [1], CNNs are effective for larger image recognition tasks such as with the CIFAR-10 dataset [2], and RNNs are commonly used for time-series forecasting, like predicting future sales based on historical sales data. 

To provide accurate predictions based on input data, neural networks are trained using labeled datasets. The MNIST (Modified National Institute of Standards and Technology) dataset [1] contains 60,000 training and 10,000 test images of handwritten digits (grayscale, 28x28 pixels). The CIFAR-10 [2] dataset consists of 60,000 color images (32x32 pixels), with 50,000 training images and 10,000 test images, divided into 10 classes. The CIFAR-100 dataset [3], as the name implies, has 100 image classes, with each class containing 600 images (500 training and 100 test images per class). Once the test results reach the desired level, the neural network can be deployed to production.

Figure 1-1: Deep Learning Introduction.

AI/ML Networking: Part-IV: Convolutional Neural Network (CNN) Introduction

Feed-forward Neural Networks are suitable for simple tasks like basic time series prediction without long-term relationships. However, FNNs is not a one-size-fits-all solution. For instance, digital image training process uses pixel values of image as input data. Consider training a model to recognize a high resolution (600 dpi), 3.937 x 3.937 inches digital RGB (red, green, blue) image. The number of input parameters can be calculated as follows:

Width: 3.937 in x 600 ≈ 2362 pixels
Height: 3.937 in x 600 ≈ 2362 pixels
Pixels in image: 2362 x 2362 = 5,579,044 pixels
RGB (3 channels): 5,579,044 pxls x 3 channels = 16 737 132
Total input parameters: 16 737 132
Memory consumption: ≈ 16 MB

FNNs are not ideal for digital image training. If we use FNN for training in our example, we fed 16,737,132 input parameters to the first hidden layer, each having unique weight. For image training, there might be thousands of images, handling millions of parameters demands significant computation cycles and is a memory-intensive process. Besides, FNNs treat each pixel as an independent unit. Therefore, FNN algorithm does not understand dependencies between pixels and cannot recognize the same image if it shifts within the frame. Besides, FNN does not detect edges and other crucial details. 

A better model for training digital images is Convolutional Neural Networks (CNNs). Unlike in FFN neural networks where each neuron has a unique set of weights, CNNs use the same set of weights (Kernel/Filter) across different regions of the image, which reduces the number of parameters. Besides, CNN algorithm understands the pixel dependencies and can recognize patterns and objects regardless of their position in the image. 

The input data processing in CNNs is hierarchical. The first layer, convolutional layers, focuses on low-level features such as textures and edges. The second layer, pooling layer, captures higher-level features like shapes and objects. These two layers significantly reduce the input data parameters before they are fed into the neurons in the first hidden layer, the fully connected layer, where each neuron has unique weights (like FNNs).

AI/ML Networking: Part-III: Basics of Neural Networks Training Process

Neural Network Architecture Overview

Deep Neural Networks (DNN) leverage various architectures for training, with one of the simplest and most fundamental being the Feedforward Neural Network (FNN). Figure 2-1 illustrates our simple, three-layer FNN.

Input Layer: 

The first layer doesn’t have neurons, instead the input data parameters X1, X2, and X3 are in this layer, from where they are fed to first hidden layer. 

Hidden Layer: 

The neurons in the hidden layer calculate a weighted sum of the input data, which is then passed through an activation function. In our example, we are using the Rectified Linear Unit (ReLU) activation function. These calculations produce activation values for neurons. The activation value is modified input data value received from the input layer and published to upper layer.

Output Layer: 

Neurons in this layer calculate the weighted sum in the same manner as neurons in the hidden layer, but the result of the activation function is the final output.

The process described above is known as the Forwarding pass operation. Once the forward pass process is completed, the result is passed through a loss function, where the received value is compared to the expected value. The difference between these two values triggers the backpropagation process. The Loss calculation is the initial phase of Backpropagation process. During backpropagation, the network fine-tunes the weight values , neuron by neuron, from the output layer through the hidden layers. The neurons in the input layer do not participate in the backpropagation process because they do not have weight values to be adjusted.

After the backpropagation process, a new iteration of the forward pass begins from the first hidden layer. This loop continues until the received and expected values are close enough to expected value, indicating that the training is complete.

Figure 2-1: Deep Neural Network Basic Structure and Operations.

Forwarding Pass 

Next, let's examine the operation of a Neural Network in more detail. Figure 2-2 illustrates a simple, three-layer Feedforward Neural Network (FNN) data model. The input layer has two neurons, H1 and H2, each receiving one input data value: a value of one (1) is fed to neuron H1 by input neuron X1, and a value of zero (0) is fed to neuron H2 by input neuron X2. The neurons in the input layer do not calculate a weighted sum or an activation value but instead pass the data to the next layer, which is the first hidden layer.

The hidden layer in our example consists of two neurons. These neurons use the ReLU activation function to calculate the activation value. During the initialization phase, the weight values for these neurons are assigned using the He Initialization method, which is often used with the ReLU function. The He Initialization method calculates the variance as 2/n where n is the number of neurons in the previous layer. In this example, with two input neurons, this gives a variance of  1 (=2/2). The weights are then drawn from a normal distribution ~N(0,√variance), which in this case is  ~N(0,1). Basically, this means that the randomly generated weight values are centered around zero with a standard deviation of one.

In Figure 2-2, the weight value for neuron H3 in the hidden layer is 0.5 for both input sources X1 (input data 1) and X2 (input data 0). Similarly, for the hidden layer neuron H4, the weight value is 1 for both input sources X1 (input data 1) and X2 (input data 0). Neurons in the hidden and output layers also have a bias variable. If the input to a neuron is zero, the output would also be zero if there were no bias. The bias ensures that a neuron can still produce a meaningful output even when the input is zero (i.e., the neuron is inactive). Neurons H3 and O5 have a bias value of 0.5, while neuron H4 has a bias value of 0 (I am using zero for simplify the calculation). 

Let’s start the forward pass process from neuron H3 in the hidden layer. First, we calculate the weighted sum using the formula below, where Z3 represents the weighted sum of input. Here, Xn is the actual input data value received from the input layer’s neuron, and Wn  is the weight associated with that particular input neuron.

The weighted sum calculation (Z3) for neuron H3:

Z3 = (X1 ⋅ W31) + (X2 ⋅ W32) + b3

Given:

Z3 = (1 ⋅ 0.5) + (0 ⋅ 0.5) + 0

Z3 = 0.5 + 0 + 0

Z3 = 0.5

To get the activation value a3 (shown as H3=0.5 in figure), we apply the ReLU function. The ReLU function outputs zero (0) if the calculated weighted sum Z is less than or equal to zero; otherwise, it outputs the value of the weighted sum Z.

The activation value a3 for H3 is:

ReLU (Z3) = ReLU (0.5) = 0.5

The weighted sum calculation for neuron H4:

Z4 = (X1 ⋅ W41) + (X2 ⋅ W42) + b4

Given:

Z4 = (1 ⋅ 1) + (0 ⋅1) + 0.5

Z4 = 1 + 0 + 0.5

Z4 = 1.5

The activation value using ReLU for Z4 is:

ReLU (Z4) = ReLU (1.5) = 1.5

 

Figure 2-2: Forwarding Pass on Hidden Layer.

After neurons H3 and H4 publish their activation values to neuron O5 in the output layer, O5 calculates the weighted sum Z5 for inputs with weights W53=1and W54=1. Using Z5, it calculates the output using the ReLU function. The difference between the received output value (Yr) and the expected value (Ye) triggers a backpropagation process. In our example, Yr−Ye=0.5.

Backpropagation process

The loss function measures the difference between the predicted output and the actual expected output. The loss function value indicates how well the neural network is performing. A high loss value means the network's predictions are far from the actual values, while a low loss value means the predictions are close.

After calculating the loss, backpropagation is initiated to minimize this loss. Backpropagation involves calculating the gradient of the loss function with respect to each weight and bias in the network. This step is crucial for adjusting the weights and biases to reduce the loss in subsequent forwarding pass iterations.

Loss function is calculated using the formula below:

Loss (L) = (H3 x W53 + H4 x W54 + b5 – Ye)2

Given:

L = (0.5 x 1 + 1.5 x 1 + 0.5 - 2)2

L = (0.5 + 1.5 + 0.5 - 2)2

L = 0.52

L= 0.25

 

Figure 2-3: Forwarding Pass on Output Layer.

The result of the loss function is then fed into the gradient calculation process, where we compute the gradient of the loss function with respect to each weight and bias in the network. The gradient calculation result is then used to fine-tune the old weight values. The Eta hyper-parameter η (the learning rate) controls the step size during weight updates in the backpropagation process, balancing the speed of convergence with the stability of training. In our example, we are using a learning rate of 1/100 = 0.01. The term hyper-parameters refers to parameters that affect the final result.

First, we compute the partial derivative of the loss function (gradient calculation) with respect to the old weight values. The following example shows the gradient calculation for weight W53. The same computation applies to W54  and b3.

Gradient Calculation:

∂L   = 2W53 x (Yr – Ye)

∂W53

 Given

 = 2 x 0.5 x (2.5 - 2)

 = 1 x 0.5

 = 0.5

 

New weight value calculation.

W53 (new) = W53(old) – η x ∂L/∂W53

Given:

W53 (new) = 1–0.01 x 0.5

W53 (new) = 0.995

 

Figure 2-4: Backpropagation - Gradient Calculation and New Weight Value Computation.

Figure 2-5 shows the formulas for calculating the new bias b3. The process is the same than what was used with updating the weight values.

Figure 2-5: Backpropagation - Gradient Calculation and New Bias Computation.

After updating the weights and biases, the backpropagation process moves to the hidden layer. Gradient computation in the hidden layer is more complex because the loss function only includes weights from the output layer as you can see from the Loss function formula below:

Loss (L) = (H3 x W53 + H4 x W54 + b5 – Ye)2

The formula for computing the weights and biases for neurons in the hidden layers uses the chain rule. The mathematical formula shown below, but the actual computation is beyond the scope of this chapter.

∂L   =    ∂L  x  ∂H3    

∂W31   ∂H3    ∂W31    

After the backpropagation process is completed, the next iteration of the forward pass starts. This loop continues until the received result is close enough to the expected result.

If the size of the input data exceeds the GPU’s memory capacity or if the computing power of one GPU is insufficient for the data model, we need to decide on a parallelization strategy. This strategy defines how the training workload is distributed across several GPUs. Parallelization impacts network load if we need more GPUs than are available on one server. Dividing the workload among GPUs within a single GPU-server or between multiple GPU-servers triggers synchronization of calculated gradients between GPUs. When the gradient is calculated, the GPUs synchronize the results and compute the average gradient, which is then used to update the weight values.

The upcoming chapter introduces pipeline parallelization and synchronization processes in detail. We will also discuss why lossless connection is required for AI/ML.

AI/ML Networking: Part-II: Introduction of Deep Neural Networks

Machine Learning (ML) is a subset of Artificial Intelligence (AI). ML is based on algorithms that allow learning, predicting, and making decisions based on data rather than pre-programmed tasks. ML leverages Deep Neural Networks (DNNs), which have multiple layers, each consisting of neurons that process information from sub-layers as part of the training process. Large Language Models (LLMs), such as OpenAI’s GPT (Generative Pre-trained Transformers), utilize ML and Deep Neural Networks.

For network engineers, it is crucial to understand the fundamental operations and communication models used in ML training processes. To emphasize the importance of this, I quote the Chinese philosopher and strategist Sun Tzu, who lived around 600 BCE, from his work The Art of War.

If you know the enemy and know yourself, you need not fear the result of a hundred battles.

We don’t have to be data scientists to design a network for AI/ML, but we must understand the operational fundamentals and communication patterns of ML. Additionally, we must have a deep understanding of network solutions and technologies to build a lossless and cost-effective network for enabling efficient training processes.

In the upcoming two posts, I will explain the basics of: 

a) Data Models: Layers and neurons, forward and backward passes, and algorithms. 

b) Parallelization Strategies: How training times can be reduced by dividing the model into smaller entities, batches, and even micro-batches, which are processed by several GPUs simultaneously.

The number of parameters, the selected data model, and the parallelization strategy affect the network traffic that crosses the data center switch fabric.

After these two posts, we will be ready to jump into the network part. 

At this stage, you may need to read (or re-read) my previous post about Remote Direct Memory Access (RDMA), a solution that enables GPUs to write data from local memory to remote GPUs' memory.

AI/ML Networking Part I: RDMA Basics

Remote Direct Memory Access - RDMA Basics

Introduction

Remote Direct Memory Access (RDMA) architecture enables efficient data transfer between Compute Nodes (CN) in a High-Performance Computing (HPC) environment. RDMA over Converged Ethernet version 2 (RoCEv2) utilizes a routed IP Fabric as a transport network for RDMA messages. Due to the nature of RDMA packet flow, the transport network must provide lossless, low-latency packet transmission. The RoCEv2 solution uses UDP in the transport layer, which does not handle packet losses caused by network congestion (buffer overflow on switches or on a receiving Compute Node). To avoid buffer overflow issues, Priority Flow Control (PFC) and Explicit Congestion Notification (ECN) are used as signaling mechanisms to react to buffer threshold violations by requesting a lower packet transfer rate.

Before moving to RDMA processes, let’s take a brief look at our example Compute Nodes. Figure 1-1 illustrates our example Compute Nodes (CN). Both Client and Server CNs are equipped with one Graphical Processing Unit (GPU). The GPU has a Network Interface Card (NIC) with one interface. Additionally, the GPU has Device Memory Units to which it has a direct connection, bypassing the CPU. In real life, a CN may have several GPUs, each with multiple memory units. Intra-GPU communication within the CN happens over high-speed NVLinks. The connection to remote CNs occurs over the NIC, which has at least one high-speed uplink port/interface.

Figure 1-1 also shows the basic idea of a stacked Fine-Grained 3D DRAM (FG-DRAM) solution. In our example, there are four vertically interconnected DRAM dies, each divided into eight Banks. Each Bank contains four memory arrays, each consisting of rows and columns that contain memory units (transistors whose charge indicates whether a bit is set to 1 or 0). FG-DRAM enables cross-DRAM grouping into Ranks, increasing memory capacity and bandwidth.

The upcoming sections introduce the required processes and operations when the Client Compute Node wants to write data from its device memory to the Server Compute Node’s device memory. I will discuss the design models and requirements for lossless IP Fabric in later chapters.

Figure 1-1: Fine-Grained DRAM High-Level Architecture.

BGP EVPN Fabric - Remote Leaf MAC Learning Process

Remote VTEP Leaf-102: Low-Level Control Plane Analysis

In this section, we will first examine the update process of the BGP tables on the VTEP switch Leaf-102 when it receives a BGP Update message from Spine-11. After that, we will go through the update processes for the MAC-VRF and the MAC Address Table. Finally, we will examine how the VXLAN manager on Leaf-102 learns the IP address of Leaf-10's NVE interface and creates a unidirectional NVE peer record in the NVE Peer Database based on this information.

Remote Learning: BGP Processes

We have configured switches Leaf-101 and Leaf-102 as Route Reflector Clients on the Spine-11 switch. Spine-11 has stored the content of the BGP Update message sent by Leaf-101 in the neighbor-specific Adj-RIB-In of Leaf-101. Spine-11 does not import this information in its local BGP Loc-RIB because we have not defined a BGP import policy. Since Leaf-102 is an RR Client, the BGP process on Spine-11 copies this information in the neighbor-specific Adj-RIB-Out table for Leaf-102 and sends the information to Leaf-102 in a BGP Update message. The BGP process on Leaf-102 stores the received information from the Adj-RIB-In table to the BGP Loc-RIB according to the import policy of EVPN Instance 10010 (import RT 65000:10010). During the import process, the Route Distinguisher values are also modified to match the configuration of Leaf-102: change the RD value from 192.168.10.101:32777 (received RD) to 192.168.10.102:32777 (local RD).

Figure 3-13: MAC Address Propagation Process – From BGP Adj-RIB-Out.

EVPN Instance Deployment Scenario 1: L2-Only EVPN Instance

In this scenario, we are building a protected Broadcast Domain (BD), which we extend to the VXLAN Tunnel Endpoint (VTEP) switches of the EVPN Fabric, Leaf-101 and Leaf-102. Note that the VTEP operates in the Network Virtualization Edge (NVE) role for the VXLAN segment. The term NVE refers to devices that encapsulate data packets to transport them over routed IP infrastructure. Another example of an NVE device is the MPLS Provider Edge (MPLS-PE) router at the edge of the MPLS network, doing MPLS labeling. The term “Tenant System” (TS) refers to a physical host, virtual machine, or an intra-tenant forwarding component attached to one or more Tenant-specific Virtual Networks. Examples of TS forwarding components include firewalls, load balancers, switches, and routers. 

We begin by configuring L2 VLAN 10 to Leaf-101 and Leaf-102 and associate it with the vn-segment 10010. From the NVE perspective, this constitutes an L2-Only network segment, meaning we do not configure an Anycast Gateway (AGW) for the segment, and it does not have any VRF association.

Next, we deploy a Layer 2 EVPN Instance (EVI) with VXLAN Network Identifier (VNI) 10010. We utilize the 'auto' option to generate the Route Distinguisher (RD) and the Route Target (RT) import and export values for the EVI. The RD value is derived from the NVE Interface IP address and the VLAN Identifier (VLAN 10) associated with the EVI, added to the base value 32767 (e.g., 192.168.100.101:32777). The use of the VLAN ID as part of the automatically generated RD value is the reason why VLAN is configured before the EVPN Instance. Similarly, the RT values are derived from the BGP ASN and the VNI (e.g., 65000:10010).

As the final step for EVPN Instance deployment, we add EVI 10010 under the NVE interface configuration as a member vni with the Multicast Group 239.1.1.1 we are using for Broadcast, Unknown Unicast, and Multicast (BUM) traffic. 

For connecting TS1 and TS2 to the Broadcast domain, we will configure Leaf-101's interface Eth 1/5 and Leaf-102's interface Eth1/3 as access ports for VLAN 10.

A few words regarding the terminology utilized in Figure 3-2. '3-Stage Routed Clos Fabric' denotes both the physical topology of the network and the model for forwarding data packets. The 3-Stage Clos topology has three switches (ingress, spine, and egress) between the attached Tenant Systems. Routed, in turn, means that switches forward packets based on the destination IP address.

With the term VXLAN Segment, I refer to a stretched Broadcast Domain, identified by the VXLAN Network Identifier value defined under the EVPN Instance on Leaf switches.

Figure 3-2: L2-Only Intra VN Connection.

Deploying and Analyze EVPN Instances: Deployment Scenarios

In the previous section, we built a Single-AS EVPN Fabric with OSPF-enabled Underlay Unicast routing and PIM-SM for Multicast routing using Any Source Multicast service. In this section, we configure two L2-Only EVPN Instances (L2-EVI) and two L2/L3 EVPN Instances (L2/3-EVI) in the EVPN Fabric. We examine their operations in six scenarios depicted in Figure 3-1.

Scenario 1 (L2-Only EVI, Intra-VN): 

In the Deployment section, we configure an L2-Only EVI with a Layer 2 VXLAN Network Identifier (L2VNI) of 10010. The Default Gateway for the VLAN associated with the EVI is a firewall. In the Analyze section, we observe the Control Plane and Data Plane operation when a) connecting Tenant Systems TS1 and TS2 to the segment, and b) TS1 communicates with TS2 (Intra-VN Communication).

Scenario 2 (L2-Only EVI, Inter-VN): 

In the Deployment section, we configure another L2-Only EVI with L2VNI 10020, to which we attach TS3 and TS4. In the Analyze section, we examine EVPN Fabric's Control Plane and Data Plane operations when TS2 (L2VNI 10010) sends data to TS3 (L2VNI 10020), Inter-VN Communication.

Scenario 3 (L2/L3 EVI, Intra-VN): 

In the Deployment section, we configure a Virtual Routing and Forwarding (VRF) Instance named VRF-NWKT with L3VNI 10077. Next, we configure the EVI with L2VNI 10030. We attach VLAN 10 to this segment, which Anycast Gateway (AGW) we bind to the routing domain VRF-NWKT. In the Analyze section, we study the Control Plane process when TS5 joins the network, focusing mainly on TS5's host IP address propagation.

Scenario 4 (Intra-VN, Silent Host): 

In the Deployment section, we configure an EVI with L2VNI 10040 in the EVPN Fabric, where the VLAN attached to it belongs to the same routing domain VRF-NWKT as EVI 10030. This EVI includes a "Silent Host" TS8, which generates no data traffic unless requested. Besides, we publish the segment-specific subnetwork within the routing domain VRF-NWKT. In the Analyze section, we focus on examining the Control Plane aspect of the EVPN Route Type 5 (IP Prefix Route) process.

Scenario 5 (Inter-VN, Symmetric IRB): 

In this section, we examine the Integrated Routing and Bridging (IRB) Symmetric routing model between two EVPN Instances. We analyze Control Plane and Data Plane functionality by studying Inter-VN communication from the perspective of TS6 to destinations TS7 and TS8 (silent host).

Scenario 6 (Inter-VN between protected and unprotected VNs): 

In this final scenario's Deployment section, we configure the firewall to advertise the subnetworks of protected L2-Only EVPN instances to the routing domain VRF-NWKT. Then, in the Analyze section, we examine how these networks appear to unprotected EVPN Instances attached to the VRF-NWKT routing domain. We also investigate Data Plane packet forwarding concerning traffic between TS5 and TS1.

We will go through each scenario in detail in the upcoming chapters.

Figure 3-1: EVPN Instance Deploying and Analyzing Scenarios.

Configuration of BGP afi/safi L2VPN EVPN and NVE Tunnel Interface

Overlay Network Routing: MP-BGP L2VPN/EVPN

EVPN Fabric Data Plane – MP-BGP

Instead of being a protocol, EVPN is a solution that utilizes the Multi-Protocol Border Gateway Protocol (MP-BGP) for its control plane in an overlay network. Besides, EVPN employs Virtual eXtensible Local Area Network (VXLAN) encapsulation for the data plane of the overlay network.

Multi-Protocol BGP (MP-BGP) is an extension of BGP-4 that allows BGP speakers to encode Network Layer Reachability Information (NLRI) of various address types, including IPv4/6, VPNv4, and MAC addresses, into BGP Update messages. The MP_REACH_NLRI path attribute (PA) carried within MP-BGP update messages includes Address Family Identifier (AFI) and Subsequent Address Family Identifier (SAFI) attributes. The combination of AFI and SAFI determines the semantics of the carried Network Layer Reachability Information (NLRI). For example, AFI-25 (L2VPN) with SAFI-70 (EVPN) defines an MP-BGP-based L2VPN solution, which extends a broadcast domain in a multipoint manner over a routed IPv4 infrastructure using an Ethernet VPN (EVPN) solution.

BGP EVPN Route Types (BGP RT) carried in BGP update messages describe the advertised EVPN NLRIs (Network Layer Reachability Information) type. Besides publishing IP Prefix information with IP Prefix Route (EVPN RT 5), BGP EVPN uses MAC Advertisement Route (EVPN RT 2) for advertising hosts’ MAC/IP address reachability information. The Virtual Network Identifiers (VNI) describe the VXLAN segment of the advertised MAC/IP addresses. 

Among these two fundamental route types, BGP EVPN can create a shared delivery tree for Layer 2 Broadcast, Unknown Unicast, and Multicast (BUM) traffic using Inclusive Multicast Route (EVPN RT 3) for joining an Ingress Replication tunnel. This solution does not require a Multicast-enabled Underlay Network. Another option for BUM traffic is Multicast capable Underlay Network.

While EVPN RT 3 is used for building a Multicast tree for BUM traffic, The Tenant Routed Multicast (TRM) solution provides tenant-specific multicast forwarding between senders and receivers. TRM is based on the Multicast VPN (BGP AFI:1/SAFI:5 – Ipv4/Mcast-VPN). TRM uses MVPN Source Active A-D Route (MVPN RT 5) for publishing Multicast stream source address and group). 

Using BGP EVPN's native multihoming solution, we can establish a Port-Channel between Tenant Systems (TS) and two or more VTEP switches. From the perspective of the TS, a traditional Port-Channel is deployed by bundling a set of Ethernet links into a single logical link. On the multihoming VTEP switches, these links are associated with a logical Port-Channel interface referred to as Ethernet Segments (ES).

EVPN utilizes the EVPN Ethernet Segment Route (EVPN RT 4) as a signaling mechanism between member units to indicate which Ethernet Segments they are connected to. Additionally, VTEP switches use this EVPN RT 4 for selecting a Designated Forwarder (DF) for Broadcast, Unknown unicast, and Multicast (BUM) traffic.

When EVPN Multihoming is enabled on a set of VTEP switches, all local MAC/IP Advertisement Routes include the ES Type and ES Identifier. The EVPN multihoming solution employs the EVPN Ethernet A-D Route (EVPN RT 1) for rapid convergence. Leveraging EVPN RT 1, a VTEP switch can withdraw all MAC/IP Addresses learned via failed ES at once by describing the ESI value in MP-UNREACH-NLRI Path Attribute. 

Note! ESI multi-homing is supported only on the first-generation Cisco Nexus 9300 switches. Nexus 9200, 9300-EX switches and newer models doesn’t support ESI multi-homing. 

An EVPN fabric employs a proactive Control Plane learning model, while networks based on Spanning Tree Protocol (STP) rely on a reactive flood-and-learn-based Data Plane learning model. In an EVPN fabric, data paths between Tenant Systems are established prior to data exchange. It's worth noting that without enabling ARP suppression, local VTEP switches flood ARP Request messages. However, remote VTEP switches do not learn the source MAC address from the VXLAN encapsulated frames.

BGP EVPN provides various methods for filtering reachability information. For instance, we can establish an import/export policy based on BGP Route Targets (BGP RT). Additionally, we can deploy ingress/egress filters using elements such as prefix-lists or BGP path attributes, like BGP Autonomous System numbers. Besides, BGP, OSPF, and IS-to-IS all support peer authentication.

 

EVPN Fabric Data Plane –VXLAN

The Virtual eXtensible LAN (VXLAN) is an encapsulation schema that enables Broadcast Domain/VLAN stretching over a Layer 3 network. Switches or hosts performing encapsulation/decapsulation are called VXLAN Tunnel End Points (VTEP). VTEPs encapsulate the Ethernet frames, originated by local Tenant Systems (TS), within outer MAC and IP headers followed by UDP header with the destination port 4789 and source port is calculated from the payload. Between the UDP header and the original Ethernet frame is the VXLAN header describing the VXLAN segment with VXLAN Network Identifier (VNI). A VNI is a 24-bit field, theoretically allowing for over 16 million unique VXLAN segments. 

VTEP devices allocate Layer 2 VNI (L2VNI) for Intra-VN connection and Layer 3 VNI (L3VNI) for Inter-NV connection. There are unique L2VNI for each VXLAN segment but one common L3VNI  for tenant-specific Inter-VN communication. Besides, the Generic Protocol Extension for VXLAN (VXLAN-GPE) enables leaf switches to add Group Policy information to data packets. 

When a VTEP receives a EVPN NLRI from the remote VTEP with importable Route Targets, it validates the route by checking that it has received from the configured BGP peer and with the right remote ASN and reachable source IP address. Then, it installs the NLRI (RD, Encapsulation Type, Next Hop, other standard and extended communities and VNIs) information into BGP Loc-RIB. Note that the local administrator part of the RD may change during the process if the VN segment is associated with another VLAN than in the remote VTEP. Remember that VLANs are locally significant, while EVPN Instances has fabric-wide meaning. Next, the best MAC route (or routes, ECMP is enabled) is encoded into L2RIB with the topology information (VLAN Id associated with the VXLAN segment) and the next-hop information. Besides, L2RIB describes the route source as BGP. Finally, L2FM programs the information into MAC address table and sets the NVE peer interface Id as next-hop. Note that VXLAN Manager learns VXLAN peers from the data plane based on the source IP address. 

Our EVPN Fabric is a Single-AS solution, where Leaf and Spine switches are in the same BGP AS area, making Leaf-Spine switches iBGP neighbors. We assign a BGP AS area 6500 to all switches and configure both Spine switches as BGP Route Reflectors, as shown in Figure 2-6. We reserve the IP subnet 192.168.10.0/24 for the Overlay network's BGP process, from which we take IP addresses for the logical interface Loopback 10. We use these addresses as a) BGP Router Identifiers (BRIDs), b) defining BGP neighbors and c) source addresses for BGP Update messages.

Leaf switches act as VXLAN Tunnel Endpoints (VTEPs), responsible for encapsulating/decapsulating data packets to/from Customer networks on the Fabric's Transport network side. A logical Network Virtual Edge (NVE) interfaces of Leaf switches use VXLAN tunneling, where the tunnel source IP address is the IP address of Loopback 20. We reserve the subnet 192.168.20.0/24 for this purpose, as shown in Figure 2-6. 

In Figure 2-6, I have listed the VTEP Loopback identifier and IP address sections belonging to the Underlay network. The reason is that the source/destination IP addresses used for tunneling between VTEP devices must be routable by the devices in the Transport network (Underlay Network). In the context of BGP EVPN, the term "Overlay" refers to the fact that it advertises only the MAC and IP addresses and subnets required for IP communication among devices connected to EVPN segments.

The following image also lists mandatory NX-OS features that we must enable to configure both the BGP EVPN Control Plane and the Data Plane.

Figure 2-6: EVPN Fabric Overlay Network Control Plane and Data Plane.

Image 2-7 depicts our implementation of a Single-AS EVPN Fabric. The Spine switch serves as a BGP Route Reflector, forwarding BGP Update messages from Leaf switches to other Leaf switches. The BGP process on Leaf switches sets the IP address of the Loopback 10 interface as the Next-hop in the MP_REACH_NLRI Path Attribute for all advertised EVPN NLRI Route Types.

The Network Virtual Edge (NVE) interfaces use the IP address of Loopback 10 for VXLAN tunneling. The NVE interface sub-command "host reachability protocol BGP" instructs the NVE interface to use the Control Plane learning model based on the received BGP Updates about EVPN NLRIs.

Figure 2-7: EVPN Fabric Overlay Network Control Plane and Data Plane Building Blocks.

BGP EVPN Configuration

Example 2-18 shows the configuration of Spine-12 for BGP. The first two commands enable BGP EVPN. In the actual BGP configuration, we first specify the BGP AS number as 65000. Then, we attach the IP address we defined for Loopback 10 as the BGP Route ID. The command Address-family l2vpn evpn with the subcommand maximum-paths 2 enables flow-based load sharing across two BGP peers if their EVPN NLRI AS_PATH attributes are identical. The commonly used term for this is Equal Cost Multi-Pathing (ECMP). 

Using the neighbor command, we define the BGP neighbor's IP address. For each BGP neighbor, we define a BGP AS number and the source IP address for the locally generated BGP Update messages. With the command address-family l2vpn, we indicate that we want to exchange EVPN NLRI information with this neighbor. 

Depending on the advertised EVPN Route Type, a set of BGP Extended Community attributes are carried with advertised EVPN NLRIs. Hence, we need the command send-community extended. By default, the BGP loop prevention mechanism prevents iBGP peers from advertising NLRI information learned from other iBGP peers. We bypass this mechanism by configuring the Spine switches as BGP Route Reflectors using the neighbor-specific route-reflector-client command.

feature bgp

nv overlay evpn

!

router bgp 65000

  router-id 192.168.10.12

  address-family l2vpn evpn

    maximum-paths 2

  neighbor 192.168.10.101

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

!

  neighbor 192.168.10.102

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

!

  neighbor 192.168.10.103

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

!

  neighbor 192.168.10.104

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

      route-reflector-client

Example 2-18: Spine Switches BGP Configuration.

 

Example 2-19 illustrates the BGP configuration of switch Leaf-101. The BGP configurations of all Leaf switches are identical except for the BGP router ID.

feature bgp

nv overlay evpn

!

router bgp 65000

  router-id 192.168.10.101

  address-family l2vpn evpn

    maximum-paths 2

  neighbor 192.168.10.11

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

  neighbor 192.168.10.12

    remote-as 65000

    update-source loopback10

    address-family l2vpn evpn

      send-community

      send-community extended

Example 2-19: Leaf Switches BGP Configuration.

BGP EVPN Verification

From Example 2-20, we can see the BGP commands we have associated with the BGP neighbor Leaf-101 on Spine-11.

Spine-11# sh bgp l2vpn evpn neighbors 192.168.10.101 commands

Command information for 192.168.10.101

                 Update Source: locally configured

                     Remote AS: locally configured

 Address Family: L2VPN EVPN

                Send Community: locally configured

            Send Ext-community: locally configured

        Route Reflector Client: locally configured

Spine-11#

Example 2-20: Leaf Switches BGP Configuration.

 

Example 2-21 shows the BGP neighbors of Spine-11 with their AS numbers and statistics regarding received and sent BGP messages (Open, Keepalive, Update, and Notification). All EVPN Route Type counters are zero because we haven't yet deployed EVPN instances.

Spine-11# sh bgp l2vpn evpn summary

BGP summary information for VRF default, address family L2VPN EVPN

BGP router identifier 192.168.10.12, local AS number 65000

BGP table version is 6, L2VPN EVPN config peers 4, capable peers 4

0 network entries and 0 paths using 0 bytes of memory

BGP attribute entries [0/0], BGP AS path entries [0/0]

BGP community entries [0/0], BGP clusterlist entries [0/0]

Neighbor        V    AS    MsgRcvd    MsgSent   TblVer  InQ OutQ Up/Down  State/PfxRcd

192.168.10.101  4 65000         14         17        0    0    0 00:00:02 0

192.168.10.102  4 65000         19         20        0    0    0 00:00:02 0

192.168.10.103  4 65000          6          4        0    0    0 00:00:06 0

192.168.10.104  4 65000         14         17        0    0    0 00:00:02 0

Neighbor        T    AS PfxRcd     Type-2     Type-3     Type-4     Type-5     Type-12

192.168.10.101  I 65000 0          0          0          0          0          0

192.168.10.102  I 65000 0          0          0          0          0          0

192.168.10.103  I 65000 0          0          0          0          0          0

192.168.10.104  I 65000 0          0          0          0          0          0

Spine-11#

Example 2-21: Leaf Switches BGP Configuration.

Example 2-21 shows information and statistics about the BGP neighborship between switches Spine-11 and Leaf-101. Leaf-101 belongs to the same BGP Autonomous System (AS) area 65000 as Spine-11, making Leaf-101 an iBGP neighbor. I have highlighted the parts that confirm the functionality of our configuration. The neighborship state is "Established", indicating that the switches are ready to send and receive BGP Update messages. Spine-11 uses the logical interface Loopback10 as its source address in BGP Update messages. The Capabilities and Graceful Restart sections show that the switches support the BGP address family L2VPN EVPN. At the end of the output, we see that Leaf-101 is configured as a Route-Reflector Client.

 

Spine-11# sh bgp l2vpn evpn neighbors 192.168.10.101

BGP neighbor is 192.168.10.101, remote AS 65000, ibgp link, Peer index 3

  BGP version 4, remote router ID 192.168.10.101

  Neighbor previous state = OpenConfirm

  BGP state = Established, up for 00:02:40

  Neighbor vrf: default

  Using loopback10 as update source for this peer

  Using iod 71 (loopback10) as update source

  Last read 00:00:35, hold time = 180, keepalive interval is 60 seconds

  Last written 00:00:35, keepalive timer expiry due 00:00:24

  Received 18 messages, 0 notifications, 0 bytes in queue

  Sent 21 messages, 1 notifications, 0(0) bytes in queue

  Enhanced error processing: On

    0 discarded attributes

  Connections established 2, dropped 1

  Last update recd 00:02:35, Last update sent  = never

   Last reset by us 00:02:51, due to router-id configuration change

  Last error length sent: 0

  Reset error value sent: 0

  Reset error sent major: 6 minor: 107

  Notification data sent:

  Last reset by peer never, due to No error

  Last error length received: 0

  Reset error value received 0

  Reset error received major: 0 minor: 0

  Notification data received:

  Neighbor capabilities:

  Dynamic capability: advertised (mp, refresh, gr) received (mp, refresh, gr)

  Dynamic capability (old): advertised received

  Route refresh capability (new): advertised received

  Route refresh capability (old): advertised received

  4-Byte AS capability: advertised received

  Address family L2VPN EVPN: advertised received

  Graceful Restart capability: advertised received

  Graceful Restart Parameters:

  Address families advertised to peer:

    L2VPN EVPN

  Address families received from peer:

    L2VPN EVPN

  Forwarding state preserved by peer for:

  Restart time advertised to peer: 120 seconds

  Stale time for routes advertised by peer: 300 seconds

  Restart time advertised by peer: 120 seconds

  Extended Next Hop Encoding Capability: advertised received

  Receive IPv6 next hop encoding Capability for AF:

    IPv4 Unicast  VPNv4 Unicast

  Message statistics:

                              Sent               Rcvd

  Opens:                         4                  2

  Notifications:                 1                  0

  Updates:                       2                  2

  Keepalives:                   12                 12

  Route Refresh:                 0                  0

  Capability:                    2                  2

  Total:                        21                 18

  Total bytes:                 327                306

  Bytes in queue:                0                  0

  For address family: L2VPN EVPN

  BGP table version 10, neighbor version 10

  0 accepted prefixes (0 paths), consuming 0 bytes of memory

  0 received prefixes treated as withdrawn

  0 sent prefixes (0 paths)

  Community attribute sent to this neighbor

  Extended community attribute sent to this neighbor

  Third-party Nexthop will not be computed.

  Advertise GW IP is enabled

  Route reflector client

  Last End-of-RIB received 00:00:05 after session start

  Last End-of-RIB sent 00:00:05 after session start

  First convergence 00:00:05 after session start with 0 routes sent

  Local host: 192.168.10.11, Local port: 33940

  Foreign host: 192.168.10.101, Foreign port: 179

  fd = 90

Example 2-21: Leaf Switches BGP Configuration.

Overlay Network Data Plane: VXLAN 

NVE Interface Configuration

Example 2-22 shows the configuration of the NVE interface and the required feature configuration for client overlay networks. The "feature nv overlay" enables VXLAN overlay networks. The "feature vn-segment-vlan-based" specifies that only the MAC addresses of the VLAN associated with the respective EVPN instance (EVI) are stored in the MAC-VRF's Layer2 RIB (L2RIB). In other words, the EVPN instance forms a single broadcast domain. Under the NVE interface, we define the logical interface Loopback20's IP address as the tunnel source address. Additionally, we specify that the NVE interface implements the Control Plane learning model, meaning the switch learns remote MAC addresses from BGP Update messages, not from the data traffic received through the tunnel interface (Data Plane learning).

 

feature nv overlay

feature interface-vlan

feature vn-segment-vlan-based

!

interface nve1

  no shutdown

  host-reachability protocol bgp

  source-interface loopback20

Example 2-22: Leaf Switches BGP Configuration.

NVE Interface Verification

Example 2-23 shows the summary information about the settings of the interface NVE 1. Leaf-101 uses Loopback20 as a source interface when sending traffic over the interface NVE1. Besides, Leaf-101 uses the Control Plane learning model. Leaf-101 encodes the router MAC address to BGP Update messages as "Router MAC" Extended community associated with EVPN Route type2 (MAC-IP Advertisement Route) when the update carries both MAC and IP addresses. The remote leaf switches use it as a source MAC address in the inner Ethernet when frame when forwarding Inter-VN traffic.

Leaf-101# show nve interface nve 1

Interface: nve1, State: Up, encapsulation: VXLAN

 VPC Capability: VPC-VIP-Only [not-notified]

 Local Router MAC: 5003.0000.1b08

 Host Learning Mode: Control-Plane

 Source-Interface: loopback20 (primary: 192.168.20.101, secondary: 0.0.0.0)

Example 2-23: Leaf Switches BGP Configuration.

Example 2-24 demonstrates that Leaf-101 currently lacks any NVE peers because its VXLAN manager initiates an NVE peer relationship with other VTEPs upon receiving the first data packet over the NVE interface.

Leaf-101# show nve peers detail

Leaf-101#

Example 2-24: Leaf Switches BGP Configuration.

At this stage, we have configured the EVPN Fabric to the point where we can deploy our first EVPN instances and test and analyze both the Intra-VN and Inter-VN Control Plane and Data Plane perspectives.