Monday 22 March 2021

Cisco SD-WAN - Part II: Manual vEdge Provision Process

Introduction

 

This chapter explains how we can provision vEdge devices manually. It starts by explaining how to build an initial system and tunnel interface configurations. Then it goes through the various certificate installation steps (CA root certificate, Certificate Signing Request (CSR), and granted certificate). After the initial configuration and certificate process section, this chapter shows how we can verify the Control Plane operation. Figure 2-1 illustrates our example topology. For simplicity, there are only two vEdge devices used in this chapter.


Figure 2-1: SD-WAN Topology.



vEdge Configuration

 

System Information

 

The only difference in vEdge initial system configuration compared to vManagem vBond, and vSmart is the device-specific host-name and system-ip values.

vedge# conf t

Entering configuration mode terminal

vedge(config)# system

vedge(config-system)# host-name vEdge-1

vedge(config-system)# site-id 10

vedge(config-system)# system-ip 10.100.100.101

vedge(config-system)# organization-name nwkt

vedge(config-system)# vbond 10.100.0.11

vedgeconfig-system)# ntp server 10.100.0.14

vedge(config-server-10.100.0.14)# vpn 0

vedge(config-server-10.100.0.14)# exit

vedge(config-ntp)# exit

vedge(config-system)# commit

Commit complete.

vEdge-1(config-system)#

Example 2-1: The Initial Configuration of vEdge-1-Step#1: System Configuration.

 

Underlay Network: VPN 0

 

VPN 0 is always used only for Underlay Network connections. The VPN 0 is like a Front-Door VRF in Cisco legacy Intelligent WAN (IWAN) SD-WAN solution. In our example, interfaces ge0/0 and ge0/1 on vEdges are attached to VPN 0 and IP addresses are bind to them statically. Both interfaces are working as IPSec tunnel-interface. The definition color identifies the transport connections. The color of the interface ge0/0 is public-internet while the color of interface g0/1 is mpls. The command color [color] restrict means that tunnels are only established between the TLOCs (Transport Locator) belonging to the same color. If the restrict option is left out, the vEdge tries to establish a tunnel with every remote TLOCs (MPLS-to-MPLS and MPLS-Public-Internet, Public-Internet-to-MPLS, and  Public-Internet-to-Public-Internet) it learns via Overlay Management Protocol (OMP). These kinds of failed tunnels are shown in Dashboard > Main Dashboard windows in the Side Health field as Partial WAN Connectivity. The Main dashboard is shown in figure 2-4 (without any issues).

Note that there are 22 pre-defined colors which are divided into Public and Private colors based on their Network Address Translation (NAT) solution. If NAT is required (or might later be required) use public colors (3g, lte, biz-internet, public-internet, blue, green, red, bronze, silver, gold, custom1-3). If NAT is not needed, use private colors (metro-ethernet, mpls, private 1-6).

The command max-control-connection 0 under tunnel-interface assigned to the interface ge0/1 means that vEdge doesn’t try to establish control connections to vManage, vBond, or vSmart. Without the command, vEdge-1 and vEdge-3 will try to establish control connections over MPLS without success (no route). These kinds of failures are shown in Dashboard > Main Dashboard windows in the Control Status field as Partial.

I have also configured VPN 512 for Out of Band Management which is used for copying the root certificate file into vEdges by using WinSCP in the same way that was done with control plane devices vBond and vSmart.

vEdge-1(config-system)# vpn 0

vEdge-1(config-vpn-0)#  interface ge0/0

vEdge-1(config-interface-ge0/0)# ip address 10.100.0.101/24

vEdge-1(config-interface-ge0/0)# tunnel-interface

vEdge-1(config-tunnel-interface)# color public-internet restrict

vEdge-1(config-tunnel-interface)# encapsulation ipsec

vEdge-1(config-tunnel-interface)# allow-service all

vEdge-1(config-tunnel-interface)# no shutdown

vEdge-1(config-tunnel-interface)# exit

vEdge-1(config-interface-ge0/0)# exit

vEdge-1(config-vpn-0)# interface ge0/1

vEdge-1(config-interface-ge0/1)# ip address 10.200.0.101/24

vEdge-1(config-interface-ge0/1)# tunnel-interface

vEdge-1(config-tunnel-interface)# color mpls restrict

vEdge-1(config-tunnel-interface)# max-control-connections 0

vEdge-1(config-tunnel-interface)# encapsulation ipsec

vEdge-1(config-tunnel-interface)# allow-service all

vEdge-1(config-tunnel-interface)# no shutdown

vEdge-1(config-tunnel-interface)# exit

vEdge-1(config-interface-ge0/1)# exit

vEdge-1(config-system)# vpn 512

vEdge-1(config-vpn-512)# interface eth0

vEdge-1(config-interface-eth0)# ip dhcp-client

vEdge-1(config-interface-eth0)# no shutdown

vEdge-1(config-interface-eth0)# !

vEdge-1(config-interface-eth0)# ip route 0.0.0.0/0 192.168.10.1

vEdge-1(config-vpn-512)# commit

Example 2-2: The Initial Configuration of vEdge-1-Step#2: VPN Specific Interface Settings.

 

 

 

Certification enrollment

 

The CA root certificate enrollment process follows the same principles as what was introduced in chapter 1 with vBond and vMange. I have already copied the PKI.pem file into the vEdge directory home/admin, where it can be installed.

 

vEdge-3# request root-cert-chain install home/admin/PKI.pem

Uploading root-ca-cert-chain via VPN 0

Copying ... /home/admin/PKI.pem via VPN 0

Updating the root certificate chain..

Successfully installed the root certificate chain

Example 2-3: Certificate Enrollment-Step#1: Installing the CA Root Certificate.

 

Next, we generate a Certificate Signing Request (CSR) with the name csr.txt into the home/admin directory.

 

vEdge-3# request csr upload home/admin/csr.txt

Uploading CSR via VPN 0

Enter organization-unit name            : nwkt

Re-enter organization-unit name          : nwkt

Generating private/public pair and CSR for this vedge device

Generating CSR for this vedge device   ........[DONE]

Copying ... /home/admin/csr.txt via VPN 0

CSR upload successful

Example 2-4: Certificate Enrollment-Step#2: Generating CSR.

 

Then we go into the Linux shell and verify that the file is created. After that, we printed it out and copy it to the clipboard.

 

vEdge-3# vshell

vEdge-3:~$ ls

PKI.pem  archive_id_rsa.pub  csr.txt

vEdge-3:~$ more csr.txt

-----BEGIN CERTIFICATE REQUEST-----

MIIDQjCCAioCAQAwgcExCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh

MREwDwYDVQQHEwhTYW4gSm9zZTENMAsGA1UECxMEbndrdDEUMBIGA1UEChMLVmlw

dGVsYSBMTEMxQTA/BgNVBAMTOHZlZGdlLTg3NzAwZDE0LTBiNGItNGRjYi1hOTNk

LTQ3YzEyYTViYzMyMS0wLnZpcHRlbGEuY29tMSIwIAYJKoZIhvcNAQkBFhNzdXBw

b3J0QHZpcHRlbGEuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA

qrdJX6GajIK9kZY1Go358nYcyCxh6ie8w6uXvefUhXytsMc2cIVbiTimhYTrRB5+

0Ag2R6WjoohJrlpvDom/mJuNo6GuNjzJOXEhEOLVH2zDwvTnI3nhMmiCWLTNMkGe

NeQSKbHu7VwJ9/zbpPYJgmtsTVF4pzXVN3XVIRAq65VaZ9Tlg4kZUdFXnrKolx0p

hW8DB9QhIL9CO5fGmXuK+Ahc810gV6ObAy+fS4Gbt9fAZToXUK65ToeHWjTzgIsD

3OV4Wieae9PAJd2TZEGismiSONv8JeYVjxXIpzdk+j4NXBv9QgpZxOoEnRpTl/ds

nEfjNpAmHL4m6eaQh2XrJwIDAQABoDswOQYJKoZIhvcNAQkOMSwwKjAJBgNVHRME

AjAAMB0GA1UdDgQWBBSnmgfFMK/zFlb0zYjfdgMZojPhzDANBgkqhkiG9w0BAQsF

AAOCAQEAUsxQlaor8Clt4aDhK3S+t/jo+48i2f1WvKro7CLMC5mB9cKrmugM+fhl

F33mnQtR6/F08yAnMXzaPQuHIN7P6f4nZ0lYNGa8HWXAanlkjqKILoYBfB4LJ7G8

z9UZM03e/0myjCdik2oLhPO0tza/fZ0izf5UzBKcgGWJzn9fPhbgKD+g5ejaF4Vk

kyYYZGnWK7C5okdGqsEv6MMkWxmS5u7hAI3j82gmV7drFL/qK/z23NngBq9Rtz3B

8pCjQDOIUvpWrJQRgY2yAHDsp0kOgQ2UuJdRAvmuaUPr1krmrjdM2+EpItfl2SB4

+5SVRtKDDRiQQG/hLKBGsX+OSMS7zA==

-----END CERTIFICATE REQUEST-----

vEdge-3:~$

Example 2-5: Certificate Enrollment-Step#3: printing the CSR.

 

Use the command crypto pki server PKI request pkcs10 terminal in order to paste CSR into our IOS-XE CA-Server. When done, type quit and then press enter to get Granted Certificate. Copy the Granted Certificate into the clipboard.

 

CA-Server#crypto pki server PKI request pkcs10 terminal

PKCS10 request in base64 or pem

 

% Enter Base64 encoded or PEM formatted PKCS10 enrollment request.

% End with a blank line or "quit" on a line by itself.

-----BEGIN CERTIFICATE REQUEST-----

MIIDQjCCAioCAQAwgcExCzAJBgNVBAYTAlVTMRMwEQYDVQQIEwpDYWxpZm9ybmlh

MREwDwYDVQQHEwhTYW4gSm9zZTENMAsGA1UECxMEbndrdDEUMBIGA1UEChMLVmlw

dGVsYSBMTEMxQTA/BgNVBAMTOHZlZGdlLTg3NzAwZDE0LTBiNGItNGRjYi1hOTNk

LTQ3YzEyYTViYzMyMS0wLnZpcHRlbGEuY29tMSIwIAYJKoZIhvcNAQkBFhNzdXBw

b3J0QHZpcHRlbGEuY29tMIIBIjANBgkqhkiG9w0BAQEFAAOCAQ8AMIIBCgKCAQEA

qrdJX6GajIK9kZY1Go358nYcyCxh6ie8w6uXvefUhXytsMc2cIVbiTimhYTrRB5+

0Ag2R6WjoohJrlpvDom/mJuNo6GuNjzJOXEhEOLVH2zDwvTnI3nhMmiCWLTNMkGe

NeQSKbHu7VwJ9/zbpPYJgmtsTVF4pzXVN3XVIRAq65VaZ9Tlg4kZUdFXnrKolx0p

hW8DB9QhIL9CO5fGmXuK+Ahc810gV6ObAy+fS4Gbt9fAZToXUK65ToeHWjTzgIsD

3OV4Wieae9PAJd2TZEGismiSONv8JeYVjxXIpzdk+j4NXBv9QgpZxOoEnRpTl/ds

nEfjNpAmHL4m6eaQh2XrJwIDAQABoDswOQYJKoZIhvcNAQkOMSwwKjAJBgNVHRME

AjAAMB0GA1UdDgQWBBSnmgfFMK/zFlb0zYjfdgMZojPhzDANBgkqhkiG9w0BAQsF

AAOCAQEAUsxQlaor8Clt4aDhK3S+t/jo+48i2f1WvKro7CLMC5mB9cKrmugM+fhl

F33mnQtR6/F08yAnMXzaPQuHIN7P6f4nZ0lYNGa8HWXAanlkjqKILoYBfB4LJ7G8

z9UZM03e/0myjCdik2oLhPO0tza/fZ0izf5UzBKcgGWJzn9fPhbgKD+g5ejaF4Vk

kyYYZGnWK7C5okdGqsEv6MMkWxmS5u7hAI3j82gmV7drFL/qK/z23NngBq9Rtz3B

8pCjQDOIUvpWrJQRgY2yAHDsp0kOgQ2UuJdRAvmuaUPr1krmrjdM2+EpItfl2SB4

+5SVRtKDDRiQQG/hLKBGsX+OSMS7zA==

-----END CERTIFICATE REQUEST-----

quit

% Granted certificate:

-----BEGIN CERTIFICATE-----

MIIDrDCCApSgAwIBAgIBDzANBgkqhkiG9w0BAQsFADAcMRowGAYDVQQDExFyb290

Y2EubndrdC5sb2NhbDAeFw0yMTAzMTgxNTM4NDdaFw0yMjAzMTgxNTM4NDdaMIHB

MQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTERMA8GA1UEBxMIU2Fu

IEpvc2UxDTALBgNVBAsTBG53a3QxFDASBgNVBAoTC1ZpcHRlbGEgTExDMUEwPwYD

VQQDEzh2ZWRnZS04NzcwMGQxNC0wYjRiLTRkY2ItYTkzZC00N2MxMmE1YmMzMjEt

MC52aXB0ZWxhLmNvbTEiMCAGCSqGSIb3DQEJARYTc3VwcG9ydEB2aXB0ZWxhLmNv

bTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKq3SV+hmoyCvZGWNRqN

+fJ2HMgsYeonvMOrl73n1IV8rbDHNnCFW4k4poWE60QeftAINkelo6KISa5abw6J

v5ibjaOhrjY8yTlxIRDi1R9sw8L05yN54TJogli0zTJBnjXkEimx7u1cCff826T2

CYJrbE1ReKc11Td11SEQKuuVWmfU5YOJGVHRV56yqJcdKYVvAwfUISC/QjuXxpl7

ivgIXPNdIFejmwMvn0uBm7fXwGU6F1CuuU6Hh1o084CLA9zleFonmnvTwCXdk2RB

orJokjjb/CXmFY8VyKc3ZPo+DVwb/UIKWcTqBJ0aU5f3bJxH4zaQJhy+JunmkIdl

6ycCAwEAAaNTMFEwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBTSwYkk0XxU

fShDc060d/h/LXehrzAdBgNVHQ4EFgQUp5oHxTCv8xZW9M2I33YDGaIz4cwwDQYJ

KoZIhvcNAQELBQADggEBAATPB1iYqcg0o9n7sgyA2DGFayZYEmIHS7R/2YMbFaQ9

yrbzD9lKoISOvyHDT4SSRdFpBYM2ZHMFdM62JsRqpCZz5Leswv++Y4DMbu5t8MKW

arDYzhYVmggTlmMQ+BLU1WOE+gJn4gIxVNKJcphWJOA5oQDcVnlTbB5iKCDiIfVx

0GWV8UKLNpFV8pypmUIUz7UDAx7UQcSgOYsvzJ80C2sM1XFIuF8+C/ieFUxXaz3i

QPlHUvFTE3Tz9uTKpk/lN0UDib9RxfXUOqHX+UQ8ok80xQN5Td6IsHUWSn97q8Pm

GCz80wPh9hmfrPq0Dvz5ak0Gde5foIPfoKCNkKDfIvw=

-----END CERTIFICATE-----

 

CA-Server#

Example 2-6: Certificate Enrollment-Step#4: Generating Granted Certificate.


Create a certificate file cert.txt by first using the command cat <<”” > cert.txt and then by pasting the granted certificate from the clipboard.

 

vEdge-3:~$ cat <<"" > cert.txt

> -----BEGIN CERTIFICATE-----

> MIIDrDCCApSgAwIBAgIBDzANBgkqhkiG9w0BAQsFADAcMRowGAYDVQQDExFyb290

> Y2EubndrdC5sb2NhbDAeFw0yMTAzMTgxNTM4NDdaFw0yMjAzMTgxNTM4NDdaMIHB

> MQswCQYDVQQGEwJVUzETMBEGA1UECBMKQ2FsaWZvcm5pYTERMA8GA1UEBxMIU2Fu

> IEpvc2UxDTALBgNVBAsTBG53a3QxFDASBgNVBAoTC1ZpcHRlbGEgTExDMUEwPwYD

> VQQDEzh2ZWRnZS04NzcwMGQxNC0wYjRiLTRkY2ItYTkzZC00N2MxMmE1YmMzMjEt

> MC52aXB0ZWxhLmNvbTEiMCAGCSqGSIb3DQEJARYTc3VwcG9ydEB2aXB0ZWxhLmNv

> bTCCASIwDQYJKoZIhvcNAQEBBQADggEPADCCAQoCggEBAKq3SV+hmoyCvZGWNRqN

> +fJ2HMgsYeonvMOrl73n1IV8rbDHNnCFW4k4poWE60QeftAINkelo6KISa5abw6J

> v5ibjaOhrjY8yTlxIRDi1R9sw8L05yN54TJogli0zTJBnjXkEimx7u1cCff826T2

> CYJrbE1ReKc11Td11SEQKuuVWmfU5YOJGVHRV56yqJcdKYVvAwfUISC/QjuXxpl7

> ivgIXPNdIFejmwMvn0uBm7fXwGU6F1CuuU6Hh1o084CLA9zleFonmnvTwCXdk2RB

> orJokjjb/CXmFY8VyKc3ZPo+DVwb/UIKWcTqBJ0aU5f3bJxH4zaQJhy+JunmkIdl

> 6ycCAwEAAaNTMFEwDwYDVR0TAQH/BAUwAwEB/zAfBgNVHSMEGDAWgBTSwYkk0XxU

> fShDc060d/h/LXehrzAdBgNVHQ4EFgQUp5oHxTCv8xZW9M2I33YDGaIz4cwwDQYJ

> KoZIhvcNAQELBQADggEBAATPB1iYqcg0o9n7sgyA2DGFayZYEmIHS7R/2YMbFaQ9

> yrbzD9lKoISOvyHDT4SSRdFpBYM2ZHMFdM62JsRqpCZz5Leswv++Y4DMbu5t8MKW

> arDYzhYVmggTlmMQ+BLU1WOE+gJn4gIxVNKJcphWJOA5oQDcVnlTbB5iKCDiIfVx

> 0GWV8UKLNpFV8pypmUIUz7UDAx7UQcSgOYsvzJ80C2sM1XFIuF8+C/ieFUxXaz3i

> QPlHUvFTE3Tz9uTKpk/lN0UDib9RxfXUOqHX+UQ8ok80xQN5Td6IsHUWSn97q8Pm

> GCz80wPh9hmfrPq0Dvz5ak0Gde5foIPfoKCNkKDfIvw=

> -----END CERTIFICATE-----

> 

> 

vEdge-3:~$ exit

vEdge-3#

Example 2-7: Certificate Enrollment-Step#5: Creating Certificate File.

 

Next, install the certificate.

 

vEdge-3# request certificate install  home/admin/cert.txt

Installing certificate via VPN 0

Copying ... /home/admin/cert.txt via VPN 0

cp -f "/usr/share/viptela/tmp_csr/server.key" "/usr/share/viptela/server.key"

moving temp Cert "/usr/share/viptela/server.crt.tmp" to Cert "/usr/share/viptela/vedge_certs/client_0F.crt"

Successfully installed the certificate

Example 2-8: Certificate Enrollment-Step#6: Installing Granted Certificate.

 

After installing certificates, we can register devices into vMange. This is done by assigning both the chassis number and the serial number of vEdge to vMange and vBond. The process is shown in examples 2-9, 2-10, and 2-11.

 

vEdge-3# show certificate serial

Chassis number: 87700d14-0b4b-4dcb-a93d-47c12a5bc321 serial number: 0F

Example 2-9: Certificate Enrollment-Step#7: vEdge Serial Number Verification.

 

 

vmanage#request vedge add chassis-num 87700d14-0b4b-4dcb-a93d-47c12a5bc321 serial-num 0F

status success

Example 2-10: Certificate Enrollment-Step#8: Adding vEdge into vManage.

 

 

vbond# request vedge add chassis-num 87700d14-0b4b-4dcb-a93d-47c12a5bc321 serial-num 0F

status success

Example 2-11: Certificate Enrollment-Step#9: Adding vEdge into vBond.

 

We can verify that vEdges are registered into vMange by navigating to the Configuration/Devices window and by selecting the WAN Edge List tab. As can be seen from the figure below, the state of both vEdges is successful.


Figure 2-2: vMange: Configuration>Devices> Wan Edge List.


As the last step, we need to send the updated vEdge list to other control devices by navigating the Configuration>Certificates window and selecting the Send to Controllers from the WAN Edge List tab.

Figure 2-3: Updating the vEdge List.



Figure 2-4 verifies that our example SD-WAN infrastructure is now up and running.


Figure 2-4: Updating the vEdge List.


Onboarding Process

 

Figure 2-5 illustrates the onboarding process after initial configuration and successful certification enrollment from the vEdge-1 perspective. (1) vEdge-1 does bidirectional authentication with vBond. (2) Devices establish DTLD tunnel. (3) vBond instructs vEdge-1 how to reach vManage and vSmart devices over the DTLS tunnel. (4) vEdge does bidirectional authentication with vManage. (5) Devices establish DTLD tunnel. (6) vMange send the configuration defined in configuration templates to vEdge-1 over the DTLS tunnel. (7) vEdge does bidirectional authentication with vSmart. (8) Devices establish DTLD tunnel. (9) vEdge and vSmart exchange OMP routing information over DTLS tunnel. We don’t have any configured service VPN at this phase, so only TLOC routes are advertised (10) vEdge tears down the DTLS tunnel with vBond.



Figure 2-5:
 vEdge-1 Onboarding Process.


The control connections are established over DTLS tunnels while Data Plane tunnels are using IPSec as a tunneling solution. Note that control connections are only established over Public-Internet and IPSec tunnels are only established between the interfaces with the same color. We will get back on these in later sections.


Figure 2-6: vEdge-1 Tunnels for Management Plane, Control Plane, and Data Plane.


Control Connection verification

 

We can verify vEdge-1 control connections from the vManage GUI by navigating to Monitor > Network and choosing vEdge-1 from the Host Name list. The current Control Connection can be verified by scrolling down to Control Connection on the left menu. Figure 2-7 shows that  vEdge-1 has DTLS tunnels to vSmart and vManage, both using public-internet. 


Figure 2-7: GUI-based vEdge Control Connections Verifications.


You can also check the control connections by selecting the Troubleshooting from the left menu and then by selecting the Control Connections (Live view) from the Connectivity sections.


Figure 2-8: GUI-based vEdge Control Connections Verifications.

 

CLI-based verification can be done by using the command show control connections details. vBond is also listed in output but with the System-IP 0.0.0.0.

vEdge-1# show control connections detail

""

--------------------------------------------------------------------------------------

 LOCAL-COLOR- public-internet SYSTEM-IP- 10.100.100.13   PEER-PERSONALITY- vsmart

--------------------------------------------------------------------------------------

site-id             100

domain-id           1

protocol            dtls

private-ip          10.100.0.13

private-port        12446

public-ip           10.100.0.13

public-port         12446

 

state               up [Local Err: NO_ERROR] [Remote Err: NO_ERROR]

uptime              0:01:50:35

hello interval      1000

hello tolerance     12000

controller-grp-id   0

 

  <Tx and Rx statistics ommited>

 

--------------------------------------------------------------------------------------

 LOCAL-COLOR- public-internet SYSTEM-IP- 0.0.0.0   PEER-PERSONALITY- vbond

--------------------------------------------------------------------------------------

site-id             0

domain-id           0

protocol            dtls

private-ip          10.100.0.11

private-port        12346

public-ip           10.100.0.11

public-port         12346

 

state               up [Local Err: NO_ERROR] [Remote Err: NO_ERROR]

uptime              0:01:50:53

hello interval      1000

hello tolerance     12000

controller-grp-id   0

 

<Tx and Rx statistics ommited>

 

--------------------------------------------------------------------------------------

 LOCAL-COLOR- public-internet SYSTEM-IP- 10.100.100.12   PEER-PERSONALITY- vmanage

--------------------------------------------------------------------------------------

site-id             100

domain-id           0

protocol            dtls

private-ip          10.100.0.12

private-port        12446

public-ip           10.100.0.12

public-port         12446

 

state               up [Local Err: NO_ERROR] [Remote Err: NO_ERROR]

uptime              0:01:50:37

hello interval      1000

hello tolerance     12000

controller-grp-id   0

 

 <Tx and Rx statistics ommited>

Example 2-12: CLI-based vEdge Control Connections Verifications.

 

You can find current OMP peers by selecting Real Time from the left menu and selecting OMP peers from the Device Options: filed. From the figure below we can see that vEdge-1 has OMP peering with vSmart.  Note that you can select what columns you want to see on the screen by clicking the green icon right next to the refresh icon on the right corner.



Figure 2-9: GUI-based vEdge OMP Peer Verifications.


The same thing can also be verified from the vEdge-1 CLI by using the command show omp peers. The output also shows that vEdge-1 doesn’t have received, installed, or sent (R/I/S) any prefixes from the vSmart. This is because we haven’t added any service VPNs into example SD-WAN infrastructure yet.

 

vEdge-1# show omp peers

R -> routes received

I -> routes installed

S -> routes sent

 

                         DOMAIN    OVERLAY   SITE

PEER             TYPE    ID        ID        ID        STATE    UPTIME           R/I/S

--------------------------------------------------------------------------------------

10.100.100.13    vsmart  1         1         100       up       0:04:37:47       0/0/0

Example 2-13: Verifying OMP Peers.


The only routing information exchanged between vEdges and vSmart via OMP is Transport Locations (TLOCs) information which describes the System-IP (10.100.100.101), Color (mpls/public-internet), and Encapsulation type (IPSEC). Figure 2-10 shows the basic operation of TLOC advertisement. vEdge-1 advertises two TLOCs, one for the color mpls and the other one to color public-internet. Updates are sent only to vSmart (vEdges doesn’t establish OMP peering between themself) that reflects the update to vEdge-3. In that sense, vSmart is like an iBGP route-reflector. The OMP TLOC update carries also a set of attributes, just like BGP Network Layer Reachability Information (NLRI). Attributes Public-IP and Private-IP addresses are used as destination IP addresses in the tunnel header. Which one is used depends on the TLOC color. The color mpls is a private color, which means that private-ip is always used in the data plane even though the device is behind the NAT device. The second TLOC with the color public-internet is public color and with these TLOCs the public-IP address is used. There is no NAT in our example network, so the public-ip and private-ip attributes are the same on both TLOCs. Note that all OMP updates are sent over DTLS tunnels.




Figure 2-10: TLOC Updates.

 

The example below shows the OMP TLOC update generated by vEdge about its local TLOCs. Note that the restrict TLOC attribute states that the receiving vEdge should only try to establish an IPSec tunnel between the same colors.

 

vEdge-1# show omp tlocs advertised detail

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 mpls

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.200.0.101

     public-port       12346

     private-ip        10.200.0.101

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

            ADVERTISED TO:

peer    10.100.100.13

    Attributes:

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.200.0.101

     public-port       12346

     private-ip        10.200.0.101

     private-port      12346

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

     gen-id            0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border              not set

     unknown-attr-len  not set

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 public-internet

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.100.0.101

     public-port       12346

     private-ip        10.100.0.101

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

            ADVERTISED TO:

peer    10.100.100.13

    Attributes:

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.100.0.101

     public-port       12346

     private-ip        10.100.0.101

     private-port      12346

     public-ip       ::

     public-port     0

     private-ip      ::

     private-port     0

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

     gen-id            0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border              not set

     unknown-attr-len  not set

vEdge-1#

Example 2-14: TLOCs Advertised by vEdge-1.


The example below shows the TLOC generated by vEdge-1 and reflected by vSamrt from the vEdge-3 perspective.

  

vEdge-3# show omp tlocs received

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 mpls

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.13

status          C,I,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.200.0.101

     public-port       12346

     private-ip        10.200.0.101

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

 

---------------------------------------------------

tloc entries for 10.100.100.101

                 public-internet

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            10.100.100.13

status          C,I,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.100.0.101

     public-port       12346

     private-ip        10.100.0.101

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           10

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000008

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

 

---------------------------------------------------

tloc entries for 10.100.100.103

                 mpls

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.200.0.103

     public-port       12346

     private-ip        10.200.0.103

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           30

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000007

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

 

---------------------------------------------------

tloc entries for 10.100.100.103

                 public-internet

                 ipsec

---------------------------------------------------

            RECEIVED FROM:

peer            0.0.0.0

status          C,Red,R

loss-reason     not set

lost-to-peer    not set

lost-to-path-id not set

    Attributes:

     attribute-type    installed

     encap-key         not set

     encap-proto       0

     encap-spi         258

     encap-auth        sha1-hmac,ah-sha1-hmac

     encap-encrypt     aes256

     public-ip         10.100.0.103

     public-port       12346

     private-ip        10.100.0.103

     private-port      12346

     public-ip         ::

     public-port       0

     private-ip        ::

     private-port      0

     bfd-status        up

     domain-id         not set

     site-id           30

     overlay-id        not set

     preference        0

     tag               not set

     stale             not set

     weight            1

     version           3

    gen-id             0x80000007

     carrier           default

     restrict          1

     on-demand          0

     groups            [ 0 ]

     bandwidth         0

     qos-group         default-group

     border             not set

     unknown-attr-len  not set

vEdge-3#

Example 2-15: TLOCs Received by vEdge-3.

 

 

Using information received from TLOC updates, vEdges can build IPSEC tunnels between themselves. You can find current IPsec peers by selecting Real Time from the left menu and selecting IPsec Outbound Connections in the Device Options: field.


Figure 2-11: GUI-based vEdge Ipsec Tunnel Verifications.

 

You can also monitor device-specific Ipsec tunnels by selecting the Tunnel option under the WAN section.


Figure 2-12: GUI-based vEdge Control Connections Verifications.


vEdges sends user data over these IPsec tunnels and they also monitor the tunnel quality using a slightly modified Bidirect Forwarding Detection (BFD) solution.


Figure 2-13: Ipsec Tunnels Between vEdges.

 

BFD sessions can be seen by selecting Real Time from the left menu and selecting IPsec Outbound Connections in the Device Options: field.


Figure 2-14: GUI-based vEdge Control Connections Verifications.

 

The SD-WAN infra is now ready. The focus of the next chapter is to show how to implement customers into our example network.

I have published these four books. You may find those useful.



2 comments:

  1. I found sdwan blog by chance and was pleasantly surprised. I had no idea there was so much written about SD-WAN; I am grateful for the article, it has opened my eyes to how this technology can help network providers.

    Also Read my article on SD-WAN and How Banking & Financial sector uses SDWAN

    ReplyDelete
  2. Great article is you aren't using version 18x. 20x requires edge wan serial file.

    ReplyDelete

Note: only a member of this blog may post a comment.