The title of this section might seem a little
overwhelming, but it reflects the focus of the chapter. The SONiC Redis
database programming pipeline is examined through the configuration of two
Layer-2 VLANs, VLAN 10 and VLAN 20, together with their respective Layer-3
routing interfaces.
VLAN 10 and VLAN 20 represent two separate
Layer-2 broadcast domains, with each VLAN associated with its own IP subnet.
VLAN 10 uses the 10.0.10.0/24 subnet, while VLAN 20 uses the 10.0.20.0/24
subnet. Because the two hosts belong to different broadcast domains and IP
subnets, communication between them requires Layer-3 routing. In this example,
the routing is performed locally by Leaf-1 through the routing interfaces
configured for the two VLANs.
Figure 5-15 depicts the basic building blocks
and IP addressing scheme used in this example. PC1, with IP address
10.0.10.101/24, belongs to VLAN 10 and is connected to the Leaf-1 Ethernet0
interface. PC2, with IP address 10.0.20.101/24, belongs to VLAN 20 and is
connected to the Leaf-1 Ethernet1 interface. The routing interfaces for VLAN 10
and VLAN 20 use IP addresses 10.0.10.1/24 and 10.0.20.1/24, respectively.
The configuration process is first examined
from the database perspective. The changes made to CONFIG_DB, APPL_DB, and the
config_db.json file are analyzed as the VLANs, and their routing interfaces are
configured. This time, the sonic-cli is used instead of the SONiC Linux bash
CLI.
The configuration is then traced further down
the SONiC pipeline. The corresponding entries created in ASIC_DB are examined,
including the relationships between the SAI objects representing the VLANs,
routing interfaces, ports, and other required objects. The ASIC programming
follows the same pipeline described earlier in this chapter, ultimately
translating the configuration into SAI objects that are programmed into the
hardware ASIC.
Figure 5-15: Vlans and Routing
Interfaces - Topology.
Configuring VLANs and
Routing Interfaces Using sonic-cli
The sonic-cli
provides a traditional CLI-style switch configuration experience. From the
administrator's perspective, VLANs, interfaces, IP addresses, and Layer-3
routing can be configured using commands like those found on traditional
network switches. The complexity of the underlying SONiC configuration and ASIC
programming pipeline is hidden behind this CLI.
Figure 5-16
illustrates the configuration of VLAN 20 and its Layer-3 routing interface. The
interface Vlan 20 command is used to enter the VLAN interface configuration
mode, after which the IP address 10.0.20.1/24 is assigned using the ip address
10.0.20.1/24 command. This IP address serves as the Layer-3 interface for the
10.0.20.0/24 subnet and provides the routing interface for the VLAN 20
broadcast domain.
The figure
also shows how Ethernet1 is associated with VLAN 20. The interface
configuration mode is entered with interface Ethernet 1, followed by switchport
access vlan 20 to configure the interface as an access port belonging to VLAN
20. If interfaces are administratively disabled by default in the SONiC
distribution, the interface is enabled with the no shutdown command.
From the
configuration perspective, the process therefore resembles configuration on a
traditional switch. However, the apparent simplicity of the CLI hides a
considerably more complex set of operations inside SONiC. A single CLI
configuration can result in changes to CONFIG_DB, updates to APPL_DB, creation
of SAI objects in ASIC_DB, and, ultimately, programming of the hardware ASIC.
These operations are performed by the SONiC components described earlier in
this chapter; they are not exposed to the administrator through the sonic-cli
interface.
Basic
configuration verification can also be performed directly from the CLI.
Commands such as show vlan, show ip interfaces, and show ip route provide a
high-level view of the configured VLANs, Layer-3 interfaces, and routing
information.
Figure 5-16: Configuring Routing
Interface for VLAN 20.
CONFIG_DB, APPL_DB, and Config_db.json
Figure 5-17 shows what happens behind the
scenes when a VLAN routing interface is configured using sonic-cli. As soon as
Enter is pressed after each configuration command, the corresponding
configuration is written to CONFIG_DB.
The configuration creates the
VLAN_INTERFACE|Vlan20|10.0.20.1/24 entry. The VLAN interface name and IP prefix
are encoded directly in the Redis key. The NULL/NULL value indicates that no
additional field-value attributes are associated with this configuration entry.
The intfmgrd daemon, the SONiC interface
manager, monitors the VLAN interface configuration in CONFIG_DB and translates
it into the application-specific representation used by APPL_DB. In this
example, the corresponding INTF_TABLE:Vlan20:10.0.20.1/24 entry contains attributes
such as family, scope, and origin.
The family attribute identifies the address
family as IPv4. The origin attribute indicates that the address is statically
configured rather than learned dynamically from a routing protocol or another
peer. The scope attribute is set to global, indicating that the address has
global scope within the Linux networking model.
The configuration then continues through the
normal SONiC programming pipeline, where the APPL_DB information is processed
by the appropriate orchestration components and eventually results in the
corresponding SAI objects being created in ASIC_DB. The ASIC programming then
follows the same pipeline described earlier in this chapter, ultimately
programming the configuration into the hardware ASIC.
Once the configuration is complete, the write
memory command saves the current configuration to config_db.json. This file
serves as the persistent startup configuration and can be used to restore the
configuration when SONiC is restarted.
The important point is that none of this
database-level complexity is exposed to the administrator through the sonic-cli
configuration commands. The CLI provides a traditional switch configuration
interface, while SONiC internally translates the configuration through
CONFIG_DB, APPL_DB, ASIC_DB, SAI, and finally the hardware ASIC.
Figure 5-17: Configuring Routing
Interface for VLAN 20.
ASIC_DB
Figure 5-18 illustrates the relationships
between the ASIC_DB objects created for inter-VLAN routing. The left side of
the figure represents VLAN 10, while the right side represents VLAN 20. The
middle of the figure contains objects shared by both VLANs, including the
switch and virtual router objects.
Both VLAN routing interfaces are represented
by SAI_OBJECT_TYPE_ ROUTER_INTERFACE objects and are associated with the same
virtual router. For VLAN 10, the router interface is oid:0x6000000000a6a, while
the router interface for VLAN 20 is oid:0x6000000000a6d. Both reference the
same SAI_ROUTER_INTERFACE_ATTR_VIRTUAL_ROUTER_ID, oid :0x300000000003a. This
shared virtual router allows both VLAN interfaces to participate in the same
Layer-3 routing context.
Each router interface is also associated with
its corresponding VLAN object through SAI_ROUTER_INTERFACE_ATTR_VLAN_ID. The
VLAN 10 router interface references VLAN object oid:0x26000000000a48, while the
VLAN 20 router interface references VLAN object oid:0x26000000000a6c. Both
router interfaces use the same source MAC address, 0C:9A:32:79:00:0A.
The routing table contains two route entries
for each VLAN. The first entry represents the directly connected subnet. For
VLAN 10, the route is 10.0.10.0/24, and for VLAN 20 it is 10.0.20.0/24. These
routes reference the corresponding VLAN router interfaces through
SAI_ROUTE_ENTRY_ ATTR_NEXT_HOP_ID. The 10.0.10.0/24 route therefore points to
router interface oid:0x6000000000a6a, while the 10.0.20.0/24 route points to
router interface oid:0x6000000000a6d.
The second route for each VLAN is a host
route for the IP address configured on the VLAN routing interface itself:
10.0.10.1/32 and 10.0.20.1/32. Both entries use oid:0x1000000000001 as their
NEXT_HOP_ID and specify SAI_PACKET_ACTION_FORWARD. These /32 routes represent
the IP addresses owned by the switch. For example, when a packet destined for
10.0.20.1 is received, it is treated as local traffic rather than being
forwarded toward another host. The packet is therefore delivered to the
switch's control-plane networking stack, where it can be processed by the
appropriate application, such as an FRR daemon.
A relationship can therefore be seen in the
ASIC_DB representation: each VLAN has its own VLAN object and VLAN router
interface, but both router interfaces share the same virtual router. The
directly connected routes then point to their respective router interfaces,
while the /32 routes for the switch's own IP addresses use a separate next-hop
object for local handling.
The ASIC_DB route entries also contain the
switch identifier oid:0x21000000000000 and the virtual router identifier
oid:0x300000000003a as part of the route-entry key. These identifiers establish
the switch and routing context to which each route belongs.
Figure 5-18 provides a view of how the
high-level concept of two VLAN broadcast domains and their Layer-3 interfaces
is represented as a collection of interrelated SAI objects in ASIC_DB. The
relationships between the VLAN objects, router interfaces, virtual router, and
route entries form the hardware-oriented representation of the inter-VLAN
routing configuration.
Chapter Summary
This concludes Chapter 5, which examined how
Layer-2 and Layer-3 connectivity is established between two hosts connected to
the same SONiC switch. The chapter first showed how VLANs and host-facing
interfaces are configured to provide intra-VLAN communication within the same
Layer-2 broadcast domain. It then examined how routing interfaces are
configured to enable inter-VLAN communication between different VLANs and IP
subnets.
The configuration was followed through the
SONiC programming pipeline, from the sonic-cli and CONFIG_DB, through APPL_DB
and the SONiC orchestration components, to the SAI objects represented in
ASIC_DB and ultimately the hardware ASIC. This demonstrated how a simple switch
configuration command hides a sequence of database updates, daemon operations,
SAI object creation, and hardware programming.
The result is a complete view of how SONiC
provides both intra-VLAN and inter-VLAN connectivity for hosts connected to the
same switch, and how the high-level configuration is translated into the
objects and relationships required for packet forwarding.
Appendix Ch5-2
This appendix provides the configuration and
Redis database outputs used to analyze the VLAN and inter-VLAN routing
configuration presented in Chapter 5. The outputs show how the configuration is
represented at different stages of the SONiC configuration pipeline, from the
persistent config_db.json file and CONFIG_DB through APPL_DB to the SAI objects
stored in ASIC_DB.
The examples focus primarily on VLAN 20, its
routing interface 10.0.20.1/24, and the associated Ethernet1 interface.
Additional CONFIG_DB entries, including device metadata, VLAN, and VLAN member
configuration, are included to provide the necessary context for the
relationships discussed in the chapter.
The ASIC_DB output includes the
router-interface objects and their object identifiers (OIDs), allowing the
relationships between the VLAN interfaces, VLAN objects, and shared virtual
router to be correlated with the explanations and figures in Chapter 5.
admin@sonic:~$
sonic-db-dump -n CONFIG_DB --pretty -k "VLAN_INTERFACE|Vlan20*"
{
"VLAN_INTERFACE|Vlan20": {
"expireat": 1786882177.7618697,
"ttl": -0.001,
"type": "hash",
"value": {
"NULL": "NULL"
}
},
"VLAN_INTERFACE|Vlan20|10.0.20.1/24": {
"expireat": 1786882177.7618823,
"ttl": -0.001,
"type": "hash",
"value": {
"NULL": "NULL"
}
}
}
admin@sonic:~$
sonic-db-dump -n APPL_DB --pretty -k "INTF_TABLE:Vlan20*"
{
"INTF_TABLE:Vlan20": {
"expireat": 1786882214.629678,
"ttl": -0.001,
"type": "hash",
"value": {
"NULL": "NULL"
}
},
"INTF_TABLE:Vlan20:10.0.20.1/24": {
"expireat": 1786882214.6296875,
"ttl": -0.001,
"type": "hash",
"value": {
"family": "IPv4",
"origin": "STATIC",
"scope": "global"
}
}
}
admin@sonic:~$
admin@sonic:~$
sonic-db-dump -n ASIC_DB --pretty -k
"ASIC_STATE:SAI_OBJECT_TYPE_ROUTER_INTERFACE:*"
{
"ASIC_STATE:SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x60000000009c7":
{
"expireat": 1786882264.3216379,
"ttl": -0.001,
"type": "hash",
"value": {
"SAI_ROUTER_INTERFACE_ATTR_MTU": "9100",
"SAI_ROUTER_INTERFACE_ATTR_TYPE":
"SAI_ROUTER_INTERFACE_TYPE_LOOPBACK",
"SAI_ROUTER_INTERFACE_ATTR_VIRTUAL_ROUTER_ID":
"oid:0x300000000003a"
}
},
"ASIC_STATE:SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000a6a":
{
"expireat": 1786882264.3215926,
"ttl": -0.001,
"type": "hash",
"value": {
"SAI_ROUTER_INTERFACE_ATTR_MTU": "9100",
"SAI_ROUTER_INTERFACE_ATTR_NAT_ZONE_ID": "0",
"SAI_ROUTER_INTERFACE_ATTR_SRC_MAC_ADDRESS":
"0C:9A:32:79:00:0A",
"SAI_ROUTER_INTERFACE_ATTR_TYPE":
"SAI_ROUTER_INTERFACE_TYPE_VLAN",
"SAI_ROUTER_INTERFACE_ATTR_V4_MCAST_ENABLE":
"false",
"SAI_ROUTER_INTERFACE_ATTR_VIRTUAL_ROUTER_ID":
"oid:0x300000000003a",
"SAI_ROUTER_INTERFACE_ATTR_VLAN_ID":
"oid:0x26000000000a48"
}
},
"ASIC_STATE:SAI_OBJECT_TYPE_ROUTER_INTERFACE:oid:0x6000000000a6d":
{
"expireat": 1786882264.3216293,
"ttl": -0.001,
"type": "hash",
"value": {
"SAI_ROUTER_INTERFACE_ATTR_MTU": "9100",
"SAI_ROUTER_INTERFACE_ATTR_NAT_ZONE_ID": "0",
"SAI_ROUTER_INTERFACE_ATTR_SRC_MAC_ADDRESS":
"0C:9A:32:79:00:0A",
"SAI_ROUTER_INTERFACE_ATTR_TYPE":
"SAI_ROUTER_INTERFACE_TYPE_VLAN",
"SAI_ROUTER_INTERFACE_ATTR_V4_MCAST_ENABLE":
"false",
"SAI_ROUTER_INTERFACE_ATTR_VIRTUAL_ROUTER_ID":
"oid:0x300000000003a",
"SAI_ROUTER_INTERFACE_ATTR_VLAN_ID":
"oid:0x26000000000a6c"
}
}
}admin@sonic:~$
admin@sonic:~$ sonic-db-dump -n CONFIG_DB --pretty -k
'DEVICE_METADATA|localhost'
{
"DEVICE_METADATA|localhost": {
"expireat": 1786882935.9643338,
"ttl": -0.001,
"type": "hash",
"value": {
"default_config_profile":
"l3",
"frr_mgmt_framework_config":
"true",
"hostname": "Leaf-1",
"hwsku":
"DellEMC-S5248f-P-25G-DPB",
"mac":
"0c:9a:32:79:00:0a",
"platform":
"x86_64-kvm_x86_64-r0",
"type":
"LeafRouter"
}
}
}
admin@sonic:~$
}admin@sonic:~$sonic-db-dump
-n CONFIG_DB --pretty -k 'PORT|Ethernet1''
{
"PORT|Ethernet1": {
"expireat": 1786883135.760981,
"ttl": -0.001,
"type": "hash",
"value": {
"NULL": "NULL",
"access_vlan": "20",
"admin_status": "up",
"adv_speeds": "all",
"alias": "Eth1/2",
"autoneg": "off",
"description":
"**PC2**",
"fec": "none",
"index": "2",
"lanes": "50",
"link_training":
"off",
"mtu": "9100",
"speed": "25000",
"unreliable_los":
"auto"
}
}
}
admin@sonic:~$
dmin@sonic:~$
sudo cat /etc/sonic/config_db.json
"DEVICE_METADATA":
{
"localhost": {
"default_config_profile":
"l3",
"frr_mgmt_framework_config": "true",
"hostname":
"Leaf-1",
"hwsku":
"DellEMC-S5248f-P-25G-DPB",
"mac": "0c:9a:32:79:00:0a",
"platform":
"x86_64-kvm_x86_64-r0",
"type":
"LeafRouter"
}
},
"Ethernet1":
{
"access_vlan":
"20",
"admin_status":
"up",
"adv_speeds":
"all",
"alias":
"Eth1/2",
"autoneg":
"off",
"description":
"**PC2**",
"fec": "none",
"index": "2",
"lanes": "50",
"link_training":
"off",
"mtu": "9100",
"speed":
"25000",
"unreliable_los":
"auto"
},
"VLAN": {
"Vlan10": {
"members": [
"Ethernet0"
],
"vlanid": "10"
},
"Vlan20": {
"admin_status":
"up",
"autostate":
"enable",
"description":
"second_vlan",
"members": [
"Ethernet1"
],
"vlanid": "20"
}
},
"VLAN_INTERFACE": {
"Vlan10": {},
"Vlan10|10.0.10.1/24": {},
"Vlan20": {},
"Vlan20|10.0.20.1/24": {}
},
"VLAN_MEMBER": {
"Vlan10|Ethernet0": {
"tagging_mode":
"untagged"
},
"Vlan20|Ethernet1": {
"tagging_mode":
"untagged"
}
},
