Introduction
In this chapter, we
examine what happens under the hood when we configure a Layer-2 VLAN on a SONiC
switch, enable two host-facing interfaces, and associate them with the
previously created VLAN as untagged access ports. Figure 5-1 illustrates the
conceptual configuration in which Host-1 and Host-2 are connected to interfaces
Eth1/1 and Eth1/2, respectively. Both interfaces are members of VLAN 10, which
at this stage functions as a Layer-2 broadcast domain. The host-facing
interfaces are single-lane 25-Gigabit Ethernet interfaces.
We begin by creating VLAN 10 and examining the complete SONiC processing sequence, from the user's VLAN configuration to ASIC programming. Next, we study what happens when the interfaces are enabled and associated with VLAN 10. Finally, we verify that Host-1 and Host-2 have IP connectivity through Leaf-1. Although this is a very basic configuration example, many SONiC components participate in the process. The database examples presented in this chapter are taken from the lab environment. The communication between processes is explained at a conceptual level because the focus is on SONiC operation rather than the underlying Linux communication mechanisms.
Figure
5-1: Host Connectivity.
Step 1: Configure Vlan 10
Figure 5-2 depicts the
conceptual sequence of processes and events that occur when the administrator
creates a Layer-2 VLAN on a SONiC switch. In the SONiC distribution used in
this book, VLAN 10 is created by entering the SONiC command config vlan add 10
in a Linux Bash shell (phase 1). The command updates
CONFIG_DB (phase 2).
From GONFIG_DB to APPL_DB
When CONFIG_DB is
updated, the Redis server generates a notification indicating that the VLAN
table has changed. The notification contains information about the newly
created VLAN 10.
The vlanmgrd
daemon, which belongs to the Configuration Manager component in the SWSS
container, subscribes to the VLAN table in CONFIG_DB. Therefore, it receives
the notification over a pre-established Inter-Process Communication (IPC)
channel implemented as a UNIX domain socket (phase 3). The Process Identifiers
(PIDs) shown in Figure 5-2 are conceptual and are included only to identify the
communicating processes. For example, the redis-server and vlanmgrd daemons
each run as separate Linux processes and therefore have their own unique PID. Communication
between the processes takes place through a UNIX domain socket, whose endpoints
are represented by file descriptors (FDs) owned by the communicating processes.
In the example shown in
Figure 5-2, PID 1329 identifies the redis-server process, while FD 411
identifies the UNIX domain socket endpoint owned by that process. Likewise, PID
1845 identifies the vlanmgrd process, while FD 7 identifies the corresponding UNIX
domain socket endpoint owned by vlanmgrd. The socket connection is therefore
established between the socket endpoints represented by the file descriptors,
while the PIDs simply identify the processes that own those endpoints.
After receiving the
notification, vlanmgrd obtains the configuration associated with the updated
VLAN entry (phase 4). Depending on the SONiC distribution and implementation,
the required information may be available directly in the notification message or
it may be read from CONFIG_DB. In the example shown in Figure 5-2, vlanmgrd
reads the required information from the notification message.
Next, vlanmgrd converts
the received configuration into the format expected by APPL_DB and publishes
the corresponding entry to the Redis server using the existing UNIX domain
socket connection (phase 5). Translation is required because the Redis server
does not modify or translate the information it receives. Its role is simply to
store the received data in the selected database. After receiving the
translated entry from vlanmgrd, the Redis server stores it in APPL_DB (phase
6).
Figure
5-2: Updating Redis Databases - from CONFIG_DB
to APPL_DB.
Verifying CONFIG_DB and APPL_DB
The Redis server used
by SONiC maintains several logical databases, each serving a specific role in
the processing pipeline. Separating configuration, operational, and
hardware-related information into different databases allows SONiC components
to exchange only the information relevant to their function. Each database is
organized into tables. A table contains one or more keys, and each key stores
one or more field-value pairs that define the attributes of the corresponding
object.
Example 5-1 shows the
VLAN table entry for VLAN 10 in CONFIG_DB, while Example 5-2 shows the
corresponding VLAN object in APPL_DB.
The CONFIG_DB database
stores the configuration requested by the administrator. It represents the
desired state of the system and therefore contains only the information
required to define the object. In this example, the VLAN object contains only
the VLAN identifier (vlanid) that was specified by the config vlan add 10
commands.
The APPL_DB database
stores operational objects that have been prepared for further processing by
downstream SONiC components. When `vlanmgrd` receives the configuration from
CONFIG_DB, it validates the information and translates it into the format
expected by APPL_DB. As a result, the APPL_DB entry contains additional
field-value pairs describing the operational state of the VLAN, including the
administrative status, operational status, MAC address, MTU, operational MTU,
and the time of the last state change. These attributes provide the information
required by subsequent SONiC components to continue processing the VLAN
configuration.
admin@sonic:~$
sonic-db-dump -n CONFIG_DB --pretty -k 'VLAN|Vlan10'
{
"VLAN|Vlan10": {
"expireat": 1785914956.0346863,
"ttl": -0.001,
"type": "hash",
"value": {
"vlanid": "10"
}
}
Example
5-1: CONFIC_DB - Vlan Table Key
VLAN|Vlan10.
admin@sonic:~$ sonic-db-dump
-n APPL_DB --pretty -k 'VLAN_TABLE:Vlan10'
{
"VLAN_TABLE:Vlan10": {
"expireat": 1785915016.382373,
"ttl": -0.001,
"type": "hash",
"value": {
"admin_status": "up",
"last_change":
"1785914543",
"mac":
"0c:9a:32:79:00:0a",
"mtu": "9100",
"oper_mtu": "9100",
"oper_status": "down"
}
}
Example
5-2: APPL_DB - Vlan Table Key VLAN_TABLE:Vlan10.
Note: The output generated by the sonic-db-dump
utility includes the metadata fields expireat, ttl, and type.
These describe the Redis object rather than the VLAN configuration itself. The
expireat and ttl fields indicate whether the object expires, while type
identifies the Redis data structure used to store the object. In these
examples, the value hash indicates that the VLAN object is stored as a Redis
hash containing multiple field-value pairs.
From APPL_DB to ASIC_DB
Figure 5-3 depicts the
conceptual processing sequence that takes place after the Redis server updates
the VLAN_TABLE in APPL_DB. Once the update has been stored, the Redis server
generates a notification describing the change. Because orchagent subscribes to
the VLAN_TABLE in APPL_DB, it receives the notification (phase 7) and retrieves
the corresponding VLAN object from APPL_DB (phase 8).
Within orchagent, the
VLAN object is processed by VlanOrch, an orchestration component
responsible for VLAN objects. VlanOrch invokes the standard SAI API by calling sai_create_vlan(),
passing the VLAN identifier as a parameter (phase 9). The SAI API call is then
serialized by libsairedis into the Redis representation used for ASIC
objects (phase 10). This translation is necessary because the Redis server does
not interpret or translate SAI API calls. Its role is simply to store the
information it receives in the appropriate database.
After the SAI request
has been translated, orchagent sends the resulting Redis object to the Redis
server using the existing UNIX domain socket connection (phase 11). The Redis
server stores the received object in ASIC_DB (phase 12).
Unlike CONFIG_DB and
APPL_DB, which identify objects using descriptive names such as Vlan10, ASIC_DB
represents hardware objects by assigning each one a unique Object Identifier
(OID). From the ASIC's perspective, every hardware resource is represented as
an object. Consequently, the ASIC does not identify the VLAN by the name
Vlan10; instead, it refers to the VLAN by its assigned OID. In the example
shown in Figure 5-3, VLAN 10 is represented by the object oid:0x26000000000a65.
Figure
5-3: Updating Redis Databases - from APPL_DB
to ASIC_DB.
Verifying ASIC_DB
Example
5-3 shows the VLAN objects stored in ASIC_DB. The wildcard character (*) at the
end of the sonic-db-dump
command is used because the Object Identifier (OID)
assigned to VLAN 10 is not known in advance. Therefore, the command displays
all VLAN objects currently stored in ASIC_DB.
Notice
that the default VLAN (VLAN 1) is also present. Each VLAN is represented by a
separate ASIC object whose key contains a unique OID. The actual VLAN number is
not encoded in the key itself but is stored as the attribute
SAI_VLAN_ATTR_VLAN_ID. In this example, the object with the key
ASIC_STATE:SAI_OBJECT_TYPE_VLAN:oid:0x26000000000a65 contains the attribute
SAI_VLAN_ATTR_VLAN_ID with the value 10, indicating that this ASIC object
represents VLAN 10.
admin@sonic:~$ sonic-db-dump
-n ASIC_DB --pretty -k 'ASIC_STATE:SAI_OBJECT_TYPE_VLAN:*'
{
"ASIC_STATE:SAI_OBJECT_TYPE_VLAN:oid:0x26000000000955": {
"expireat": 1785915438.5691075,
"ttl": -0.001,
"type": "hash",
"value": {
"NULL": "NULL",
"SAI_VLAN_ATTR_VLAN_ID":
"1"
}
},
"ASIC_STATE:SAI_OBJECT_TYPE_VLAN:oid:0x26000000000a65": {
"expireat": 1785915438.5690937,
"ttl": -0.001,
"type": "hash",
"value": {
"SAI_VLAN_ATTR_VLAN_ID":
"10"
}
}
}
Example
5-3: ASIC_DB - SAI_OBJECT_TYPE_VLAN.
Programming Vlan 10 into Hardware ASIC
Figure 5-4 illustrates
the final stage of the VLAN creation process, where VLAN 10 is programmed into
the hardware ASIC. After the Redis server stores the SAI_OBJECT_TYPE_VLAN
object for VLAN 10 in ASIC_DB, it generates a Redis notification indicating that
the ASIC database has been updated (phase 13).
The syncd daemon,
running in the Syncd container, subscribes to ASIC_STATE: SAI_OBJECT_TYPE_VLAN
and therefore receives the notification. It then retrieves the corresponding
ASIC object from ASIC_DB (phase 14). Next, syncd deserializes the Redis
representation back into the corresponding SAI operation and invokes the
standard SAI API. The request is then forwarded to the vendor SAI
implementation, which passes it to the vendor Software Development Kit (SDK)
(phase 15).
The vendor SDK
translates the standard SAI operation into the vendor-specific commands
required by the hardware ASIC. These commands are then executed by the vendor
ASIC driver, which programs VLAN 10 into the hardware (phase 16). After the
operation has been completed successfully, the vendor SAI implementation
returns the result to syncd, completing the VLAN creation process.
Note: Figure 5-4 shows
only the Object Identifier (OID) generated by SONiC. Internally, this
identifier represents a Virtual Object Identifier (VID). When the object is
programmed into the hardware ASIC, the vendor SAI implementation assigns a
hardware-specific Real Object Identifier (RID). syncd maintains the mapping
between VIDs and RIDs so that subsequent SAI operations can reference the
correct hardware object.
Example 5-4 shows the
VLAN status using the SONiC CLI. Although VLAN 10 has now been programmed into
the hardware ASIC, its status is still Inactive because no switch interfaces
have yet been configured as members of the VLAN.
Figure
5-4: Updating ASIC- from ASIC_DB to Hardware
ASIC.
Example
5-4 depicts how the Vlan 10 status using sonic CLI. The status is still
Inactive because we haven’t yet configugre any interface as a member of Vlan10.
sonic# show Vlan
Q: A - Access
(Untagged), T - Tagged
NUM
Status Q
Ports
Autostate Dynamic
10
Inactive
Enable
Example
5-4: SONIC-CLI: VLAN 10.
Summary
Figure 5-5 summarizes
the complete processing sequence through which VLAN 10 is created and
programmed into the switch ASIC.
The process begins when
the network administrator creates VLAN 10 by entering the SONiC command config vlan add 10.
The command updates CONFIG_DB with the user-defined VLAN configuration. After
the update, the Redis server generates a notification and sends it to the
vlanmgrd daemon in the SWSS container over the existing UNIX domain socket
connection.
The vlanmgrd daemon
retrieves the updated VLAN configuration, validates it, and translates it into
the operational format expected by APPL_DB. The translated object is then
published to the Redis server, which stores it in APPL_DB and generates another
notification indicating that the VLAN_TABLE table has been updated.
The orchagent process
subscribes to the APPL_DB VLAN_TABLE table and receives the notification. It
retrieves the updated VLAN object and forwards it to the VlanOrch orchestration
component. VlanOrch invokes the standard SAI API by calling sai_create_vlan(),
passing the VLAN identifier as a parameter. The SAI request is then serialized
by libsairedis (not shown in figure) into the Redis representation used for
ASIC objects and stored in ASIC_DB through the Redis server.
After ASIC_DB has been
updated, the Redis server notifies the syncd daemon running in the Syncd
container. The syncd daemon retrieves the ASIC object from ASIC_DB,
deserializes it back into the corresponding SAI operation, and forwards the
request to the vendor SAI implementation. The vendor SAI implementation passes
the request to the vendor Software Development Kit (SDK), which translates the
standard SAI operation into vendor-specific commands. These commands are
executed by the vendor ASIC driver, programming VLAN 10 into the hardware ASIC.
After the operation completes successfully, the result is returned to syncd,
completing the VLAN creation process.
This processing
sequence demonstrates the modular architecture of SONiC. Each software
component is responsible for a specific stage of the configuration pipeline,
while the Redis databases provide a common mechanism for exchanging information
between the components. This separation enables SONiC to maintain a clear
distinction between user configuration, operational processing, and hardware
programming.
Figure
5-5: The Complete Process: Vlan 10 from SONiC
CLI to Hardware ASIC.
Note: The detailed processing sequence
presented in this chapter establishes the baseline for understanding SONiC
operation. Unless additional details are required, the remaining chapters use
the simplified processing sequence shown in Figure 5-5.
