Switch Hardware Selection
We have verified and therefore recommend using the switch model listed in Aether-verified Switch Hardware. Other Stratum-enabled switches listed in White Box Switch Hardware should also work in theory but more integration work may be required.
To use the P4 UPF, you must use fabric switches based on the Intel (formerly Barefoot) Tofino chipset. There are two variants of this switching chipset, with different resources and capabilities. The Dual Pipe Tofino ASIC is less expensive, while the Quad Pipe Tofino ASIC has more chip resources and a faster embedded system with more memory and storage.
The P4 UPF and SD-Fabric features run within the constraints of the Dual Pipe system for production deployments, but for development of features in P4, the larger capacity of the Quad Pipe is desirable.
These switches feature 32 QSFP+ ports capable of running in 100GbE, 40GbE, or 4x 10GbE mode (using a split DAC or fiber cable) and have a 1GbE management network interface.
See also the Rackmount of Equipment for how the Fabric switches should be rack-mounted to ensure proper airflow within a rack.
SD-Fabric is released with Helm chart and container images. We recommend using Kubernetes and Helm to deploy SD-Fabric. Here’s a list of high level steps required to deploy SD-Fabric:
We first need to install operating system with Docker and Kubernetes on the bare-metal switches.
Prepare switches as special Kubernetes nodes
taintare used to configure switches as special Kubernetes worker nodes. This is to make sure we deploy Stratum (and only Stratum) on switches.
Prepare ONOS network configuration
Network configuration defines properties such as switch pipeconf, subnet and VLAN.
Prepare Stratum chassis configuration for each switch
Chassis config defines switch properties such as port speed and breakout.
Install SD-Fabric using Helm
Finally, we are going to install SD-Fabric with the information we prepared in Step 1 to 5.
Step 1: Access to the switch console
There are two ways to access the switch console:
Access via Baseboard Management Controller(BMC)
Access via console interface
Access via the BMC
For some platforms, you can connect to an embedded system loaded with BMC system. For example, Wedge100BF series is loaded with OpenBMC system that allows you to connect it via SSH When the switch started, OpenBMC uses DHCP to initiate it’s IP address. You may setting up static DHCP record on your DHCP server or checkout the DHCP lease file from the DHCP server to get the IP address.
$ ssh root@[Open BMC IP]
The default user and password is root` and 0penBmc.
In the OpenBMC system you can use Serial-over-LAN(sol) to access the main board.
Access via console interface
If the platform doesn’t support BMC, you can attach your laptop/PC to the switch with a console cable and use the following command to access it:
$ screen /dev/[console device] [baud rate]
The console device may be vary, it will usually be something like ttyUSB…, tty.usb…... Please checkout the console cable vendor for more information.
The baud rate can also be vary, depends on the switch vendor. Please checkout the switch user manual or contact vendor to get the information.
Step 2: Provision Switches
We follow Open Network Install Environment (ONIE) way to install SONiC image to switch. To work with the SD-Fabric environment, we have customized the SONiC image to support related features.
You can download pre-compiled images from Github Release page
If you’re not familiar with ONIE/SONiC environment, please check Getting Started to see how to install the SONiC to an ONIE supported switch.
Once SONiC is started on the switch, you need to disable SONiC services before deploy Stratum on it. Otherwise Stratum containers won’t be started.
admin@sonic$ sudo systemctl stop sonic.target sonic-delayed.taget admin@sonic$ sudo systemctl disable sonic.target sonic-delayed.taget
Step 3: Configure switches as special Kubernetes nodes
Once the Kubernetes is ready, the Stratum application will be deployed to the switch to manage it.
Unlike server, switch has less CPU and memory resources and we should avoid deploying unnecessary workloads into switch. Besides, the Stratum application should only be deployed to all switches.
To achieve the above goals, please apply the resources to your Kubernetes cluster.
Set up Label to all switch node, e.g
Set up Taint with
NoScheduleto all switch node, e.g
Properly configure the
Tolerationwhen deploying Stratum via DaemonSet
Example of a five nodes Kubernetes cluster, two switches and three servers
╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints NAME TAINT compute1 <none> compute2 <none> compute3 <none> leaf1 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] leaf2 [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]] ╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch NAME STATUS ROLES AGE VERSION leaf1 Ready worker 27d v1.18.8 leaf2 Ready worker 27d v1.18.8
Step 4: Prepare ONOS network configuration
See Network Configuration for instructions
Step 5: Prepare Stratum chassis configuration
See See Stratum Chassis Configuration for instructions
Step 6: Install SD-Fabric with Helm
To install SD-Fabric into your Kubernetes cluster, follow instructions described on the SD-Fabric Helm Chart README