Deployment Guide

Switch Hardware Selection

We have verified and therefore recommend using the switch model listed in Aether-verified Switch Hardware. Other Stratum-enabled switches listed in White Box Switch Hardware should also work in theory but more integration work may be required.

To use the P4 UPF, you must use fabric switches based on the Intel (formerly Barefoot) Tofino chipset. There are two variants of this switching chipset, with different resources and capabilities. The Dual Pipe Tofino ASIC is less expensive, while the Quad Pipe Tofino ASIC has more chip resources and a faster embedded system with more memory and storage.

The P4 UPF and SD-Fabric features run within the constraints of the Dual Pipe system for production deployments, but for development of features in P4, the larger capacity of the Quad Pipe is desirable.

These switches feature 32 QSFP+ ports capable of running in 100GbE, 40GbE, or 4x 10GbE mode (using a split DAC or fiber cable) and have a 1GbE management network interface.

See also the Rackmount of Equipment for how the Fabric switches should be rack-mounted to ensure proper airflow within a rack.

Deployment Overview

SD-Fabric is released with Helm chart and container images. We recommend using Kubernetes and Helm to deploy SD-Fabric. Here’s a list of high level steps required to deploy SD-Fabric:

Provision switch

We first need to install operating system with Docker and Kubernetes on the bare-metal switches.
Prepare switches as special Kubernetes nodes

Kubernetes label and taint are used to configure switches as special Kubernetes worker nodes. This is to make sure we deploy Stratum (and only Stratum) on switches.
Prepare ONOS network configuration

Network configuration defines properties such as switch pipeconf, subnet and VLAN.
Prepare Stratum chassis configuration for each switch

Chassis config defines switch properties such as port speed and breakout.
Install SD-Fabric using Helm

Finally, we are going to install SD-Fabric with the information we prepared in Step 1 to 5.

Step 1: Access to the switch console

There are two ways to access the switch console:

Access via Baseboard Management Controller(BMC)

Access via console interface

Access via the BMC

For some platforms, you can connect to an embedded system loaded with BMC system. For example, Wedge100BF series is loaded with OpenBMC system that allows you to connect it via SSH When the switch started, OpenBMC uses DHCP to initiate it’s IP address. You may setting up static DHCP record on your DHCP server or checkout the DHCP lease file from the DHCP server to get the IP address.

$ ssh root@[Open BMC IP]

The default user and password is root` and 0penBmc.

In the OpenBMC system you can use Serial-over-LAN(sol) to access the main board.

Access via console interface

If the platform doesn’t support BMC, you can attach your laptop/PC to the switch with a console cable and use the following command to access it:

$ screen /dev/[console device] [baud rate]

The console device may be vary, it will usually be something like ttyUSB…, tty.usb…... Please checkout the console cable vendor for more information.

The baud rate can also be vary, depends on the switch vendor. Please checkout the switch user manual or contact vendor to get the information.

Step 2: Provision Switches

We follow Open Network Install Environment (ONIE) way to install SONiC image to switch. To work with the SD-Fabric environment, we have customized the SONiC image to support related features.

You can download pre-compiled images from Github Release page

Note

If you’re not familiar with ONIE/SONiC environment, please check Getting Started to see how to install the SONiC to an ONIE supported switch.

Once SONiC is started on the switch, you need to disable SONiC services before deploy Stratum on it. Otherwise Stratum containers won’t be started.

admin@sonic$ sudo systemctl stop sonic.target sonic-delayed.taget
admin@sonic$ sudo systemctl disable sonic.target sonic-delayed.taget

Step 3: Configure switches as special Kubernetes nodes

Once the Kubernetes is ready, the Stratum application will be deployed to the switch to manage it.

Unlike server, switch has less CPU and memory resources and we should avoid deploying unnecessary workloads into switch. Besides, the Stratum application should only be deployed to all switches.

To achieve the above goals, please apply the resources to your Kubernetes cluster.

Set up Label to all switch node, e.g node-role.kubernetes.io=switch
Set up Taint with NoSchedule to all switch node, e.g node-role.kubernetes.io=switch:NoSchedule
Properly configure the NodeSelector and Toleration when deploying Stratum via DaemonSet

Example of a five nodes Kubernetes cluster, two switches and three servers

╰─$ kubectl get node -o custom-columns=NAME:.metadata.name,TAINT:.spec.taints
NAME       TAINT
compute1   <none>
compute2   <none>
compute3   <none>
leaf1      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
leaf2      [map[effect:NoSchedule key:node-role.kubernetes.io value:switch]]
╰─$ kubectl get nodes -lnode-role.kubernetes.io=switch
NAME    STATUS   ROLES    AGE   VERSION
leaf1   Ready    worker   27d   v1.18.8
leaf2   Ready    worker   27d   v1.18.8

Step 4: Prepare ONOS network configuration

See Network Configuration for instructions

Step 5: Prepare Stratum chassis configuration

See See Stratum Chassis Configuration for instructions

Step 6: Install SD-Fabric with Helm

To install SD-Fabric into your Kubernetes cluster, follow instructions described on the SD-Fabric Helm Chart README