SD-Fabric 1.1

After one year of incubation under the ONF’s member-only license, we are excited to present the first open-source release of SD-Fabric under the Apache 2.0 license

SD-Fabric 1.1 comes with numerous new features and improvements. The main focus for this release is full integration with Aether, ONF’s private-5G edge connectivity solution for enterprises. SD-Fabric brings the many benefits of network programmability to Aether, from a hardware-accelerated highly-available P4-UPF, to fabric-wide slicing and QoS, and per-packet visibility via Inband-Network Telemetry (INT).

In this release, we provide two UPF options: the switch-based P4-UPF, offering the highest performance; and the CPU-based BESS-UPF, tailored for deployment flexibility and horizontal scalability. Both UPFs come with numerous bug fixes as well as integration with Aether 2.0 features, such as support for 5G base stations, application filtering, QoS via multi-level rate-limiting, and enhanced visibility with per-flow metrics and INT integration.

Moreover, we provide many operational improvements such as better support for enterprise networks, simplified configuration, improved stability, and reduced resource usage.

We hope the transition to open source will catalyze the adoption of SD-Fabric and attract more users and contributors from the community. Please visit the SD-Fabric documentation website to learn more about all the features SD-Fabric offers and how to get started.

Feature Highlights

Open Source Release

All SD-Fabric components are now completely open source under the Apache 2.0 license. We have published all source code, docker images, helm charts, documentation, and mailing lists.

Aether Integration

This release provides the following integrations with Aether 2.0:

  • 4G/5G slice based on P4-UPF: which can now be instantiated using the Aether management portal or APIs. In this release, we support creating only one P4-UPF slice, creation of more P4-UPF slices will come in future releases. As before, more than one slice can be created when using BESS-UPF.

  • Initial INT integration: this release adds di-metrics-exporter, a new service which fetches and aggregates real-time INT-based metrics and anomalies from Intel® DeepInsight ™, to produce per-UE health status reports to be consumed by the Aether Analytics Engine (work-in-progress). SD-fabric supports monitoring of UE traffic when using both P4-UPF and BESS-UPF.

P4-UPF

This release aims at bringing feature parity with BESS-UPF. To this end, P4-UPF now supports the following new features:

  • Application filtering: where the UPF can drop or forward traffic based on application endpoints (IPv4 prefix and port range) configured using PFCP SDF filters received from SD-Core. We support all ROC policies: ALLOW-ALL, DENY-ALL, ALLOW-PUBLIC.

  • Multi-level rate-limiting: where traffic can be policed at the application level, UE level, or slice level. Per-application and per-UE rate limits can be configured by means of PFCP QERs received from SD-Core.

BESS-UPF

  • Port ranges in app filtering rules: BESS-UPF now supports installing PDRs with SDF filters matching on more than one transport-layer port.

  • Arbitrary IPv4 prefix lengths in app filtering rules: including Aether’s allow-public rule set (which blocks traffic to RFC 1918 private subnets).

  • Per-flow metrics (experimental): when enabled, the BESS pipeline collects metrics such as throughput, latency, jitter, and packet loss. Metrics are collected for each PFCP PDR rule, allowing performance monitoring for multiple application flows for the same UE. Metrics can be exported via the Prometheus endpoint in PFCP-Agent.

Slicing & QoS

We continue to make slicing and QoS more configurable to adapt to different use cases.

  • ONOS now reads slicing parameters via netcfg: to support static configuration of arbitrary slices and traffic classes.

  • Updated config generation scripts: to automatically generate both Stratum chassis_config and ONOS netcfg file using high-level parameters.

Operations

Numerous improvements to make it easier to deploy and operate SD-Fabric.

  • Support for P4Runtime port translation: which allows hiding ASIC-specific internal port numbers from user-facing configuration files. Now, when configuring switch interfaces in both the ONOS netcfg and Stratum chassis config, users can specify front panel port/channel numbers instead of SDK port numbers.

  • Remove the management server from the data plane: user traffic is now processed by P4-UPF or BESS-UPF and then forwarded upstream (or to edge-applications) directly by the fabric switches.

  • Support base stations behind L3-routed network: supporting real-world enterprise deployments.

  • Improve support for paired-leaves topology and distributed UPF: by enabling phased-recovery by default to ensure traffic is not sent to a recovering switch until it is fully programmed, avoiding unnecessary packet drops.

Bug fixes and other improvements

  • Made primary controller election in ONOS persistent during device and cluster events, ensuring flow/group operations to always be successful.

  • Improved the UP4’s distributed UPF replication protocol, fixing a known issue that was causing missing or stale flows in some switches.

  • Implemented a reconciliation mechanisms in the ONOS device manager to facilitate recovery from network partition scenarios

  • Fixed wrong flow handling during network partition scenarios

  • Enabled ZGC garbage collector in ONOS to reduce garbage collector pauses and their effects on the cluster.

  • Upgraded Atomix to 3.1.12 and Karaf to 4.2.14

Testing

  • Built a new Go-based integration test infrastructure for PFCP-Agent, including pfcpsim, a CLI-based tool to emulate SMF interactions in tests. The new infrastructure allows testing PFCP-Agent interaction for both P4-UPF and BESS-UPF.

  • Nightly SD-Fabric integration tests now include PFCP-Agent

  • Added new nightly test cases for new P4-UPF features and persistent controller election

Development

  • Created a v1model version of the fabric-tna P4 program. The new P4 program allows sharing the same pipeconf and PTF tests as the TNA version of the program, facilitating development and testing in bmv2-based emulated environments.

Upgrade Notes

If you are upgrading from SD-Fabric 1.0, there are several substantial changes that require additional attention.

ONOS netcfg

  • We renamed all pipeconf names and replaced spgw with upf (e.g., fabric-spgw-int is now named fabric-upf-int)

  • We bumped the supported Intel® P4 Studio SDE from 9_5_0 to 9_7_0

  • S1U/N3 address and UE pool configuration for the UP4 app is no longer necessary. Please use the PFCP-Agent upf.json or helm values to specify them.

  • When using 5G base stations, make sure to set the pscEncapEnabled flag to true in the UP4’s netcfg

  • Because of P4Runtime translation, all port numbers in the ONOS netcfg should match the singleton port id used in the Stratum chassis config (instead of the ASIC-specific P4 port number).

  • See ONOS Network Config for more details

PFCP agent

  • We made UE pool configuration in the PFCP Agent mandatory. Make sure to configure ue_ip_pool, under the cpiface group, in the upf.json file (or Helm values).

  • Access IP configuration parameter is now mandatory. Make sure to configure access_ip in the upf.json file (or Helm values).

  • p4rtciface.ue_ip_pool has been removed. Use cpiface.ue_ip_pool instead.

  • Introduced slice_id field. It defaults to 0 if not specified.

  • See PFCP agent config for more details

Stratum chassis config

  • Since the Stratum’s singleton port ID can now be used in the ONOS netcfg for all port numbers (thanks to P4Runtime translation), we recommend using the following easy-to-understand convention:

    • For unchannelized ports: use the same front panel port number, e.g., for port “9/0” use singleton port ID “9”

    • For channelized ports: use the formula “port number x 100 + channel”, e.g., when configuring channels 9/0, 9/1, 9/2, and 9/3, the corresponding singleton port IDs should be 900, 901, 902, and 903

  • See Stratum port number for more details

Docker image

  • All our docker images are now hosted on DockerHub. Remember to update image registry, repo and tag if you were pulling images from the Aether Registry previously

  • We renamed the tost image to sdfabric-onos

Helm value

  • logging.karafVersion should be updated from 4.2.9 to 4.2.14

Known Issues and Limitations

  • Some INT collectors might not support topologies with dual-homed hosts.

  • In P4-UPF, configuration of slice rate limits is currently not exposed to Aether. We plan to add this in the next minor release (SD-Fabric v1.1.1).

  • In P4-UPF, we currently don’t support exporting per-port or per-flow metrics via Prometheus. As a consequence, when using the Aether monitoring dashboard, the corresponding graphs will not be populated. We plan to add this in the next major release (SD-Fabric v1.2)

  • In BESS-UPF, when using application filtering with port ranges, we don’t support ranges wider than 100 ports. Please create more application filtering rules to cover the desired range.

Component Versions

SD-Fabric ONOS

Stratum

DBUF

PFCP Agent

  • Image: omecproject/upf-epc-pfcpiface

    • For P4-UPF: master-3e29b49

    • For BESS-UPF: master-9a4d86c

  • Repo: upf

    • For P4-UPF: 3e29b49e96e39e0ba7bf4190a8b0d0aa9413b08d

    • For BESS-UPF: 9a4d86c654a9f60947ebf71d015d4781ed7a95a6

BESS UPF

Helm Chart Versions

Patches