Intro

The GlobalNOC Whitebox / Baremetal Lab

With support by the National Science Foundation (NSF), the GlobalNOC at Indiana University established a lab to evaluate whitebox switching and allow other members of the Research and Education (R&E) community to use the resources and knowledge created by the effort.

As a first step, the GlobalNOC established the lab space required and stocked it with a broad range of whitebox devices representing a multitude of vendors and chipsets.

Relationships have been established with chipset vendors, device manufacturers, Network Operating System (NOS) providers and standardization organizations like the Open Compute Project (OCP).

Through these efforts, the GlobalNOC is able to provide a variety of Network Operating Systems on a variety of hardware platforms.

The first phase in our evaluation is validating that the NOS provides traditional protocol and forwarding implementations, and confirming the platform and software combinations fulfill the requirements of researchers and R&E network engineers.

The second phase will focus on the now exposed Linux functionality, and how this can be used in R&E environments. This includes installing standard Linux networking tools on the device (iPerf, nuttcp), specific tools developed for our environments (perfSONAR), and the use of Linux management tools for device management (Ansible, Chef, Puppet, Salt).

Phase three looks at the openly available APIs to control the chipsets, and how these can be leveraged for new functionality (SAI, OpenNSL, OF-DPA). A topic of specific interest is the new monitoring capabilities that give insight into the internal performance parameters of the chipset, such as Broadcom’s Broadview stack.

In phase four, Proof of Concepts for R&E specific solutions will be implemented. One example is the SciPass project. Find more information on SciPass and other solutions on the Proof of Concepts page. We are continuously looking for additional solutions to include there.

What is whitebox / baremetal switching?

Let’s first take a look at what a switch really is. Right now, switches are most often sold as complete solutions be vendors. They have all the necessary hardware and software integrated and not openly exposed. If we look at the schematics of the OpenSwitch project, for example, we see that switches are composed of several components, and do look a bit like a traditional server with drivers for the hardware, an operating system, and applications running on top.

Traditional hardware has already started moving away from self-designed chips, and companies are more and more often integrating so-called merchant silicon, which means chipsets provided by third parties like Broadcom, Freescale or Mellanox. These chipset producers also supply drivers and software development kits (SDK) to allow the other components to use their functionality.

Despite this, all other components (NOS, Command Line Interface (CLI), forwarding and management applications, protocol implementations, etc.) are still vendor specific and not accessible.

Whitebox switching now decouples these software and hardware components and offers the customers many degrees of freedom when integrating these components. Additionally, there is a strong move to supply open source alternative to most of these layers. Diagram three shows several of the options in this space.

From bottom to top, we see:

  • The chipsets, provided by merchant silicon vendors.
  • The Original Design Manufacturers (ODM) adds the chassis, fans and power modules, and provides the bare-metal device.

The Open Network Install Environment (ONIE) Bootloader, provided by the OCP, is the de facto standard mechanism for installing different NOS on whiteboxes. Some examples include:

  • OpenNetworkLinux: An open source NOS without any applications included. Allows integration of various components or self-developed software as needed. This project is maintained by the OCP. Other examples of this approach are the Facebook Open Switching System (FBOSS) or Microsoft SONIC, running Debian Linux as the underlying operating system.
  • Full commercial NOS solutions that provide all the components of a traditional switch (CLI, protocol implementations, etc.). Vendors who provide this solution also provide support for their software. One functional difference between this and traditional vendor switches is that most NOS solutions open the underlying Linux, so that applications can be easily installed. Examples include tools like Chef, Puppet, Salt and Ansible. Examples of full commercial NOS are PicOS, OcNOS and Cumulus Linux.
  • Open source solutions. A leading example of this implementation is the OpenSwitch project.
  • Lastly, there are complete solutions like Big Switch, which consist of switch firmware fully designed for a  controller- or server-based application.

Our lab has a broad selection of devices from several vendors. For a detailed list, click the link below:

Devices:

We also provide the following Network Operating Systems:

NOSs:

Documentation, tutorials and Howtos can be found here:

Documentation:

We also provide ready to go testbeds for several areas:

Testbeds:

Use cases or Proof of Concepts:

Proof of Concepts:

You can find the project description and additional information here:

Contact

How to engage us:

If you are interested in using the lab ressources, or would like to work with us on developing a use case, please contact as at:

whitebox@grnoc.iu.edu