Open networking programmability for VXLAN Data Centre infrastructures: Ansible and Cumulus Linux feasibility study

Headnote

Abstract: This proof-of-concept paper analyzes the ease of a programmability implementation inside modern VXLAN DC Spine-Leaf infrastructure through Ansible API, acting as controller configuration server, and Cumulus Linux as intermediate device Networking Operating System (NOS), conceiving what is called Open Networking Infrastructure. NetDevOps principles are used to demonstrate the implementation feasibility of this innovative sort of automated and programmatic topologies.

Keywords: Ansible; Cumulus-Linux; VXLAN; NetDevOps; open networking.

1.Introduction

Topics like VLANs, Spanning-Tree Protocol (STP), routing protocols and WAN Fat-tree Designs are now considered the "The Stone and Bronze Age" of Networking according to (Preston & Sullivan, 2017). Figure 1 shows the Four Ages of Networking in which MPLS is the technology link between the "Bronze Age" towards the "Renaissance Era". Nowadays, we are trying to evolve from "The Renaissance Age" with SDN to "The Programmable Age" using APIs, and with that, transform the traditional networking to a more scalable, effective, easy to deploy, monitor, secure, standardized, and fully programmable infrastructure (Salazar, Naranjo & Marrone, 2018).

IT Departments have been envisaging a way to move at the pace of business during last decades (Andrade, Salazar & Vintimilla, 2019). The result was the creation of new enterprise culture called DevOps, covering topics like development, execution, automation, security, agility, and proof-of-concepts scenarios before implementing the solution in production infrastructures.

In DevOps model, there are three common practices:

* Continuous Integration (CI): Common practice in which developers keep their codes in central and collaborative repository, like Git Repos. Automatic Tests are performed in those environments.

* Continuous Delivery (CD): Practice where changes in code are automatically sent to a test environment after the build stage.

* Infrastructure as Code (IaC): Practice where the IT infrastructure is provisioned and managed using scripts, as opposed to traditional networking that uses CLI or GUI.

When contemplating data networks, the term NetDevOps appears, which is nothing but the combination of programmability in the infrastructure using appropriate APIs like those shown in Fig. 2. The most used frameworks and NOS for programmability reasons are: Ansible and Cumulus Linux, bringing the Open Networking to the fore.

Modern Data Centers, as mentioned in (Naranjo & Salazar, 2017), employ underlayoverlay paradigms to solve many of their inherent problems. VXLAN is one of the most effective solutions to solve encapsulation and addressing issues, however, it could be overwhelming to deploy.

In this proof-of-concept paper, the authors unite the fundamental concepts of modern networks, like Open Networking using Cumulus Linux, and programmability with Ansible in VXLAN DC spine-leaf topology, implemented in a realistic emulated scenario, with the purpose of verifying their validity and agility.

2. Problem statement

Without a doubt, businesses are moving and changing really fast (Salazar, Venegas, Baca, Rodríguez & Marrone, 2018), therefore it is not conceivable to wait days, weeks or even months for branch offices or DC infrastructures to be ready and totally configured. Usually, when there is manual configuration, some discovery, monitor, and error validation steps are needed. Also, the troubleshooting process, in case an error appears, can become long and tedious.

NetDevOps comes into play as the right cultural and technological solution to this sort of large and complex scenarios, closely integrating people, processes, and networking technologies in an automated way.

3. Justification and importance

In DC infrastructures, the combination of switching and routing protocols with overlay encapsulation techniques builds the heart of IT topologies, but also could generate multiple points of failure and implementation delays.

NetDevOps brings a new approach to solve and implement networking infrastructures, first, using proof-of-concept scenarios before implementing them; this concept is called the DevOps-NetDevOps CI-CD Pipeline (Fig. 3).

The NetDevOps proposed solution in this investigation involves Ansible. Using its Playbook is possible to define "Build, Test, Approve, Deliver, and Deploy" stages in a local emulated DC infrastructure, and with that, automating all required configuration to implement VXLAN fabric with configuration insights and logs.

4.Theoretical framework

Network automation, part of NetDevOps, is the process of automating the configuration, management, testing, deployment and operations of physical and virtual devices within a network (Benson, Prevost & Rad, 2016).

4.1.Cumulus Linux

Cumulus Linux is a powerful Debian-based open NOS, purpose-built for automation, scalability and flexibility using web-scale principles found in the World's largest Data Centers and Campus Networks.

According to (Cavanaugh & Lumbis, 2018), Cumulus Linux provides:

* Financial scalability.

* Automation.

* Server tools for switches.

Cumulus Linux Key Networking usage purposes are:

* Ethernet Virtual Private Networks (EVPN).

* Virtual Extensible LAN (VXLAN).

* IP and BGP Unnumbered.

* VMware and OpenStack Integration.

4.2.Ansible

Ansible is an Open Source platform designed to configure and manage network devices and servers. It combines multi-node installation (deploying server configurations and batch services), ad-hoc task executions and configuration management. Additionally, Ansible is categorized as an orchestration tool in the Server Side (RedHat, 2019).

It manages nodes through SSH connectivity, and does not require any additional remote software to work properly (except for Python 2.4 or later to install it) (Singh, Thakur & Chaurasiya, 2015).

It has modules that work on JSON, and the standard output can be written in any language. It natively uses YAML scripts to describe reusable system configurations.

4.3.Cumulus Linux plus ansible environments

To automate a Cumulus-based network using Ansible, it is necessary to build a so-called Playbook. The purpose of Ansible Playbook is to translate network configurations inside Ansible programmability operations.

The structure of a Playbook is as follows:

1. Hosts File: Devices on which tasks will be executed.

2. Tasks in Playbook: Cumulus Linux commands to run on network devices.

The Ansible Playbook file extension is .yml and the "-" in the beginning of Fig. 6 command box indicates the start of a YAML formatted file.

5.Emulation of network automation with Ansible in modern DC infrastructures

This section indicates the procedure and results obtained when emulating the automation of VXLAN environment as a Data Center infrastructure using Cumulus Linux with Ansible Automation Platform.

5.1. Topology

The next topology represents a scheme of VXLAN Lightweight Network Virtualization (LNV) without needing a central controller on a bare metal switch.

This topology includes three Mellanox Spectrum switches (Spine, Leaf-1, and Leaf-2), all running Cumulus Linux. Each Leaf switch has a VLAN configured in the southbound (VLAN10) and L3 interfaces on the Spine.

Creating a logical VXLAN interface on both Leaf switches, enables the switches to operate as VTEPs (Virtual Terminal End Points). The IP address associated with VTEPs is configured with its loopback address; in Fig. 7, the loopback address is 1.1.1.1 for Leaf1, and 3.3.3.3 for Leaf2.

All routing, VXLAN, and interface configurations are executed through Ansible using a Management Server and a Management Network IP address for each device.

For this emulation, the topology shown in Fig. 7 was used.

5.2. Addressing table

5.3.Configuring Ansible Hosts and Variables

Ansible uses an Inventory file called "Hosts" to manage and provision devices. In this case, the file includes the three devices with their corresponding Management IP for SSH access. The main group named "cumulus" contains the three Multilayer Mellanox Switches with their IP addresses used to the Network Automation Controller and the devices via SSH protocol, as you can see in Fig. 8.

There is also a "vars" section which represents the parameters used for the SSH connection, for example the username, password, and private key location.

Inventory file is located in the following Ansible Network Automation Controller path:

/etc/ansible

5.4.Creating Ansible playbook with programmatic tasks

The next step is the creation of Ansible Playbook, which is the place to enlist all tasks to be configured automatically on devices:

The tasks to implement a VXLAN infrastructure are:

* Configure Interfaces with their corresponding IP addresses.

* OSPF routing between the nodes (Spine -Leaf) as the Underlay protocol.

* VTEP and service node configuration.

* VLAN assignment.

* VLAN to VXLAN Mapping (L2 Tunnel over L3 Infrastructure).

The Ansible Playbook file is located in the following path: /etc/ansible/.

1. Configure Interfaces: Physical and loopback interfaces have IPv4 addresses. For this purpose, the authors used the Cumulus' Network Command Line Utility (NCLU). With NCLU, it is possible to omit the net command at the beginning of each line.

As shown in Fig. 9, Cumulus NCLU commands must be followed by commit statement to be applied on the switch.

2. Configure OSPF routing process: Underlay Routing protocol between Spine and Leaves will be OSPF single-area protocol.

3.VTEP Configuration and service node: Service node turns the switch into a VTEP endpoint in VXLAN scenario.

* Anycast IP: It is an IP address which can be advertised from multiple locations allowing multiple devices to share the same IP address and effectively allocate load balanced traffic.

* Source node: It is the IP address which represents the Spine loopback interface.

4.Configure VLANs: For this emulation, VLANS are used only in Leaves. In this P°C Scenario it is used VLAN-10 in the VXLAN infrastructure, assigning swp6 ports as an access port in the corresponding VLAN.

5.Configure VLAN to VXLAN Mapping: The following configuration maps VLAN-10 to VXLAN VNID-2000 in Leaves (VTEPS). According to (Naranjo & Salazar, 2017), 2000 corresponds to Virtual Network Identifier (VNID). This allows to connect separated VLAN instances, allowing flexibility and better DC design environments.

5.5.Emulation results

Ansible Playbook deployment: To deploy an Ansible Playbook, it is a must to execute the following command in Management Server's terminal:

The last line indicates "changed=o" because the automated configuration is identical to the one manually configured in Ansible files. Hence, there is nothing changed, which is a good indication that Ansible Playbook (vxlan.yml) worked as expected and therefore, it is possible to safely add more devices to the automation process.

End-to-End Ping: Connectivity between end hosts as they are in the same L2 VLAN instance.

Learned MAC-Address Process: With LNV, the process of MAC-address learning is the same as in traditional L2 switch. When a host sends a packet out its interface, the MACaddress is learned at ingress and the MAC-address table is built simultaneously. If the host has not sent any packet before, and the device's MAC address does not exist in the table, the service node will replicate the packet to all VTEPs (similar to the flooding behavior when destination MAC-address is unknown).

WireShark Captures: Finally, it is possible to verify that the static VXLAN tunnel was built correctly (Leaf2 loopback IP Address: 3.3.3.3).

Elapsed Time in Ansible Executions: One of the parameters to determine if Ansible Automation Tool is a good option as Network Programmability controller is the time it takes to carry out the verification, implementation, and delivery of all configuration lines to network devices.

For testing reasons, 10 executions were made, whose response times can be seen in the following image. The time was taken from a zero setting.

According to Fig. 18, the Ansible average execution time for the case proposed in this proof-of-concept paper is 49,9 seconds.

6. Related work

As mentioned in Section I and II, technologies continue to evolve fast, and one field that witnessed those changes is networking, especially in DC areas.

Among the most relevant research works are (Yiran, Tongyang & Yidong, 2018), (Benson, Prevost & Rad, 2016) and (Masek, Stusek, Krejci, Zeman, Pokomy & Kudlacek, 2018). All of them describing automation in networking infrastructures as a necessity, as well as analyzed DevOps environments using theoretical definitions and use cases.

Another interesting work is (Salazar, Naranjo & Marrone, 2018). The authors of that investigation established new approaches to increase programmability and automation in ISP infrastructures with Segment Routing, showing that innovation not only happens in Data Centers.

The present investigation not only defines the foundation in DevOps and NetDevOps, but the authors demonstrated the feasibility of its implementation in modern DC SpineLeaf scenarios.

7. Conclusions

This Proof-of-Concept paper addressed the CI-CD pipeline in NetDevOps environments, focusing on programmability features to accelerate the deployment of VXLAN Data Centers infrastructures through Ansible Automation tool as a Network Automation Controller.

The investigation started with the State-of-Art in networking, followed by a conceptual analysis of DevOps and its relationship with NetDevOps; and as final phase, a Proofof-Concept emulation allowed the researchers to conclude that automation and programmability in every stage of any kind of data communication infrastructures is a must to adequately accompany business evolution.

Modern DCs are experiencing multiple problems at infrastructure levels due to the high demand of users and large amount of traffic that flows through their networks. VXLAN solves many of them, but, on the other hand, VXLAN could be very difficult and cumbersome to implement because of the various protocols involved as its underlay technologies (OSPF, VLAN-VXLAN mapping, among others).

To improve VXLAN implementations at early stages, NetDevOps CI-CD pipeline would increase its feasibility, and effectiveness, limiting the points of failure in configuration phases, which could increase greatly as the size of the network grows. Also troubleshooting in networking will be much easier to conduct.

It is worth to mention that programmability is not only limited to large infrastructures; it can be used regardless of the type of network to be configured, moreover when the execution times are very short compared to the time that a configuration from scratch could take in each device.

8.Acknowledgment

This research was partially supported by The Ecuadorian Institution of Higher Education, Science, Technology, and Innovation (SENESCYT in Spanish), as the Institution which collaborate in the Doctoral studies of one of the authors. We thank Cumulus Linux staff, which provided insight and expertise that assisted the proof-of-concept emulations.

References

References

Preston, H., and Sullivan, K. (2017). "Intent Networks: How to be a Network Engineer in a Programmable Age". Cisco DevNet Webinar Series, 2017.

Salazar Ch., G. D., Naranjo, E. F., & Marrone, L. (2018). SDN-Ready WAN networks: Segment Routing in MPLS-Based Environments. In 2018 9th IEEE Annual Ubiquitous Computing, Electronics & Mobile Communication Conference (UEMCON) (pp. 173-178). IEEE.

Andrade-Salinas, G., Salazar-Chacon, G., & Vintimilla, L. M. (2019). Integration of IoT Equipment as Transactional Endorsing Peers over a Hyperledger-Fabric Blockchain Network: Feasibility Study. In International Conference on Applied Technologies (pp. 95-109). Springer, Cham.

Shah, J., and Dubaria, D. (2019) "DevOps for Networking". The Linux Foundation. ONS Open Networking, San Jose, CA, 2019.

Naranjo, E. F. and Salazar Ch., G. D. (2017). Underlay and overlay networks: The approach to solve addressing and segmentation problems in the new networking era: VXLAN encapsulation with Cisco and open source networks. In 2017 IEEE Second Ecuador Technical Chapters Meeting (ETCM) (pp. 1-6).

Salazar Ch., G. D., Venegas, C., Baca, M., Rodríguez, I., & Marrone, L. (2018). Open Middleware proposal for IoT focused on Industry 4.0. In 2018 IEEE 2nd Colombian Conference on Robotics and Automation (CCRA) (pp. 1-6). IEEE.

Benson, J., Prevost, J. and Rad, P. (2016). "Survey of automated software deployment for computational and engineering research". In proc 2016. Annual IEEE Systems Conference (SysCon). Orlando, FL. USA.

Cavanaugh, S., and Lumbis, P. (2018) "Ansible 101 on Cumulus Linux", Oct. 25, 2018. [Webinar] Available: https://www.ansible.com/hubfs/2018_Content/Ansible%20 101%20on%20Cumulus%20Linux%20Webinar%20Slides.pdf.

Cumulus Docs. "LNV Full Example". Accessed July 2019. [Online]. Available: https://docs.cumulusnetworks.com/cumulus-linux/Network-Virtualization/ Lightweight-Network-Virtualization-Overview/LNV-Full-Example/

RedHat Ansible Whitepaper. "Continuous Integration and Delivery with Ansible".

Singh, N., Thakur, S., Chaurasiya, H, and Nagdev, H. (2015). "Automated provisioning of application in IaaS cloud using Ansible configuration management". In proc 2015. 1st International Conference on Next Generation Computing Technologies (NGCT). Dehradun, India.

Salazar Ch., G. D., Venegas, C., & Marrone, L. (2019). MQTT-Based Prototype Rover with Vision-As-A-Service (VAAS) in an IoT Dual-Stack Scenario. In 2019 Sixth International Conference on eDemocracy & eGovernment (ICEDEG) (pp. 344-349). IEEE.

Yiran, W., Tongyang, Z., and Yidong, G. (2018). "Design and Implementation of continuous integration scheme based on Jenkins and Ansible" In proc 2018. International Conference on Artificial Intelligence and Big Data (ICAIBD).

Masek, P., Stusek, M., Krejci, J., Zeman, K., Pokorny, J., and Kudlacek, M. (2018). "Unleashing full potencial of Ansible Framework: University Labs Administration". In proc 2018. 20th Conference of Open Innovation Association (FRUCT). Jyvaskyla, Finland.

Jansen, D. and Krattiger, L. (2017) "Building Data Centers with VXLAN BGP EVPN: A Cisco NX-OS Perspective", 1st ed. San Jose, California: Ed. Ciscopress, April 2017.