Network Topology Visualizer: OpenWISP

July 2017 Update: you can now use the automatic openwisp2 installer to deploy the network topology visualizer.

Recently I've been working on a network topology visualizer based on NetJSON: OpenWISP Network Topology.

Just released django-netjsongraph 0.1 https://t.co/dO4EwSBJoN, see a demo at https://t.co/F0huVgZvUU #mesh #python pic.twitter.com/vBjNzeLfTi

— Federico Capoano (@nemesisdesign) December 23, 2015

There are several motivations that led me to write this django app:

• I liked the idea of being able to visualize and monitor a mesh network by costantly analyzing its topology
• I needed this kind of functionality for Ninux.org, an wireless community network that aims to build free networks in the Italian peninsula
• I wanted to show the potential of a standard data interchange format like NetJSON, which makes it easy to visualize and monitor a network topology

Initially I was developing this feature in a larger web app, but then I understood it would have been better to publish it in a separate package in order to make it more reusable and maintainable.

The app is subdivided into several components:

1. Network Topology Parser
2. Network Topology Collector
3. Network Topology Visualizer
4. HTTP API
5. Admin Interface

Let's dig into the details about how these components are implemented

1. Network Topology Parser

Netdiff is an experimental #python library that provides utilities for parsing topologies of #mesh networks: http://t.co/koOEGW9uH1

— ninux.org (@ninuxorg) May 4, 2015

Most of the logic of this component is in reality implemented as a dedicated python library called netdiff.

Netdiff in charge of parsing different network topology formats and detecting changes over time (new links, changes in metrics, offline links).

Each format is implemented as a parser and the current parsers support the following formats:

• NetJSON NetworkGraph
• OLSR (jsoninfo/txtinfo)
• BATMAN-advanced (jsondoc/txtinfo)
• BMX6 (q6m)
• CNML 1.0

Adding support for a new format is just a matter of writing a new parser, add it to the python path and include it in the NETJSONGRAPH_NETDIFF_PARSERS setting.

You might be wondering, where is the OLSRd2 parser?
OLSRd2 ships a netjsoninfo plugin that returns NetJSON output natively, so no special parsers are needed for it, nice isn't it?

2. Network Topology Collector

This component is made of traditional django models (as in MVC) and django management commands which take care of using the parser component to detect the current state of the network topology and save it to one of the supported databases (django supports several relational databases).

It is made in a way that should facilitate including django-netjsongraph in larger projects and extend it. At the moment of this writing I haven't tested such a setup yet but it is my intention to do it as soon as possible because is something I really need.

The system allows to define multiple networks and collect topology data for each of them.

FETCH Strategy

The classic way to collect topology data is to fetch topology data from a URL, I called this "FETCH strategy".

This strategy is the simplest to implement but requires more effort from users because they have to publish the topology data on URL which is reachable by the collector. This involves writing scripts that fetch topology data and send it to a server (via SSH) which has a public ip address and a webserver installed.

Many users have requested the possibility to push topology data directly from nodes to the collector in a POST HTTP request.

To accomodate this need I implemented the RECEIVE strategy.

RECEIVE Strategy

Playing django-netjsongraph 0.2, send topology data from multiple nodes! https://t.co/2kD3zeAlF4 #netjson #python pic.twitter.com/AKefFF6qAE

— Federico Capoano (@nemesisdesign) January 24, 2016

This strategy allows nodes to send network topology data in the payload of a POST HTTP request.

This way a simple shell script like the following one is sufficient to start collecting topology data:

    #!/bin/sh
    BASE_URL="https://network-topology-visualizer.mynetwork.org"
    UUID="4e38397a-4254-4521-89a6-045db25b8de6"
    KEY="S9wCfjeFN8irasZNiV7dGcyK68BTES0m"

    COLLECTOR_URL="$BASE_URL/api/receive/$UUID/?key=$KEY"
    DATA=$(echo "/netjsoninfo filter graph ipv4_0" | nc 127.0.0.1 2009)
    curl -s -X POST -d "$DATA" --header "Content-Type: text/plain" $COLLECTOR_URL

The preceding script queries the olsrd2 netjsoninfo plugin, retrieves the NetJSON NetworkGraph and POSTS it to the collector.

As you can see there is a simple authentication mechanism which requires each POST request to include a "key" parameter which must match the topology's key. Each topology object added to the system will have a uuid and a key.

The cool thing about the RECEIVE strategy is that more nodes can send their topology to the collector, this allows ot mitigate the "Single Point of Failure" problem in which the URL that needs to be fetched is not reachable for some reason.

This feature also allows to monitor and visualize the network when it's broken in several smaller parts, like when important links fail (provided that some nodes are connected to the internet with private gateway, if they share the same gateway they won't be able to reach the collector if central links fail).

3. Network Topology Visualizer

This component is what allows casual users to see the network and inspect its nodes.

The engine of this component is implemented in a javascript library called netjsongraph.js.

netjsongraph.js is based on the popular d3.js javascript visualization library and takes care of reading the json graph format (NetJSON NetworkGraph) and generate a force-directed graph.

netjsongraph JS is a new #javascript library based on #d3 that displays a #NetJSON network topology: http://t.co/bREHWHhGNz

— ninux.org (@ninuxorg) August 18, 2015

4. HTTP API

This component is based on Django REST Framework, its main purpose is to provide HTTP resources for retrieving topology data in NetJSON format.

This API also allow nodes to POST topology data when using the "RECEIVE strategy".

5. Admin

This is the web interface (based on the django-admin) that allows administrators to add new topologies to the system.

Next steps

The next phase of development of this project will involve adding more topology formats, integrate the app in larger projects and prepare an easy-to-install package with meaningful deafult settings.

Another use case I would like to test is the "generic topology collector" concept proposed at the Battlemesh v8 by Axel Neumann (core developer of BMX routing protocol): if each node of a distance vector routing protocol could return it's known topology in NetJSON, django-netjsongraph could work as a generic topology collector for distance vector routing protocols.

This should already work, but I'm sure that collecting topology data from 2-3 nodes is quite different than doing it for a few hundred nodes and that is exactly what I would really like to test.

August 2017 Update: I had to close comments because of spam. To ask questions, please use one of the OpenWISP Support Channels.