Monitoring Freifunk Nodes With Prometheus

Update (2020-03-07)

We now collect the number of connected clients as well as whether the node is online.

We recently installed a Freifunk node from Freifunk Dreiländereck (FF3L) in our hackerspace. While changing network configuration during the testing phase, the node went offline without us noticing. Since we're using Prometheus for monitoring our space's infrastructure, I went ahead and hacked together a solution which I want to present here:

Idea

Instead of monitoring the node directly, I decided to tap into the status information already collected by the Freifunk community. They are publishing some interesting statistics, however we only really cared about one information: Whether FF3L currently considers the node to be online and reachable.

FF3L publishes these status information at the following endpoint:

https://map.freifunk-3laendereck.net/data/nodes.json

Many (if not all?) Freifunk communities provide such an endpoint, though with some you may have to search for a while to find it.

Implementation

The nodes endpoint yields the information of all nodes at once. Unfortunately, I didn't find a way to reduce the request to specific nodes; if someone knows more about this, don't hesitate to tell me. (As far as I can tell there are multiple implementations of this endpoint, some of which appear to support filters.)

This is how a single node object from the API response looks like:

{
  "nodeinfo": {
    "software": {
      "firmware": {
        "base": "gluon-v2019.1",
        "release": "v2019.1.0+001"
      },
      ...
    },
    "network": {
      ...
    },
    "location": {
      "latitude": ...,
      "longitude": ...
    },
    "system": {
      "role": "node",
      "site_code": "ff3l",
      "domain_code": "3land"
    },
    "node_id": "...",
    "hostname": "...",
    ...
  },
  "flags": {
    "online": true
  },
  "statistics": {
    "uptime": 626614.08,
    "clients": 2,
    ...
  },
  "lastseen": "...",
  "firstseen": "..."
}

We were especially interest in the .flags.online and .statistics.clients fields; our implementation extracts nothing but these two field. The fields .nodeinfo.node_id and .nodeinfo.hostname are suitable for filtering for your own nodes.

Using a bit of "curl|jq magic", we can create a shell script for parsing the data and converting it into a format understood by Prometheus. We added the script as a Textfile Collector to an existing Prometheus Node Exporter instance. The output then looks like this:

# HELP freifunk_node_online 1 if the Freifunk node is online, 0 otherwise
# TYPE freifunk_node_online gauge
# HELP ff3l_node_clients Number of clients connected to the node
# TYPE ff3l_node_clients gauge
freifunk_node_online{node="<node0_id>",hostname="<node0_hostname>"} 1
freifunk_node_clients{node="<node0_id>",hostname="<node0_hostname>"} 2
freifunk_node_online{node="<node1_id>",hostname="<node1_hostname>"} 0
freifunk_node_clients{node="<node1_id>",hostname="<node1_hostname>"} 0

The resulting script can be found on Gitlab.