Your first IoT service

Once you have a Sibylla platform, it is necessary to develop an edge gateway able to send to the platform the data detected by the sensors.

A really basic sensor system can be represented by the computer being used, whose resources (memory, disk, CPU) represent the physical quantities to be monitored. You can imagine the computer as a thing, which has three devices that continuously transmit data.

The transmission channel can be chosen between the HTTP and MQTT protocol, since the basic installation of the Sibylla platform provides for gateways capable of receiving data on both.

The last thing you need is an Edge Gateway able to retrieve the usage data of the various PC resources, package them in JSON format, and send them to the platform via one of the transmission channels. This role can be played by a unix bash script like this one

 


#!/bin/bash

# DEFAULT VALUES
INTERVAL=5
CHANNEL=http

# PARAMETERS READING
POSITIONAL=()
while [[ $# -gt 0 ]]
do
    key="$1"

    case $key in
        -c|--channel)
        CHANNEL="$2"
        shift # past argument
        shift # past value
        ;;
        *)    # unknown option
        POSITIONAL+=("$1") # save it in an array for later
        INTERVAL=$1
        shift # past argument
        ;;
    esac
done
set -- "${POSITIONAL[@]}" # restore positional parameters

host=$(hostname)

while true;
do

    ts=`date +%s%N | cut -b1-13`

    MEMORY=`free -m | awk 'NR==2{printf "\"memory\": {\"used\": %s, \"total\": %s, \"unit\": \"MB\", \"percentage\": %.2f}\n", $3,$2,$3*100/$2}'`
    MEMORY="{\"hostname\": \"$host\", \"timestamp\": $ts, $MEMORY}"
    DISK=`df -m /home | awk 'NR==2{printf "\"disk\": {\"used\": %d, \"total\": %d, \"unit\": \"MB\", \"percentage\": %.2f}\n", $3,$2,$3*100/$2}'`
    DISK="{\"hostname\": \"$host\", \"timestamp\": $ts, $DISK}"

    CPUs=""
    while IFS= read -r line; do
        IFS=' ' read -ra RES <<< "$line"
        let numerator=${RES[1]}+${RES[3]}
        let numerator=${numerator}*100
        let denominator=${RES[1]}+${RES[3]}+${RES[4]}
        [[ ${RES[0]} = 'cpu' ]] && core="total" || core="${RES[0]}"
        CPUs+="\"${core}\": `echo "scale=2; $numerator/$denominator" | bc`, "
    done < <( grep 'cpu' /proc/stat )
    CPUs=${CPUs::-2}

    CPU="{\"hostname\": \"$host\", \"timestamp\": $ts, \"cpu\": {${CPUs}}}"

    case $CHANNEL in
        mqtt) # MQTT
        printf "MQTT MEMORY: $MEMORY\n"
        mosquitto_pub -h localhost -p 1883 -t SIBYLLA/1/$host/MEMORY -m "$MEMORY"
        printf "MQTT DISK:   $DISK\n"
        mosquitto_pub -h localhost -p 1883 -t SIBYLLA/1/$host/DISK -m "$DISK"
        printf "MQTT CPU:    $CPU\n"
        mosquitto_pub -h localhost -p 1883 -t SIBYLLA/1/$host/CPU -m "$CPU"
        ;;
        *)    # default (HTTP)
        printf "\nHTTP MEMORY: $MEMORY\n"
        curl -w "%{http_code}" --silent -X POST  -H "Content-Type: application/json" -u "sibylla:sibylla" -d "$MEMORY" http://localhost:21100/sibylla/gateway/$host/MEMORY
        printf "\nHTTP DISK:   $DISK\n"
        curl -w "%{http_code}" --silent _X POST  -H "Content-Type: application/json" -u "sibylla:sibylla" -d "$DISK" http://localhost:21100/sibylla/gateway/$host/DISK
        printf "\nHTTP CPU:    $CPU\n"
        curl -w "%{http_code}" --silent -X POST  -H "Content-Type: application/json" -u "sibylla:sibylla" -d "$CPU" http://localhost:21100/sibylla/gateway/$host/CPU
        ;;
    esac

    sleep ${INTERVAL:-5}
done

Copy the script above and past it into a new file called for convenience first-sibylla-monitoring.sh.

nano first-sibylla-monitoring.sh

Configure the correct id and ports in the curl and mosquitto_pub operations, unless the default ones currently indicated are good, and save the script.
Set the script as executable

chmod +x first-sibylla-monitoring.sh

Now you are ready to run the first IoT service: monitoring the usage of your computer resources. As already mentioned, there are two ways to send data to the Sibylla platform.

HTTP data transmission

The first time you transmit data via HTTP protocol, you have to set an user for the HTTP gateway, in order to implement secure data transmission.This is done executing the following operations

cd $SIBYLLA_HOME/sibylla-network-1/sibylla-gateway-http
java -jar sibylla-gateway-http-1.0.0-RELEASE-*.jar --useradd sibylla

When requested the user password, insert sibylla. You can choose different user and password, not necessarily sibylla:sibylla, but selecting this pair of credentials you completely match the couple in the curl operation written in the script above. If you want prefer customize them, you have to put them into the script.

With the platform already running and ready to receive data you can open a bash console, locate to the position where the first-sibylla-monitoring.sh is saved, and type the command

./first-sibylla-monitoring.sh 3 -c http

and hit Return. The script starts endlessly to send data to the platform, and to stop it, all you have to do is interrupt it with a Ctrl-C command.
The number following the script name (3) is the time in seconds between sending and sending (default to 5). The -c parameter (equivalent to --channel) is used to specify the transmission channel protocol (default to http).
The output of this command is the following:

HTTP MEMORY: {"hostname": "YOUR-HOST", "timestamp": 1559917147276, "memory":{"used": 6333, "total": 11925, "unit": "MB", "percentage": 53.11}}
201
HTTP DISK:   {"hostname": "YOUR-HOST", "timestamp": 1559917147276, "disk": {"used": 72165, "total": 140677, "unit": "MB", "percentage": 51.30}}
000201
HTTP CPU:    {"hostname": "YOUR-HOST", "timestamp": 1559917147276, "cpu": {"total": 59.12, "cpu0": 35.75, "cpu1": 75.71, "cpu2": 75.33, "cpu3": 75.02}}
201
HTTP MEMORY: {"hostname": "YOUR-HOST", "timestamp": 1559917151058, "memory":{"used": 5303, "total": 11925, "unit": "MB", "percentage": 44.47}}
202
HTTP DISK:   {"hostname": "YOUR-HOST", "timestamp": 1559917151058, "disk": {"used": 72164, "total": 140677, "unit": "MB", "percentage": 51.30}}
202
HTTP CPU:    {"hostname": "YOUR-HOST", "timestamp": 1559917151058, "cpu": {"total": 59.12, "cpu0": 35.75, "cpu1": 75.71, "cpu2": 75.33, "cpu3": 75.02}}
202
...

As you can see, the first execution loop gives all 201 (CREATED), the other 202 (ACCEPTED). This is because the first time a message is sent on a network for a new device, this is created, due to the autoprovisioning property set as default on the platform. From the second run onwards, the deviced are already been created, and you have a simple new data storage for them.

MQTT data transmission

Specifying the -c mqtt parameter makes our Edge Gateway to use the MQTT protocol

./first-sibylla-monitoring.sh 3 -c mqtt

Every three seconds trhee distinct data packet are sent, one for memory, one for disk and one for CPU info:

MQTT MEMORY: {"hostname": "YOUR-HOST", "timestamp": 1559915171355, "memory":{"used": 4693, "total": 11925, "unit": "MB", "percentage": 39.35}}
MQTT DISK:   {"hostname": "YOUR-HOST", "timestamp": 1559915171355, "disk": {"used": 72145, "total": 140677, "unit": "MB", "percentage": 51.28}}
MQTT CPU:    {"hostname": "YOUR-HOST", "timestamp": 1559915171355, "cpu": {"total": 59.10, "cpu0": 35.39, "cpu1": 76.13, "cpu2": 75.81, "cpu3": 75.56}}
...

Verifying data reception

All that remains is to verify that the data sent through the script have actually reached the platform.
Let’s open the Sibylla Console, reach the Networks page and select the YOUR-HOST network

As you can see, a new thing, YOUR-HOST, has been created, and linked to it there are now three devices: CPU, DISK and MEMORY. if you want to inspect if data is arriving for that devices, you can click on one of them (say MEMORY) and see its dedicated console page


If the platform web-socket gateway is running the incoming data is visible on the Real time chart. Otherwise you can perform a query specifying the last minutes inside the Start time and End time fields.