Evaluating Weather APIs

Many Internet-of-Things applications require knowledge of various environmental factors such as temperature, humidity, and light levels. While various sensors exist that fill this need locally–such as the ubiquitous DHT11/22 temperature and humidity sensor–sometimes an application requires broader weather data or just a sense of whether or not it is raining outside. Other applications might benefit from weather forecasts.

Thankfully the Internet provides us hobbyists and makers with a ton of APIs that provide weather data. Honestly the sheer volume of the APIs can be overwhelming. For a project that I’ve been playing around with–an indoor environmental monitoring system–I wanted some external data for comparison and worked with quite a few different APIs before I found the mix that worked for me. In the next few posts I’ll share my experiences and some sample code for working with each one.

But first here is a look at the contenders I will be evaluating

As a preview I’ve found that each has their pluses and minuses. Some provide lots of information, but have quirks in their API that make them annoying to use. Others lack data that seems essential. Ultimately–as I have cycles and bandwidth to spare–I combined all four in my final solution.

Advertisements

MQTT Cookbook: Round-trip Time

Most MQTT brokers provide statistics on what they are doing (number of messages sent etc.) but do not provide meaningful information about how well the broker and client are performing. To gather that sort of information you need to consider the entire chain between the client and the broker. Although there may be several different ways to approach this, the simplest is to measure the round-trip time or RTT. In the case of MQTT that means measuring how long it takes from the publication of a message to its distribution to subscribers.

Thankfully the loose coupling between publishers and subscribers in MQTT allows the same client to be both. This allows us to measure the entire chain in a few lines of code.

import paho.mqtt.client as mqtt
from time import time, sleep
import uuid

INTERVAL = 1
QOS = 0


def on_connect(client, userdata, flags, rc):
    client.subscribe(topic)
    client.publish(topic, time(), qos=QOS)


def on_message(client, userdata, message):
    msg = message.payload.decode('utf-8')
    rtt = time() - float(msg)
    rtt_array.append(rtt)
    rtt_max = max(rtt_array)
    rtt_average = sum(rtt_array) / len(rtt_array)
    rtt_min = min(rtt_array)
    print('Current: %s' % rtt)
    print('Maximum: %s' % rtt_max)
    print('Average: %s' % rtt_average)
    print('Minimum: %s' % rtt_min)
    sleep(INTERVAL)
    client.publish(topic, time(), qos=QOS)


def on_log(client, userdata, level, buf):
    print(level, buf)


rtt_array = []
topic = str(uuid.uuid4())
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.on_log = on_log
client.connect("test.mosquitto.org")
client.loop_forever()

 

The script might seem a bit odd unless you understand how the call backs are structured in MQTT. The program flow goes something like this:

  1. The client connects to the broker triggering on_connect
  2. The client subscribes to the randomly determined topic
  3. The client publishes to the randomly determined topic
  4. The reception of the message that the client published triggers the on_message callback.
  5. The payload of the message is the time the message was sent which is then compared to the current time to get the RTT
  6. The RTT is added to the list rtt_array and used to calculate average, maximum, and minimum.
  7. After sleeping for INTERVAL seconds, a new message is published to the the topic returning the script to step 3.

As this script runs indefinitely you will have to break it manually by hitting CTRL+C. Also, the preset interval is pretty short (1 second), you will likely want to extend that to 10 or 60 if you are running this for more than a short burst.

Right now the script is configured to connect to the test.mosquitto.org server but it can easily be adapted to connect to your local broker.

Another interesting use of this script is that it makes it simple to test the performance consequences of using higher QoS settings. I notice around a 50% performance hit at QoS 2.

That’s pretty much it. If anyone has suggestions about how to improve it, tell me about them in the comments.