AccuWeather API Python Code

Here is a quick and dirty snippet of Python code to get you started with the AccuWeather API. Note that it offers no fault tolerance, optimization, or other things you might expect to find in finished software and only extracts a small subset of the data available in the JSON data.

import json
import time
import urllib.request

API = YOUR_API
LOCATION_ID = LOCATION



def get_weather(api, location_id):
    url = 'http://dataservice.accuweather.com/currentconditions/v1/%s?apikey=%s&details=true' % (location_id, api)
    print(url)
    with urllib.request.urlopen(url) as url:
        data = json.loads(url.read().decode())
    print(data)
    return (data[0]['Temperature']['Imperial']['Value'],
            data[0]['RelativeHumidity'],
            data[0]['Wind']['Direction']['Degrees'],
            data[0]['Wind']['Speed']['Imperial']['Value'],
            data[0]['UVIndex'],
            data[0]['CloudCover'],
            data[0]['Pressure']['Metric']['Value'],
            data[0]['Precip1hr']['Metric']['Value'],
            data)


timestamp = time.time()

temperature, humidity, wind_bearing, wind_speed, uv_index, cloud_cover, pressure, precipitation, raw \
    = get_weather(API, LOCATION_ID)

 

Advertisements

AccuWeather API

First up in my series on weather APIs is the one from AccuWeather.

There are a few really great things about this API–or rather this set of APIs–but first let me tell you about the downsides. Accuweather doesn’t use a standardized system for locations so, as far as I can tell, there isn’t a way to simply input Lat/Lon coordinates. Instead your first stop on any journey has to be with their location finding API. Not only is this counter-intuitive, but since they only provide 40 free hits to their API a day, having to make regular checks for different locations could quickly eat up your allotment. Furthermore since forecast, current conditions, and indexes are all separate APIs, you are likely to be making a lot more calls than you would with a service like Darksky. I don’t mean to complain–after all they do offer access for free and 40 API calls is more than enough for the average hobbyist–but it seems stingy in comparison to most other services.

On the other hand, the API provides a superb data set that is easy to navigate and is clearly described in their documentation. Just take a look at their current conditions API which is well documented and includes all sorts of numbers that I didn’t even know were things–honestly did you know what “wet bulb temperature” was before reading that page? Similarly the forecast API offers things like pollen and air pollution information which must APIs lack.

Importantly AccuWeather does not suffer from the “disappearing metric” problem which plagues many APIs–more on this later.

The AccuWeather API may be right for you if your needs are limited to a single location and only a few updates a day. But if your location needs are more complex or want minute-by-minute updates. Also I think that AccuWeather only outputs to JSON, so if you are an XML head you are out of luck. For my particular use–graphing weather data for general interest and to compare to internal sensor data–AccuWeather works well and may come the closest to being an all-in-one solution. I just really wish that they supported more options for location input.

In my next post, I will share some example Python code which will show you how easy it is to get up and running with this API.

 

Link Wednesday (9.20.17)

DIY Git Repository Server
I’ve used local git repositories to keep track of all my various python projects, but until recently I did not use any remotes. However, lately I’ve realized the advantage of increasing separation between my local/experimental branch and master. It really only took me 15 minutes or so to set up a git server on one of my Pis.  In doing so I looked at several tutorials but this was by FAR the best and most comprehensive.

Crontab Guru
Crontab isn’t hard to use. Or at least it shouldn’t be. But I find that every time I set up a new task I am wracked with doubts about the exact time it will be triggered. Why? I don’t know…the syntax is clear and I’ve used it for years. To make me feel better, however, some kind person made crontab.guru a website which will explain (in entirely unambiguous terms) what a particular crontab string  (i.e. */5 * * * *)  will do. It may be a silly safety blanket, but it has made me far happier while crontabbing.

 

Evaluating Weather APIs

Many Internet-of-Things applications require knowledge of various environmental factors such as temperature, humidity, and light levels. While various sensors exist that fill this need locally–such as the ubiquitous DHT11/22 temperature and humidity sensor–sometimes an application requires broader weather data or just a sense of whether or not it is raining outside. Other applications might benefit from weather forecasts.

Thankfully the Internet provides us hobbyists and makers with a ton of APIs that provide weather data. Honestly the sheer volume of the APIs can be overwhelming. For a project that I’ve been playing around with–an indoor environmental monitoring system–I wanted some external data for comparison and worked with quite a few different APIs before I found the mix that worked for me. In the next few posts I’ll share my experiences and some sample code for working with each one.

But first here is a look at the contenders I will be evaluating

As a preview I’ve found that each has their pluses and minuses. Some provide lots of information, but have quirks in their API that make them annoying to use. Others lack data that seems essential. Ultimately–as I have cycles and bandwidth to spare–I combined all four in my final solution.

Link Wednesday (9.13.17)

I’ve been keeping an archive of useful how-to articles and have decided to share them as a more-or-less weekly feature… Not to mention as an easy way of archiving them for my own future use.

Americanize Raspberry Pi
http://rohankapoor.com/2012/04/americanizing-the-raspberry-pi/

I don’t know why it took me literally years of working with Raspbian before I realized that the default configuration had several quirks from across the pond. Running through the steps in this article fully “Americanizes” your Pi–i.e. it sets the locales and download locations correctly. Now it is part of my basic New Pi Procedure.

Change Network Interface Name
http://ask.xmodulo.com/change-network-interface-names-permanently-linux.html

Has anyone else noticed that new systems tend to give their network interfaces weird names (like eno63214A… that sort of thing)? If anyone knows why, I’d love to find out but until then I will just vaguely blame it on IPv6. Not to resist the future, but the new style of name makes using wildcards with system monitors like Grafana highly annoying. Plus if eth0 was a good enough for my father and a generation of other Linux nerds, it’s good enough for me, dag nabbit! Unfortunately I could not figure out how to change the interface on my own. Thankfully Google came to the rescue!

Raspberry Pi Undervolt Warning
https://www.raspberrypi.org/forums/viewtopic.php?f=29&t=82373&start=75#p739517

After a long search I discovered that there is really no way to use software to find a Raspberry Pi’s current input voltage or power usage without adding additional hardware. However, there is a tricky way to make sure that your Pi isn’t “browning out” by monitoring the LED that indicates a low-power state. It’s a super clever hack and one that I plan on integrating with my larger monitoring system at some point. (Note this may only work with RPI1 & 2.)

What people don’t get about MQTT

I’ve been spending a lot of time reading and responding to MQTT questions on Stackoverflow and I’m struck that the same basic misunderstandings keep coming up. I’m no expert on MQTT certainly (there’s a guy who posts under the handle hardlib who knows far more than me) but I do have experience in education–namely in breaking subjects down in a way that people without a broad knowledge base can understand–so I’m hoping that I can correct a few misconceptions with this post.

MQTT is a protocol, not a program

This seems to be the most fundamental misunderstanding. Many questions ask things like “how do I reply to MQTT messages” or “how can I edit MQTT messages on a server?” There are potentially meaningful answers to these questions but they all miss the fact that MQTT is simply a well defined and useful protocol. Upper-level considerations such as how an application responds to a message, how messages are retained on servers, and so on and so forth are all handled by the broker, publisher, and client–not by the MQTT specification itself. Therefore if you want to meaningfully ask a question about MQTT you must (generally) at least specify the broker you are using if not explain the entire ecosystem that you are building or using.

Indeed, if you have anything beyond a passing interest in MQTT you really should read the standard  which is superbly written and quite clear.

MQTT is not an instant messaging platform

Unfortunately Facebook’s use of MQTT seems to have suggested to every developer that MQTT is a person-to-person messaging standard. But it isn’t! MQTT is a machine-to-machine standard. This is an important distinction because MQTT was never designed to supply the kinds of things a messaging platform requires (message history, the ability to retract messages etc.). Now, as Facebook proves, you can use MQTT as the underlying communications protocol for a messaging platform or application but you will have to program all of the additional facilities that you require that go beyond the standard and a few niceties added by specific brokers. So asking about how to build an instant messaging application with MQTT is a bit like asking how to build a website that uses HTTP: it is a meaningful, but absurdly broad question.

MQTT topics do not exist without content

Probably the most common misunderstanding I see in questions revolves around the idea that MQTT topics exist without content. It is an easy mistake to make given that MQTT hierarchies look a lot like file paths or other familiar things. But MQTT just doesn’t work like that. Topics in MQTT are not indicators of a place or anything like that, they are simply routing instructions that the broker uses to move payloads from publishers to subscribers.

Here is an example that might clarify this a bit. Say you have a directory called /spam/eggs/ in which there is a file called payload. The full path of that file would be /spam/eggs/payload. In this case the directory /spam and /spam/eggs literally exist on your hard drive. However in the case of MQTT the equivalent statement /spam/eggs/payload simply tells the broker to send the message to all subscribers subscribed to #, / #, /spam/#, /spam/eggs/#, or /spam/eggs/payload. There are a few others that I left out for brevity, but you get the point. This distinction may not seem important, but I’ve seen it at the root of many false understandings of how MQTT works.

MQTT topics should not start with a /

To make the point above, I intentionally made a subtle mistake. Did you catch it? Adding the / to the front of the topic created an entirely unnecessary topical level. This is more of a pet peeve than a real problem, but I see people doing this constantly. The addition of this first topic level is not only unnecessary but, if you aren’t aware of it, it can lead to real confusion.

I hope this has been of use, and I’m more than willing to answer any questions or make any corrections necessary to these points. But while I’m writing, do you have anything that drives you crazy about how people understand MQTT?

 

 

 

MQTT Cookbook: Round-trip Time

Most MQTT brokers provide statistics on what they are doing (number of messages sent etc.) but do not provide meaningful information about how well the broker and client are performing. To gather that sort of information you need to consider the entire chain between the client and the broker. Although there may be several different ways to approach this, the simplest is to measure the round-trip time or RTT. In the case of MQTT that means measuring how long it takes from the publication of a message to its distribution to subscribers.

Thankfully the loose coupling between publishers and subscribers in MQTT allows the same client to be both. This allows us to measure the entire chain in a few lines of code.

import paho.mqtt.client as mqtt
from time import time, sleep
import uuid

INTERVAL = 1
QOS = 0


def on_connect(client, userdata, flags, rc):
    client.subscribe(topic)
    client.publish(topic, time(), qos=QOS)


def on_message(client, userdata, message):
    msg = message.payload.decode('utf-8')
    rtt = time() - float(msg)
    rtt_array.append(rtt)
    rtt_max = max(rtt_array)
    rtt_average = sum(rtt_array) / len(rtt_array)
    rtt_min = min(rtt_array)
    print('Current: %s' % rtt)
    print('Maximum: %s' % rtt_max)
    print('Average: %s' % rtt_average)
    print('Minimum: %s' % rtt_min)
    sleep(INTERVAL)
    client.publish(topic, time(), qos=QOS)


def on_log(client, userdata, level, buf):
    print(level, buf)


rtt_array = []
topic = str(uuid.uuid4())
client = mqtt.Client()
client.on_connect = on_connect
client.on_message = on_message
client.on_log = on_log
client.connect("test.mosquitto.org")
client.loop_forever()

 

The script might seem a bit odd unless you understand how the call backs are structured in MQTT. The program flow goes something like this:

  1. The client connects to the broker triggering on_connect
  2. The client subscribes to the randomly determined topic
  3. The client publishes to the randomly determined topic
  4. The reception of the message that the client published triggers the on_message callback.
  5. The payload of the message is the time the message was sent which is then compared to the current time to get the RTT
  6. The RTT is added to the list rtt_array and used to calculate average, maximum, and minimum.
  7. After sleeping for INTERVAL seconds, a new message is published to the the topic returning the script to step 3.

As this script runs indefinitely you will have to break it manually by hitting CTRL+C. Also, the preset interval is pretty short (1 second), you will likely want to extend that to 10 or 60 if you are running this for more than a short burst.

Right now the script is configured to connect to the test.mosquitto.org server but it can easily be adapted to connect to your local broker.

Another interesting use of this script is that it makes it simple to test the performance consequences of using higher QoS settings. I notice around a 50% performance hit at QoS 2.

That’s pretty much it. If anyone has suggestions about how to improve it, tell me about them in the comments.