jrh

 

A datacenter decommissioning last year gave me access to some old Netbotz (now APC) monitoring gear. I jumped at the chance to save it from the garbage, as I remembered it being a pretty slick way to get basic environmental data (temp, door switches, etc.) and it was a small physical footprint way to control a number of networked cameras at once.

My recollection was right, and while the management server appliance proved both annoying to work with and massive overkill for my home projects, I’ve had some luck scraping data out of its web UI.

I wrote a few python classes to make it reasonable to programmatically access the data generated by the cameras and sensors in my backyard chicken coop; you can see the end result at pachube.

The code is available here.  It’s driven by a small db schema that holds basic location info for the monitoring gear and metadata about sensors being watched (e.g. polling intervals and alert thresholds), and produces simple name/value pairs with sensor data satisfying given conditions.  By “conditions” here I mean things like “it’s time to report this sensor value” or “this value changed too much, so I’m reporting on it” — this is by no means meant to replace actual monitoring systems (though it could easily be used to interface with them).

While I wrote fairly thorough pydoc, the overall documentation is hardly extensive, so I’d be happy to answer questions if getting at netbotz data from python is interesting to anybody else.

 

Pachube provides an API-exposed store of sensor data from pretty much anything, but with a focus on sensor data.  It’s free to use and has what seems to be a pretty complete, reasonably designed, and easy to use API with a nicely flexible authorization model.

I’ve been working on some code to scrape data out of the Netbotz web UI (which does not have an easy to use API) and, as I got to working on the data storage backend for the scraper, remembered Pachube and decided to give it a try rather than reinventing what was effectively the same wheel.

I have generally positive impressions of Pachube after a couple of days messing around with it.  It was pretty easy to get going, and after stumbling into one quickly-fixed bug was up and sending data in from my prototype scraper pretty quickly.

I’m using the following python class to simplify the interaction.  It’s trivial, but I didn’t see anything similar already done, so am throwing it here for future google results:


#!/usr/bin/python

# This code is released into the public domain (though I'd be interested in seeing
#  improvements or extensions, if you're willing to share back).

import mechanize
import json
import time

class PachubeFeedUpdate:

  _url_base = "http://api.pachube.com/v2/feeds/"
  _feed_id = None
  _version = None
  ## the substance of our update - list of dictionaries with keys 'id' and 'current_value'
  _data = None
  ## the actual object we'll JSONify and send to the API endpoint
  _payload = None
  _opener = None

  def __init__(self, feed_id, apikey):
    self._version = "1.0.0"
    self._feed_id = feed_id
    self._opener = mechanize.build_opener()
    self._opener.addheaders = [('X-PachubeApiKey',apikey)]
    self._data = []
    self._payload = {}

  def addDatapoint(self,dp_id,dp_value):
    self._data.append({'id':dp_id, 'current_value':dp_value})

  def buildUpdate(self):
    self._payload['version'] = self._version
    self._payload['id'] = self._feed_id
    self._payload['datastreams'] = self._data

  def sendUpdate(self):
    url = self._url_base + self._feed_id + "?_method=put"
    try:
      self._opener.open(url,json.dumps(self._payload))
    except mechanize.HTTPError as e:
      print "An HTTP error occurred: %s " % e

Usage is pretty straightforward.  For example, assuming you have defined key (with a valid API key from Pachube) and feed (a few clicks in the browser once you’ve registered) it’s basically like:


pfu = PachubeFeedUpdate(feed,key)
# do some stuff; gather data, repeating as necessary for any number of datastreams
pfu.addDatapoint(<datastream_id>,<data_value>))
# finish up and submit the data
pfu.buildUpdate()
pfu.sendUpdate()

The resulting datapoints basically end up looking like they were logged at the time the sendUpdate() call is made.  In my situation, I want to send readings from a couple of dozen sensors each into their own Pachube datastream in one shot, so this works fine.  If, instead, for some reason you need to accumulate updates over time without posting them, you’d need to take a different approach.

 

My twitter stream — okay, “trickle” might be more accurate — has for the past few weeks mostly been about my continued work on our backyard chicken coop (more on that later).

While the coop isn’t quite done, earlier this week we took delivery of its future inhabitants.  As usual, @amyj has better pictures, but I’ve got webcams and sensors!  You can follow along with the growth and play of our baby chicks here.  They mostly just eat and poop, but they’re really cute while they’re doing it.  [Note: you can click around in the interface to see different views, sensor readings, etc.]

At the moment there are eight chicks, due to an order processing error (we only ordered four).  Hopefully we’ll find the extras a new home soon, and it’ll just be the four again.

 

Jacs, at 2611 Monroe St. on Madison’s near West side, “soft-opened” earlier this week in the wake of the recently shuttered Monroe St. Bistro.  We (me, AmyJ, and our 2 and 5 year-old kids) spontaneously stopped in for lunch today.  Jacs wasn’t perfect (what is after 5 days?), but we all agreed that it shows promise, and are happy to have it in the neighborhood.

While we’d intended to have lunch, Jacs serves only brunch between 10am and 3pm on weekends.  The brunch menu had some very tasty looking breakfasts (omelets, eggs benedict, various lighter breakfasts, etc.) and a handful of sandwiches.

I was happy to see the moules frites still on the menu, as they were the only thing on the MSB’s menu that I ever found pleasing without caveat (and the only area in which MSB came anywhere close to its generally superior neighbor up the street, Brasserie V).  I don’t know that they’re the same, but my understanding is that there’s some continuity in ownership and/or management between the two, so there’s reason for hope.
Continue reading »

 

I set out this weekend to get an Arduino board to control my Roomba.  (The Roomba has a great – and generally open – interface, and iRobot deserves significant credit for encouraging creative repurposing/extensions of their products.)  I’ve got a few project ideas in mind, but for an initial step just wanted to verify that the Arduino could a) send control commands (“move forward”, “turn right”, etc.) from the Arduino, and b) read sensor data (“something is touching my left bumper”, “I’m about to fall down the stairs”).  This post contains my notes, which hopefully will help others doing this sort through some of the issues in a bit less time than that I spent.  Continue reading »

 

I was struck by a comment the lovely amyj made in a recent blog post:

I remember when Ben used to say “Happy Day” instead of Happy Birthday. I don’t know when he stopped…you never know when they will stop doing those things…you always noticed the “firsts”, the “lasts” seem to slip by…. I wonder when the last time Nora will make her little snoring sound when she lays down to sleep or notices something sleeping or puts her babies or animals to sleep …

Indeed.  For years I’ve intended to more systematically capture the countless precious moments, hilarious tidbits, etc. that our kids produce on a daily basis.  It occurred to me that posterous.com, which I’d briefly toyed with over the holidays but not really found a real use for, would be perfect for this.

And so I now intend to start recording those moments — the things the kids say that make me smile or laugh, the fun things we do, et cetera.  At the risk of  overwhelming cheesiness, John Lennon put it well:   “life is what happens to you while you’re busy making other plans”.

Here it is.  Honestly, it will probably bore you with its detail.  I’m the intended audience, I’m just keeping it public.  So follow along if you’d like.

 

I haven’t posted here in forever – lately I’ve spent all of my non-working online time with things a bit lighter-weight than blogging.  Places I have a presence include:

If we know each other and you’re on the above, send me a friend/link invite so we can connect there as well.

I’ll still occasionally post here (probably about as often as I have the past few years), though by far most of my day-to-day activity is in one of the above.

 

… and it makes me feel like I have laryngitis.

I had a near miss at a failblog-worthy photo when I happened upon the following scene in an airport last week:

sevenhabitsfail.jpg

The title of the book isn’t really visible in the hastily-taken cell phone camera shot, but it’s The Seven Habits of Highly Effective People.  Seriously.

Twitter seems to be back now.

 

I’ve been playing with Arduino boards in my limited spare time over the past few months.  It’s a fun way to spend quality hands-on geek time that is clearly distinct (at least to me) from my day job.  Plus, I’m able to start actually instantiating some of the ubiquitious computing / distributed sensor ideas that have been floating around in my head.

I’ve been working on a simple wireless light, temp, and motion sensor.  Light was a trivial CDS photocell connected to the analog port of the arduino.  My first attempt at temp is using the Dallas Semiconductor DS-18B20 digital one-wire sensor, which is pretty slick for $4.25.

There was some good sample code on the main arduino site, but I spent a small bit of time to flesh it out more completely, adding the ability to configure sensor resolution and extracting the temp value from the returned data.  Code is here, if this is interesting or useful to you.

 

I was going through an old pile of paper in my office recently and encountered a set of note cards I’d accumulated years ago, back in the very small, scrappy, it-definitely-might-not-make-it startup stage. Most of the cards contained miscellaneous reminders, todos, or ideas I thought worthy of further exploration.

A few of the cards, though, had the record of an idiom mixing game my colleagues and I played back in the day (I’ve had the distinct pleasure of working with strongly multi-disciplinary and linguistically inclined geeks).

At its basic level, the game produced comprehensible phrases that amusingly combined two familiar idioms, such as “there are other fish to skin”, or “there are other cats in the sea”.

These are good for a chuckle, but not fundamentally anything more than language slapstick. Some combined idioms of similar intent in ways that made more vibrant images than did the originals, such as

“that opens up a whole new can of monkeys”

(A “can of worms” is one thing, but monkeys make everything funnier.) Rather than just “getting ducks in a row” or having things “fall in line”, we had

“all the ducks are falling in line”

Other are amusing but confusing, and almost seem as if they mean something, at least until you actually think about them. Example:

“Happier than a clam in pigsh*t”

The pinnacle of our mixed idiom game, though, were those hard to find combinations whose meanings were a novel blend of the original idioms. Most of these tended to mockingly riff on various elements of commonly accepted corporate-speak.

“I’m just putting them on the table as I see them”

for example, takes the casual (if sometimes cowardly) innocence of the defensive verbal communication standby “I’m just calling them as I see them” with the trite business-ese of “putting something on the table” to create an all-new description of an impetuous laziness thrust upon others.

Better still, in my opinion, is the lighthearted cynical foreshadowing of:

“We’ll burn that bridge when we come to it”

But my favorite, by far, is a sadly apt commentary on organizational politics gone awry:

“I dropped the ball in your court”

Have more? Oh yes you do … comment away!

© 2011 Joshua Heling Suffusion theme by Sayontan Sinha