User:WillWare/Google Wave Robot

Clipper, a Google Wave gadget for remote clients -- originally envisioned as a Robot, hence the misleading page title

I'm in the Boston area so I'm hoping to have this ready for the Cambridge Hackathon on November 21st. This project is now hosted on Google Code.

Overview
Suppose your burglar alarm or lab automation system runs on some random PC somewhere, and your ISP blocks port 80 or doesn't offer a static IP address. It can't be a server, therefore it can't be a Google Wave robot, so how do you allow it to participate in a Wave? Clipper is a Google Gadget that permits secure access to those remote clients within a Wave or elsewhere on the web. Example remote clients include:
 * Smart-home systems that report home security status or accept commands for X10-controlled appliances ("Clipper, I'm coming home, turn on the driveway light")
 * Laboratory automation systems that are controlling and monitoring experiments running overnight or over the weekend
 * Software build systems running overnight
 * Test automation systems that run a roomful of embedded devices through a test suite
 * Shell access to some remote Linux box somewhere

Architecture
Clipper is made up of two pieces.
 * A Google Gadget, which can safely and securely reside on any web page or on a Wave.
 * The Reflector is a server that stores information going back and forth between the Gadget and the remote client. My current plan is to build the Reflector as a web app on App Engine but its protocol is simple and doesn't require this.

The gadget and the toaster are both clients, the reflector is a server. The toaster makes HTTP POSTs including status to the reflector. Status takes the form of an HTML snippet. The reflector is an App Engine web app.

The reflector hangs onto any pending command and responds with it. The gadget is also a client of the reflector, it either sends an HTTP POST with a command or an empty HTTP POST, and the reflector always sends back status, the HTML snippet, to be stuffed into the DIV in the gadget.

User prefs for the gadget:

 * the URL for the reflector, or this might be hard-wired
 * the name of the toaster (remote client)
 * width and height
 * ping frequency

Security
The gadget has two text windows, "command" and "password". The password text box hides the characters typed as asterisks. The toaster periodically sends a long random string (a salt) which goes through the reflector to the gadget. When the gadget wants to send a command, it concatenates the long random string, the command and the password, and generates a hash. The gadget sends back the command and the hash.

The toaster, knowing the password, can verify the hash. On every received command (and once in a while just for yucks) the toaster generates a new salt.

This should be safe for MITM attacks because each command gets a different salt. This security model should enable you to safely use the gadget anywhere on the web. The toaster might want to log received commands with timestamps.

Status can be any random HTML snippet of any length including a little tabbed display, so one tab could be a log of the commands over the last 24 hours.

If the gadget sends an empty command, no password is needed and no hash is done, it just gets the status from the reflector. We'll assume that the status is not a secret. If it's secret, then a bigger security model will be needed.

Interaction between the Reflector and the remote client
App Engine provides HTTPS support. It's a one-line change in app.yaml. Remote clients must always use HTTPS in communications with the Reflector.

Remote clients may want to upload binaries (images, videos, sound samples) to the Reflector, which will need a way to handle these and make them available with a "/assets/*" URL. Status reports are sent to the Reflector as JSON:

{ "client": "wills-house", "password": "044BC516AD8F....",   // used to authenticate the remote client "html": "Example status", "assets": [ { "type": "jpeg", "name", "okay-logo.jpg", "data": "0001020304FEDBCA3819....." },    ... other assets ... ],  "salt": "13252BCA9D26....." }

Because a Reflector may service multiple clients, it's important that the clients have unique names and that those names be used as subdirectories for the "/assets" directory. This prevents name collisions between the stuff two different clients may want to store as assets.

The policy for cleaning up the assets directory tree is that files are deleted if they are more than a few hours old.

On App Engine, those assets can't just be stored in the "/assets" directory of the app itself. There is no guarantee that the event handler will be running on the same machine in the cloud where that directory lives. It will be necessary to store those assets in the Datastore, which does support blobs, so that should be OK. So an asset will look like this:

class Asset(db.Model): type = db.StringProperty clientname = db.StringProperty data = db.BlobProperty

Responses coming down from the Reflector also use JSON. If there is no command, it's just an empty dictionary. If there is a command, then the response looks like this.

{ "command": "Turn on driveway light", "hash": "FEDBCA38190001020304.....", "salt": "13252BCA9D26....." }

In addition to verifying the hash, the remote client should confirm that the salt is one of the salts it has sent recently. When it receives a new command, it should generate a new salt. Because of latency through the Reflector, the client should be willing to accept any salt generated in the last half hour.

Interaction between the Reflector and the Gadget
As long as we're using JSON and mutual authentication between the Reflector and the remote client, we should do likewise between the Reflector and the Gadget.

Registering a remote client with the Reflector
A single Reflector should be able to handle multiple remote clients. There should be a GWT app that lets you register and deregister remote clients. You'd need to supply
 * The remote client's name
 * Its IP address or some other piece of identification so that the Reflector can tell a message is coming from that client and not some other client.

Gadget info
Read the bit about inserting a Gadget by URL into a Google Wave. Other info:
 * http://code.google.com/apis/gadgets/docs/gs.html
 * http://code.google.com/apis/wave/extensions/gadgets/guide.html

Criteria for successful completion

 * Ready to deploy on App Engine.
 * Tested with the actual Google Wave service -- not possible until Nov 21st.
 * Passes a test suite
 * Two or more remote clients (simulated by HTTP client bots)
 * Status reporting of clients
 * Sending commands to clients and confirming their responses
 * Handle delegation and removal of delegation
 * Handle registration and deregistration of a client

Wave info
The Complete Guide to Wave is a brilliant resource.

The Wave Robot API

 * http://code.google.com/apis/wave/extensions/robots/
 * A Subversion repo on Google Code

App Engine

 * Wikipedia article
 * Java: http://code.google.com/appengine/docs/java/gettingstarted/
 * Python: http://code.google.com/appengine/docs/python/gettingstarted/

Download google_appengine_1.2.7.zip and unpack it. Type python dev_appserver.py demos/guestbook then in a web browser, visit http://localhost:8080 and try out the Guestbook app. So far so good, but the really delightful part is to look at the guestbook source code. It's very simple, with just two very small files: demos/guestbook/app.yaml demos/guestbook/guestbook.py The app supports two kinds of interaction, an HTTP GET at to "/" and an HTTP POST to "/sign". The app.yaml file is responsible for routing HTTP requests to the right source file; in this instance all are handled by guestbook.py.

Let's look at the pieces of guestbook.py. Very simple and logical. In hindsight, I can see that I'd have grokked GWT faster if I'd studied App Engine first.
 * The Greeting class shows that App Engine provides database capabilities, no surprise there. It uses GQL instead of SQL. The fields in the greeting class are like SQL's "CREATE TABLE" statement, where you define the columns in each row.
 * The MainPage class handles HTTP GET requests to "/". Note the dynamically generated HTML form which is going to send an HTTP POST request to "/sign".
 * The Guestbook class handles HTTP POST requests to "/sign".
 * Next there is a definition of a WSGI app that maps those destinations to those handler classes, and a main wrapper.