esjewett.com

Last weekend I pushed out the very first version of something I'm calling jsglue to Github. It now lives here: http://github.com/esjewett/jsglue/tree/master

jsglue is in essence a framework for implementing web-connective applications a la Yahoo! Pipes and Tarpipe. Currently it is at best a compliment to those programs and at worst totally useless. In the future I would like to see it or something like it become an alternative to these tools, for a few reasons that I'll eventually get into in later posts.

jsglue does three things:

• It allows you to register a handler to a path.
  • The handler consists of a path and two pieces of javascript - one that constructs a response to a request sent to that path, and one that constructs one request (and in the future multiple requests, optionally) to another URL.
• It accepts HTTP requests to paths with registered handlers.
  • When this happens, it creates a response using the handler javascript for this purpose, and it adds a job to a stack that will be processed later.
• It provides a program that can be run periodically to process the stack of jobs that has built up, sending off new requests as specified by the javascript in the handler associated with each job.

That's it.

Why do I care?

Well, hopefully that will become clear of its own accord. But the key is that the full contents of the original request are exposed to the javascript processing script that constructs the new request. As such, you can do pretty much any kind of processing you like within this handler code, which is user-defined.

So why do you care? Let me count the ways:

1. Receive a request in JSON and spit it back out multi-part form-encoded (in fact, right now this is pretty much the only thing you can do). Ever tried to connect up Yahoo! Pipes with Tarpipe? It doesn't work. With this, it can.

2. Webhooks are great. Webhooks are the facility to have a web application issue an HTTP request to an arbitrary URL when some event happens in the web application. That sounds boring, but it's actually awesome. Webhooks are great, except that no one speaks the same language so every webhook-based solution is bespoke. Bespoke is great in a suit or a coffee mug, but it's bad in web infrastructure. Yahoo! Pipes can't understand webhook calls. Tarpipe usually can't understand them. Most other web applications can't understand them. There needs to be some sort of middle-person.

3. If I'm going to run a ton of my personal data through some middleman web application, you should have the option of running that web application yourself. I'm not saying you will, but I think it would be nice if you could.

Okay, 3 ways is enough for now. We'll get to more later.

Why do you *not* care?

Well, there are lots of reasons for that too.

1. This is dorky. No, there is not a UI. No, it doesn't do much of interest. It's an infrastructure prototype more than anything else. The idea is really that we need infrastructure for building applications that can do this sort of thing. I don't have a lot of time to spare, so I'm willing to just put a framework out there, and maybe a REST-only web-application if I can get the components running on a hosting service (harder than it sounds). I'll leave it to someone else to put the UI on top of it. I'm not convinced that the "pipe" metaphor is correct (I'm partial to "tubes" myself), but I don't have a better idea, so someone else will have to have that idea.

2. This code sucks. Yes it does. I urge you to fork it, improve it, or throw up your hands in disgust and start over. I just want something that does this. I don't really care if it's written by me.

3. There's no way this execution model will fly on a public site, and no one is going to run this on their own server. This is sort a feature of this design that allows your users to execute arbitrary javascript on your server. As such, I think this will primarily find use on private servers, or as a back-end engine for a public site where the inputs are carefully cleansed. Not a recipe for ultra-popularity, I'll grant. But that's not really the point either.

So what's it made out of?

Currently, there are only four main ingredients:

• Ruby is the implementation language. It's role in jsglue is to serve as duct-tape for the other components.
• Datamapper is the database interface, allowing you to use pretty much any supported database (I'm using SQLite at the moment).
• Sinatra for the HTTP web-service interfaces. These interfaces are pretty much a direct mapping onto the database. (REST-ful? Maybe.) (Incidentally, how is it that a minuscule Ruby web-framework beats out FRANK SINATRA in the Google rankings?)
• Johnson for the Javascript processing.

That's it. It's a couple hundred lines of code. I haven't really counted, or put it on Ohloh.com for that matter, but it can't be more than that. It's got some unit tests. It's going to be changing quickly as I make it more multi-purpose.

I'll document and post examples as they become available.