esjewett.com

Last weekend I pushed out the very first version of something I'm calling jsglue to Github. It now lives here: http://github.com/esjewett/jsglue/tree/master

jsglue is in essence a framework for implementing web-connective applications a la Yahoo! Pipes and Tarpipe. Currently it is at best a compliment to those programs and at worst totally useless. In the future I would like to see it or something like it become an alternative to these tools, for a few reasons that I'll eventually get into in later posts.

jsglue does three things:

• It allows you to register a handler to a path.
  • The handler consists of a path and two pieces of javascript - one that constructs a response to a request sent to that path, and one that constructs one request (and in the future multiple requests, optionally) to another URL.
• It accepts HTTP requests to paths with registered handlers.
  • When this happens, it creates a response using the handler javascript for this purpose, and it adds a job to a stack that will be processed later.
• It provides a program that can be run periodically to process the stack of jobs that has built up, sending off new requests as specified by the javascript in the handler associated with each job.

That's it.

Why do I care?

Well, hopefully that will become clear of its own accord. But the key is that the full contents of the original request are exposed to the javascript processing script that constructs the new request. As such, you can do pretty much any kind of processing you like within this handler code, which is user-defined.

So why do you care? Let me count the ways:

1. Receive a request in JSON and spit it back out multi-part form-encoded (in fact, right now this is pretty much the only thing you can do). Ever tried to connect up Yahoo! Pipes with Tarpipe? It doesn't work. With this, it can.

2. Webhooks are great. Webhooks are the facility to have a web application issue an HTTP request to an arbitrary URL when some event happens in the web application. That sounds boring, but it's actually awesome. Webhooks are great, except that no one speaks the same language so every webhook-based solution is bespoke. Bespoke is great in a suit or a coffee mug, but it's bad in web infrastructure. Yahoo! Pipes can't understand webhook calls. Tarpipe usually can't understand them. Most other web applications can't understand them. There needs to be some sort of middle-person.

3. If I'm going to run a ton of my personal data through some middleman web application, you should have the option of running that web application yourself. I'm not saying you will, but I think it would be nice if you could.

Okay, 3 ways is enough for now. We'll get to more later.

Why do you *not* care?

Well, there are lots of reasons for that too.

1. This is dorky. No, there is not a UI. No, it doesn't do much of interest. It's an infrastructure prototype more than anything else. The idea is really that we need infrastructure for building applications that can do this sort of thing. I don't have a lot of time to spare, so I'm willing to just put a framework out there, and maybe a REST-only web-application if I can get the components running on a hosting service (harder than it sounds). I'll leave it to someone else to put the UI on top of it. I'm not convinced that the "pipe" metaphor is correct (I'm partial to "tubes" myself), but I don't have a better idea, so someone else will have to have that idea.

2. This code sucks. Yes it does. I urge you to fork it, improve it, or throw up your hands in disgust and start over. I just want something that does this. I don't really care if it's written by me.

3. There's no way this execution model will fly on a public site, and no one is going to run this on their own server. This is sort a feature of this design that allows your users to execute arbitrary javascript on your server. As such, I think this will primarily find use on private servers, or as a back-end engine for a public site where the inputs are carefully cleansed. Not a recipe for ultra-popularity, I'll grant. But that's not really the point either.

So what's it made out of?

Currently, there are only four main ingredients:

• Ruby is the implementation language. It's role in jsglue is to serve as duct-tape for the other components.
• Datamapper is the database interface, allowing you to use pretty much any supported database (I'm using SQLite at the moment).
• Sinatra for the HTTP web-service interfaces. These interfaces are pretty much a direct mapping onto the database. (REST-ful? Maybe.) (Incidentally, how is it that a minuscule Ruby web-framework beats out FRANK SINATRA in the Google rankings?)
• Johnson for the Javascript processing.

That's it. It's a couple hundred lines of code. I haven't really counted, or put it on Ohloh.com for that matter, but it can't be more than that. It's got some unit tests. It's going to be changing quickly as I make it more multi-purpose.

I'll document and post examples as they become available.

As might have been assumed from the my post on automated testing in SAP systems a couple months ago, I've been delving into testing in the SAP landscape. I'm beginning to put together a series of presentations and workshops on the subject, the first of which I was delighted to deliver last week.

Being the first in the series, this presentation focuses on an overview of the leading edge of the field. It would be nice to be able to emulate the sort of testing techniques we can use in Ruby in SAP BI and EPM application development. Nice, but not necessarily realistic. I can dream!

Johnson is a Ruby wrapper of the SpiderMonkey JavaScript interpreter. In practice, this means you can use Johnson to evaluate JavaScript statements within a Ruby program.

It works really nicely, but there is no released gem and building the development gem is a bit of a headache. The instructions in the readme at the Johnson Github site appear simple but have several prerequisites that must be fulfilled before the whole process works. Here's what I did on Leopard. I'll try recreating on Windows tomorrow.

[sudo] gem sources -a http://gems.github.com
[sudo] gem install jbarnette-johnson

Multiple errors occur - if you see gems missing (like hoe, for example) then install them.

[sudo] gem install jbarnette-johnson

Errors occur, probably due to a failure to compile the native gem. Usually something about a missing rake/extensiontask.

[sudo] gem install rake-compiler
[sudo] gem install jbarnette-johnson

Yet another error occurs - a missing Manifest.txt file. Download from Github at http://github.com/jbarnette/johnson/tree/master and put it into the directory in the error message.

[sudo] gem install jbarnette-johnson

Success!

[Apr. 4, 2009 - Fixed typo, changing "rake-compile" to "rake-compiler"]

This the continuation of my ongoing OAuth Q&A, now with fewer links and more editorial commentary. See Part 1 as well.

If I have access to an authorized token and the consumer key and secret, I can make "authorized" API requests galore. So OAuth is no more secure than the username/password pattern.

That's not a question. Way to start off part 2 with a big fail!

[Ed. - Ahem . . . You're asking the questions as well as answering them.]

[Me - I deny that. But okay, let's try again at a real answer.]

This is only partly correct.

First, OAuth provides the a pattern for providers to scope access much more granularly than when a single username/password pair gives total access to an app. For example, most OAuth providers do not allow access to administrative functions when authorizing with an OAuth token, so a rogue client cannot change a user's password. Because of this, the user's access to the app can be guaranteed and damage can be limited to the scope of access granted to the client.

Second, because the user's access can be guaranteed and because client access can be managed at the level of the token, authorization for a given token can be revoked in a self-service manner. The scenario here is that a user grants access to a malicious app that vandalizes the account. The user realizes this, logs in to the provider, revokes the consumer's access, and repairs (hopefully) the damage. Because the malicious consumer cannot change the password on the account, the provider does not have to become involved in the initial response to the vandalism. Follow-on activities that the provider may need to become involved in (like account restoration or consumer key revocation) still exist, but the need for instantaneous response is lowered.

Hey, you just said "consumer key revocation", but back in Part 1 you said that the provider can't assume a consumer key uniquely identifies a particular class of consumers. What gives?

It is correct that the provider can't make this assumption based on the OAuth specification. However, the provider can impose requirements around the consumer key on consumer developers, and the provider can reserve the right to revoke a consumer key for whatever reason they choose. This could be required in the case of denial of service attacks or malicious consumers using a particular key. Providers should work closely with consumer developers to clearly lay out what actions the provider will take in these cases and what options consumer developers have.

Providers should recognize that the case of desktop and web apps are significantly different in the case of key revocation and they should plan accordingly. In the case of web apps, it should be relatively easy to replace a compromised consumer key and secret. In the case of desktop apps, the consumer developer may face the task of updating thousands or millions of installed applications in the case of a key revocation.

Let's assume that a user has installed a malicious desktop app that wants to use the API of my web app. What's the point of requiring OAuth, since this malicious desktop app already owns the user's system and will have access to anything the user types?

I'm not sure where this assumption comes from. Well, let me amend that. I know exactly where this assumption comes from, but it is still incorrect. Let me assure the reader that there are many platforms where the installation of a malicious application does not translate into ownership of the host system. The reader will find this sort of behavior in any properly configured Unix-like operating system including Linux, on the iPhone, in any modern browser, in Mac OS X, and even in some locked down versions of the Windows operating system.

On these platforms, even if the user is tricked into installing a malicious client or going to a malicious website, the user may still be careful to verify that they are only entering their password into the real site of the provider. There are a lot of options for verifying this, but the best way is for the user to make sure that he or she always enters there password into their provider's site using a modern browser with anti-phishing technology and using an https connection.

This week I was writing a Web Dynpro ABAP interface for a long-running process. In general it is probably not a good idea from a performance perspective to run intensive processes synchronously in the context of a Web Dynpro view. It's also a really bad idea because it makes the user wait for the process to finish. If the process runs longer than the server's time-out setting then you get a nasty error message.

I knew that there was a way to handle this in ABAP, but I don't really live in the language, so it took me a little while to figure it out.  Here's the skinny:

The answer is to write a function module to carry out the processing for you. Enable the function module for RFC (and by extension, set all of the parameters to "Pass by value"). You then call the function module from a Web Dynpro controller method in the background. This registers the function module for execution as an RFC function call in the background.

Once you've done all of your "CALL FUNCTION xxxxx IN BACKGROUND TASK." statements, use the "COMMIT WORK." command to trigger the execution of these functions. You end up with a code snippet that looks something like:

call function zsample_function in background task
*  exporting
*    parameter =
*  importing
*    ret =
    .

commit work.

Now your function will start running and control will be returned to the user in the Web Dynpro UI.

One important thing to remember is that this method provides no feedback to the user by default. Usually it will be a good idea to at least return a message explaining that the task has been started. If possible further feedback should be provided for long-running jobs. I was able to do this by querying a status table that my job populated, but other options are available.

Help documentation - http://help.sap.com/saphelp_nw70ehp1/helpdata/en/8f/53b67ad30be445b0ccc968d69bc6ff/frameset.htm