Mashup Guide

I'm sad about the announcement that Google is shutting down its Mashup Editor -- one reason being that I had spent a fair amount of effort writing about it in Chapter 11 of  my book.  Oh well.  The Google App Engine is touted as a suitable and more powerful alternative to the Mashup Editor -- but I have to agree with that the comment that the simplicity of the Mashup Editor was a virtue.

According to How To: Getting Started with Amazon EC2 - PaulStamatiou.com, the following services are built on top of EC2 (and thereby perhaps make scaling up EC2 easier):

• Web-based Cloud Computing Management Platform by RightScale
• scalr - Google Code
• WeoCEO - Easy to Use. EC2 Love

What other ones are out there?

Continuing on Mashup Guide :: listing keys with S3Ajax, here I present a Chickenfoot script to create a bucket and upload a file (specifically, it creates a bucket by the name of raymondyeetest and uploads a file (D:\Document\PersonalInfoRemixBook\examples\ch16\Exported Items.rdf from my WinXP machine) to the key exportitems.rdf in the bucket:

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.URL = 'http://s3.amazonaws.com';
S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

// function to read contents from a file

function read_file_contents(aFileURL) {
  // https://developer.mozilla.org/en/Code_snippets/File_I%2f%2fO#Binary_File

  var ios = Components.classes["@mozilla.org/network/io-service;1"]
                      .getService(Components.interfaces.nsIIOService);
  var url = ios.newURI(aFileURL, null, null);

  if (!url || !url.schemeIs("file")) throw "Expected a file URL.";
  var bFile = url.QueryInterface(Components.interfaces.nsIFileURL).file;
  var istream = Components.classes["@mozilla.org/network/file-input-stream;1"]
                          .createInstance(Components.interfaces.nsIFileInputStream);
  istream.init(bFile, -1, -1, false);
  var bstream = Components.classes["@mozilla.org/binaryinputstream;1"]
                          .createInstance(Components.interfaces.nsIBinaryInputStream);
  bstream.setInputStream(istream);
  var bytes = bstream.readBytes(bstream.available());
  return bytes;

}

// create a bucket

var newBucketName = 'raymondyeetest';

S3Ajax.createBucket(newBucketName, function() {
    output("created a buicket: " + newBucketName);
    s3_list();
}, function () {
      output ("error in createBucket");
   }
);

// add a key to a bucket

var fileURL = 'file:///D:/Document/PersonalInfoRemixBook/examples/ch16/Exported%20Items.rdf'
var content = read_file_contents(fileURL);
alert(content);

S3Ajax.put(newBucketName , "exportitems.rdf", content,
    {
       content_type: "application/xml; charset=UTF-8",
       meta:  {'creator':'Raymond Yee'},
       acl:    "public-read",
    },
    function(req) {
        output("Upload succeeded");
    },
    function(req, obj) {
        output("Upload failed");
    }
);

A few things to note about this code:

• read_file_contents() doesn't strike me as a terribly elegant way of reading contents from a file -- but that's what I have working for now
• a tricky part of getting this to work was to note that in FF 3.x,  charset=UTF-8 is automatically tacked on to content-type HTTP request header in a xmlhttprequest -- I don't know how to change the charset or whether you can -- and why UTF-8 is being tacked on.  But figuring out the charset was crucial to getting this working since the content-type HTTP header is used in the calculation of the Amazon S3 signature.

In a previous post (learning how to use S3Ajax to access Amazon S3), I show how to use Chickenfoot and S3Ajax to list S3 buckets. Here is a slight elaboration of the code to list the keys within buckets (in addition to the buckets):

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

function s3_listKeys(bucket) {
  S3Ajax.listKeys(bucket,null,
    function(req, obj) {
        var contents = obj.ListBucketResult.Contents;
        for (var i=0, item; item=contents[i]; i++) {
           output("key: " + bucket + ": " + item.Key +
            " [" + item.LastModified + '] ' +
             " (" + item.Size + ")");
        }
    },
     function(req,obj) {
        output("s3_listKeys failed at " + (new Date()));
     }
  )
} // s3_listKeys

S3Ajax.listBuckets(
    function(req, obj) {
        if (obj.ListAllMyBucketsResult) {
            var buckets = obj.ListAllMyBucketsResult.Buckets.Bucket;
            for (var i=0, bucket; bucket=buckets[i]; i++) {
              output (
                    "bucket: " + bucket.Name + ' ['+
                      bucket.CreationDate+']'
              );
              s3_listKeys(bucket.Name);
            }
        }
    },
    function(req) {
        output ("Buckets list failed at "+(new Date()));
    }
);

In a previous post ( Amazon S3 signature calculation in JavaScript), I show how to calculate using JavaScript a "signature" need to access Amazon S3. In this post, I show a code snippet for using Leslie Michael Orchard's S3Ajax, which I'm evaluating for using to connect Zotero with Amazon S3. (Like the code from my previous calculation, it too depends on the sha1.js library.). Here's a Chickenfoot script (which is a slight rewriting of a L. M. Orchard's listbuckets function in play.js) to list your S3 buckets (you need to substitute the appropriate key/secret and download S3Ajax.js and sha1.js and substitute the correct path for these libraries):

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

S3Ajax.listBuckets(
    function(req, obj) {
        if (obj.ListAllMyBucketsResult) {
            var buckets = obj.ListAllMyBucketsResult.Buckets.Bucket;
            for (var i=0, bucket; bucket=buckets[i]; i++) {
                output (
                    bucket.Name + ' ['+bucket.CreationDate+']'
                );
            }
        }
    },
    function(req) {
        output ("Buckets list failed at "+(new Date()));
    }
);

I will have to try some more sophisticated actions -- but the simple listing of buckets worked for me.

On pp. 478-479 of Chapter 16 of my mashup book on online storage APIs, I show how to how to reproduce the calculation of an API signature in Amazon S3 in Python and PHP. Recently, because I now want to access S3 from within the browser, I have figured out how to do the same example using JavaScript. Specifically, the following is a Chickenfoot script that generates the correct signature.   (Note that the code depends on Paul Johnson's sha1.js library. (See his instructions on how to use the library. I had to consult the page to learn about the b64pad variable).):

// reproduce results on pp. 478-479 of
// Pro Web 2.0 Mashups

// http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAuthentication.html
// http://pajhome.org.uk/crypt/md5/instructions.html

include("D:\\Document\\JavaScriptLib\\sha1.js");

var AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82";
var AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o";
var Expires = '1175139620';
var HTTPVerb = "GET"
var ContentMD5 = ""
var ContentType = ""
var CanonicalizedAmzHeaders = ""
var CanonicalizedResource = "/johnsmith/photos/puppy.jpg"
var string_to_sign = HTTPVerb + "\n" + ContentMD5 + "\n" + ContentType + "\n" +
  Expires + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource
output(string_to_sign);

b64pad = "="; // needed for "strict RFC compliance"
var sig = b64_hmac_sha1(AWSSecretAccessKey , string_to_sign);
output ("|"+sig+"|");

In Chapter 8 of my mashup book, I wrote

Ideally there would be one obvious choice for an excellent JavaScript library, and everyone would use it. The current situation is that there are many JavaScript libraries, and it is not at all obvious how they compare.

I was hoping that in the course of writing my book, I would have had an opportunity to do my in-depth study of various JavaScript libraries (of which there are many, judging from the table in the Wikipedia entry Comparison of JavaScript frameworks). I raised the question of how to choose a JavaScript library during a panel on OpenAjax as a way of studying the implications of such initiatives as OpenAjax -- that is, does OpenAjax make it easier to figure out what JavaScript library to choose -- or does it allow one to pick and choose the best parts from a whole range of libraries? These are questions I plan to raise in my spring 2009 Mixing and Remixing Information course.

At any rate, I recently had a chance to take a closer look at AJAX Libraries API - Google Code, which is "a content distribution network and loading architecture for the most popular, open source JavaScript libraries", which at present, include:

• jQuery
• jQuery UI
• Prototype
• script.aculo.us
• MooTools
• Dojo
• SWFObject
• Yahoo! User Interface Library (YUI)

From a pedagogical point of view, the Google AJAX Libraries API, by providing a single common library call (google.load) simplifies getting set up with each of the libraries, which otherwise, would require a different setup. Of course, there are reasons that you might not want to depend on Google to host JavaScript libraries you use -- but not having to take care of hosting these files yourself helps beginners jump right in.

I'm pleased to see that The Yahoo! User Interface Library (YUI) recently became part of the Google API. I use that library in my course and in my book. The post on Google AJAX Search API Blog announcing this inclusion gave a short code example on how to use YUI with the Google AJAX Libraries API -- it did not actually show any visible in the user interface. Here I show to elaborate the example slightly to actually show something visible in the UI.

To do so, you should take look at the YUI docs for the calendar widget and understand a bit about what the YUILoader (Yahoo! UI Library: YUI Loader Utility does. One thing to note is that google.load('yui', '2.6.0'); creates an instance of a global YAHOO object. Secondly, YUILoader takes care of loading the pieces of YUI required to use the widget that you want to use -- see Getting Started for more details. You will also want to load in the default CSS to get started (through the line <body class="yui-skin-sam">). The modified code I came up with is:

<head>
  <script src="http://www.google.com/jsapi" type="text/javascript" charset="utf-8"></script>
  <script type="text/javascript" charset="utf-8">
  google.load('yui', '2.6.0');
  function init() {
    var loader = new YAHOO.util.YUILoader({
      require: ["button", "calendar"],
      base: "http://ajax.googleapis.com/ajax/libs/yui/2.6.0/build/",
      onSuccess: function() {
        // start playing with buttons and calendars!
        var cal = new YAHOO.widget.Calendar("calContainer");
        cal.render();
      }
    });
    loader.insert();
  }
  google.setOnLoadCallback(init);
  </script>
</head>
<body class="yui-skin-sam">
  <div id="calContainer"></div>
</body>

which you can run at http://labs.mashupguide.net/doc/2008/11/google.ajaxlib.yui.eg.html

See Data Unbound » Fixing the OpenID setup my WordPress blogs.  Thanks for your patience while I was fixing this problem -- and let me know if you continue to have problems with logging in and with commenting.

Yesterday, I was very pleased to read Ralph LeVan's review in Ariadne (Issue 57):

I can’t imagine a more comprehensive book on mashups. This book would make a great textbook for a class on the topic. If you are a developer of mashups, this book must be in your reference library. However, if you’re looking for a gentle introduction to the topic, it may be more than you want.

(Thanks to Lorcan Dempsey for pointing out the review to me.)

A little while ago, I also learned about a review on a German blogger: Pro Web 2.0 Mashups: Remixing Data and Web Services. Buchrezension. Since I can't read German, I turned to Google Translate (Pro Web 2.0 mashups: Remixing Data and Web Services. Book reviews). From what I can glean, the review seems very positive.

I've started preparing for the first panel at next week's Mashup Camp on OpenAjax Alliance: Why Ajax Standards Matter. Some things I'm reading and synthesizing:

• naturally, the OpenAjax Alliance website
• John Resig - Thoughts on OpenAjax -- the post by the master JavaScript developer from 2007 presents some good critiques of OpenAjax -- I wonder whether they have been adequately addressed.
• coverage of OpenAjax on Ajaxian
• Apathy comes easy to OpenAjax Alliance -- The Register -- a not-so-nice piece on OpenAjax -- but what do you expect from theregister?

A big question for me: what hands-on stuff is there to try? How hard is it to download code from Open source projects and take the projects "out for a spin"? Is there a screencast showing OpenAjax in action so that we can quickly see its purported benefits rather than just read about them?

It's also a good time to study Google's AJAX Libraries API ("a content distribution network and loading architecture for the most popular open source JavaScript libraries") as a way of comparing various leading JavaScript libraries. I plan to use it in the next "Mixing and Remixing Information" course I teach in Spring 2009.