Continuing on Mashup Guide :: listing keys with S3Ajax, here I present a Chickenfoot script to create a bucket and upload a file (specifically, it creates a bucket by the name of raymondyeetest and uploads a file (D:\Document\PersonalInfoRemixBook\examples\ch16\Exported Items.rdf from my WinXP machine) to the key exportitems.rdf in the bucket:

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.URL = 'http://s3.amazonaws.com';
S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

// function to read contents from a file

function read_file_contents(aFileURL) {
  // https://developer.mozilla.org/en/Code_snippets/File_I%2f%2fO#Binary_File

  var ios = Components.classes["@mozilla.org/network/io-service;1"]
                      .getService(Components.interfaces.nsIIOService);
  var url = ios.newURI(aFileURL, null, null);

  if (!url || !url.schemeIs("file")) throw "Expected a file URL.";
  var bFile = url.QueryInterface(Components.interfaces.nsIFileURL).file;
  var istream = Components.classes["@mozilla.org/network/file-input-stream;1"]
                          .createInstance(Components.interfaces.nsIFileInputStream);
  istream.init(bFile, -1, -1, false);
  var bstream = Components.classes["@mozilla.org/binaryinputstream;1"]
                          .createInstance(Components.interfaces.nsIBinaryInputStream);
  bstream.setInputStream(istream);
  var bytes = bstream.readBytes(bstream.available());
  return bytes;

}

// create a bucket

var newBucketName = 'raymondyeetest';

S3Ajax.createBucket(newBucketName, function() {
    output("created a buicket: " + newBucketName);
    s3_list();
}, function () {
      output ("error in createBucket");
   }
);

// add a key to a bucket

var fileURL = 'file:///D:/Document/PersonalInfoRemixBook/examples/ch16/Exported%20Items.rdf'
var content = read_file_contents(fileURL);
alert(content);

S3Ajax.put(newBucketName , "exportitems.rdf", content,
    {
       content_type: "application/xml; charset=UTF-8",
       meta:  {'creator':'Raymond Yee'},
       acl:    "public-read",
    },
    function(req) {
        output("Upload succeeded");
    },
    function(req, obj) {
        output("Upload failed");
    }
);

A few things to note about this code:

• read_file_contents() doesn't strike me as a terribly elegant way of reading contents from a file -- but that's what I have working for now
• a tricky part of getting this to work was to note that in FF 3.x,  charset=UTF-8 is automatically tacked on to content-type HTTP request header in a xmlhttprequest -- I don't know how to change the charset or whether you can -- and why UTF-8 is being tacked on.  But figuring out the charset was crucial to getting this working since the content-type HTTP header is used in the calculation of the Amazon S3 signature.

In a previous post (learning how to use S3Ajax to access Amazon S3), I show how to use Chickenfoot and S3Ajax to list S3 buckets. Here is a slight elaboration of the code to list the keys within buckets (in addition to the buckets):

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

function s3_listKeys(bucket) {
  S3Ajax.listKeys(bucket,null,
    function(req, obj) {
        var contents = obj.ListBucketResult.Contents;
        for (var i=0, item; item=contents[i]; i++) {
           output("key: " + bucket + ": " + item.Key +
            " [" + item.LastModified + '] ' +
             " (" + item.Size + ")");
        }
    },
     function(req,obj) {
        output("s3_listKeys failed at " + (new Date()));
     }
  )
} // s3_listKeys

S3Ajax.listBuckets(
    function(req, obj) {
        if (obj.ListAllMyBucketsResult) {
            var buckets = obj.ListAllMyBucketsResult.Buckets.Bucket;
            for (var i=0, bucket; bucket=buckets[i]; i++) {
              output (
                    "bucket: " + bucket.Name + ' ['+
                      bucket.CreationDate+']'
              );
              s3_listKeys(bucket.Name);
            }
        }
    },
    function(req) {
        output ("Buckets list failed at "+(new Date()));
    }
);

In a previous post ( Amazon S3 signature calculation in JavaScript), I show how to calculate using JavaScript a "signature" need to access Amazon S3. In this post, I show a code snippet for using Leslie Michael Orchard's S3Ajax, which I'm evaluating for using to connect Zotero with Amazon S3. (Like the code from my previous calculation, it too depends on the sha1.js library.). Here's a Chickenfoot script (which is a slight rewriting of a L. M. Orchard's listbuckets function in play.js) to list your S3 buckets (you need to substitute the appropriate key/secret and download S3Ajax.js and sha1.js and substitute the correct path for these libraries):

include("D:\\document\\JavaScriptLib\\sha1.js");
include("D:\\document\\JavaScriptLib\\S3Ajax.js");

var AWSAccessKeyId = "[AWSAccessKeyId]";
var AWSSecretAccessKey = "[AWSSecretAccessKey]";

S3Ajax.DEBUG = true;
S3Ajax.KEY_ID = AWSAccessKeyId;
S3Ajax.SECRET_KEY = AWSSecretAccessKey;

S3Ajax.listBuckets(
    function(req, obj) {
        if (obj.ListAllMyBucketsResult) {
            var buckets = obj.ListAllMyBucketsResult.Buckets.Bucket;
            for (var i=0, bucket; bucket=buckets[i]; i++) {
                output (
                    bucket.Name + ' ['+bucket.CreationDate+']'
                );
            }
        }
    },
    function(req) {
        output ("Buckets list failed at "+(new Date()));
    }
);

I will have to try some more sophisticated actions -- but the simple listing of buckets worked for me.

On pp. 478-479 of Chapter 16 of my mashup book on online storage APIs, I show how to how to reproduce the calculation of an API signature in Amazon S3 in Python and PHP. Recently, because I now want to access S3 from within the browser, I have figured out how to do the same example using JavaScript. Specifically, the following is a Chickenfoot script that generates the correct signature.   (Note that the code depends on Paul Johnson's sha1.js library. (See his instructions on how to use the library. I had to consult the page to learn about the b64pad variable).):

// reproduce results on pp. 478-479 of
// Pro Web 2.0 Mashups

// http://docs.amazonwebservices.com/AmazonS3/2006-03-01/RESTAuthentication.html
// http://pajhome.org.uk/crypt/md5/instructions.html

include("D:\\Document\\JavaScriptLib\\sha1.js");

var AWSAccessKeyId = "0PN5J17HBGZHT7JJ3X82";
var AWSSecretAccessKey = "uV3F3YluFJax1cknvbcGwgjvx4QpvB+leU8dUj2o";
var Expires = '1175139620';
var HTTPVerb = "GET"
var ContentMD5 = ""
var ContentType = ""
var CanonicalizedAmzHeaders = ""
var CanonicalizedResource = "/johnsmith/photos/puppy.jpg"
var string_to_sign = HTTPVerb + "\n" + ContentMD5 + "\n" + ContentType + "\n" +
  Expires + "\n" + CanonicalizedAmzHeaders + CanonicalizedResource
output(string_to_sign);

b64pad = "="; // needed for "strict RFC compliance"
var sig = b64_hmac_sha1(AWSSecretAccessKey , string_to_sign);
output ("|"+sig+"|");

It's tempting to enter an idea into the Amazon.com: Startup Challenge. Alas I don't have enough time to flesh the idea sufficiently to become a serious contender.  I've long wanted to pull together some deep personal computational infrastructure by combining S3 and EC2 and SQS -- but writing up those ideas will have to wait.

I've been captivated by the potential of Amazon S3 (Amazon's "Simple Storage Service"), which is described in the following way:

Amazon S3 provides a simple web services interface that can be used to store and retrieve any amount of data, at any time, from anywhere on the web. It gives any developer access to the same highly scalable, reliable, fast, inexpensive data storage infrastructure that Amazon uses to run its own global network of web sites.

With S3 (combined with Amazon EC2 -- the elastic computing cluster), I keep thinking that I have the raw ingredients for a relatively inexpensive supercomputer. Now what to do with that computing power -- that's the subject of another post.

At a basic level, Chapter 16 is meant to be a tutorial of Amazon S3 and rival/parallel/comparable services. What are other services that I'd like to cover (if I manage to have enough time to write up)? On the list of possible services to cover are:

• Box.net with Box Enabled - Box.net Web Services API
• Streamload with its API
• Omnidrive and the OmniDrive API
• openomy and its API

First place to start: learn to program Amazon S3. One thing I'm reading is Amazon Web Services Developer Connection : Monster Muck Mashup - Mass Video Conversion Using AWS, which touches on not only S3 but also Amazon EC2 and the Simple Queue Service.