Skip to content

Updates to Yahoo! Pipes (from Dec 2007)

While finishing my book, it wasn't possible for me to keep up with all the changes that were happening with the many web applications I track in the book.  One such change came in Yahoo! Pipes:  Pipes Blog » Blog Archive » New "Fetch Page" module and nice web path enhancement….

I have to try the Pipes Fetch Page Module to do some scraping of HTML pages. Also one can start using nicer URLs for various pipes.   For instance,

http://pipes.yahoo.com/pipes/pipe.run?InputURL=http%3A%2F%2Fwww.nytimes.com%2Fservices%2Fxml%2Frss%2Fnyt%2FInternational.xml&_id=cInT4D7B3BGMoxPNiXrL0A&_render=kml

can now be substituted with

http://pipes.yahoo.com/raymondyee/locationextractor?InputURL=http%3A%2F%2Fwww.nytimes.com%2Fservices%2Fxml%2Frss%2Fnyt%2FInternational.xml&_render=kml

Tagged

The book is *almost* done

Last night, I finished writing the introduction, cover blurb, and acknowledgments for my book.  With any luck, that means that all the remains for me is editing the galley proofs that come back to me.  The current estimate is that my book will be in bookstores in February.

This blog has been very quiet for the past couple of months.  I learned that for me at least, the last stages of writing a book soaked up whatever energy I would have had to put into blogging.   Now, I expect to return to writing here, to highlight parts of my book, to publicize it, to update it as I can, and to interact with you my readers.

Once the book is out, I will then update this site to hold the published edition of the book instead of the current early drafts.

Amazon startup challenge

It's tempting to enter an idea into the Amazon.com: Startup Challenge. Alas I don't have enough time to flesh the idea sufficiently to become a serious contender.  I've long wanted to pull together some deep personal computational infrastructure by combining S3 and EC2 and SQS — but writing up those ideas will have to wait.

Notelets for 2007.09.25

I was going to try Google Presentation, the new Powerpoint-like subsystem of Google Docs, until I learned that it doesn't export presentations to Powerpoint or OpenDocument. (See Can I download a presentation to my local computer?.) Now, I might create my PPT or OpenDocument presentation and then upload it to see how good the importing facility is.

There is a new YouTube API based on GData, but there is still upload and write-capabilities: YouTube API Blog: We hear you – we know that upload and other write capabilities are wanted by the community. These features are what we're focusing on next. There is a PHP-based library that lets you do YouTube uploading: PHPTube – YouTube API for Video Upload & Download » nonsmokingarea.com.

One specific questions I have had about tags is how stable is the tagging for a given photo. I think that Cloudalicious – Watching Tag Clouds Over Time does a pretty good job at showing me that there is quick convergence. That's a rough take on what it shows….

A rant about the complexity of working with SOAP services

For the second draft of Chapters 6 and 7, I've been writing up a section on working with SOAP. I knew that it might be complicated to explain but I had no idea that it would be so difficult to distill the subject down to clear rules of thumbs that really work. I need to give up right now on trying to figure out all the kinks — of which there are many — in consuming SOAP services from a variety of languages. I've made a lot of progress but that work has been really slow-going and extremely frustrating. More to the point, the contribution that a section on SOAP/WSDL makes towards finishing the book are insufficient to justify the immense amount of effort I have to expend on the topic.

I've come to some basic conclusions about SOAP and WSDL. A fundamental problem with tightly coupled exchange protocols such as SOAP is that there too many places where errors can be made. And when you run into problems, It's hard to even know where the error are. And the protocols are pretty complicated. Underlying complexity wouldn't matter if everything just worked. But that's the case right now, forcing people who want to consume SOAP services to dive down into the details of the protocol. It seems that it's been difficult for people to write and interpret the protocol properly. Or people in scripting community have generally thought it not worth the effort to do so. SOAP+WSDL are very fragile technology

Ideally, you wouldn't have to know much about how the underlying technology works. But the documentation of SOAP libraries is often bad. You are not told what certain parameters mean. That means you need to know about how SOAP works to make sense of the parameters. That really shouldn't have to be the case. Moreover, there is more coupling between the conceptual models of SOAP and WSDL than one would know. So much easier to get started with REST, even if REST has a lot of subtleties of its own.

Comments anyone? I'd really like to be shown where my thinking is wrong and how I can start using SOAP and WSDL with ease in PHP, Python, etc.

2nd drafts of Chapters 3, 4, and 5 posted

For your reading pleasure….

Chapter 3: "Tagging and Folksonomies." (2007-08-08)

Chapter 4: "Feeds, RSS and Atom" (2007-08-08)

Chapter 5: "Remixing Weblogs" (2007-08-08)

I'll be at Mashup Camp IV today and tomorrow….

I will writing my notes here from Mashup Camp IV, where I hope to see many of the people I've been corresponding with and writing about over the last months.

Chapter 2– second draft posted

For your reading pleasure, I just posted the second draft of Chapter 2 ("Uncovering the Mashup Potential of Websites").  It is a thoroughly rewritten version of the first draft,  a much deeper piece of work.  It did take me a lot of rethinking to get this 39 page chapter in order — but I think you'll like it.  This concentrated effort also accounts for my silence on this blog….

At this point, I have written a first of draft of everything in the book, except Chapters 10, 11, and 12.  (I have a pre-first-draft version of Chapter 10 online that I've uploaded so that my students can work from it.)

Update (2010-03-03):  The link to the old version of Chapter 10 no longer works.  The link to the published version of Chapter 10 is available.

First drafts of Chapter 18 and 19 are posted.

Please let me know whether you have any comments on the following chapters:

New Drafts of Chapters 7, 9 13, and 17

I just posted the following chapters — for your edication and entertainment:

As always, I'd be grateful for any constructive feedback:  questions, comments, expressions of bewilderment. 🙂