Chapter 09

Updates to Yahoo! Pipes (from Dec 2007)

While finishing my book, it wasn't possible for me to keep up with all the changes that were happening with the many web applications I track in the book.  One such change came in Yahoo! Pipes:  Pipes Blog » Blog Archive » New "Fetch Page" module and nice web path enhancement….

I have to try the Pipes Fetch Page Module to do some scraping of HTML pages. Also one can start using nicer URLs for various pipes.   For instance,

http://pipes.yahoo.com/pipes/pipe.run?InputURL=http%3A%2F%2Fwww.nytimes.com%2Fservices%2Fxml%2Frss%2Fnyt%2FInternational.xml&_id=cInT4D7B3BGMoxPNiXrL0A&_render=kml

can now be substituted with

http://pipes.yahoo.com/raymondyee/locationextractor?InputURL=http%3A%2F%2Fwww.nytimes.com%2Fservices%2Fxml%2Frss%2Fnyt%2FInternational.xml&_render=kml

Chapter 04
Chapter 09
Uncategorized
Yahoo! Pipes

Comments (0)

Permalink

New Drafts of Chapters 7, 9 13, and 17

I just posted the following chapters -- for your edication and entertainment:

As always, I'd be grateful for any constructive feedback:  questions, comments, expressions of bewilderment. :-)

Chapter 07
Chapter 09
Chapter 13
Chapter 17

Comments (0)

Permalink

Mining the data in ProgrammableWeb for Design Patterns in Mashups

In chapter 9, I look in detail at some individual mashups. I also want to know more about mashups in general, to do a macro-analysis of mashups. That is, I would look at the broadest range of mashups to look for design patterns that cross many examples.

One way forward would be an analysis using ProgrammableWeb, probably the single best compilation available of mashups and corresponding APIs available on the public web. There are some patterns that are immediately obvious from a study of the site; I say immediately obvious because John Musser, its creator has surfaced these elements in the interface. Let me point out some of the data about mashups:

  • You can get an overview of the mashup world, newly registered ones, what's popular at the Mashup Dashboard.
  • "mapping" is the most popular tag associated with mashups, followed by "photo"
  • The Web 2.0 Mashup Matrix displays mashups by their use of every combination of 2 APIs in the ProgrammableWeb database.

In addition, to what is obvious in the data, I would like to pose more questions that should be derivable from what is in ProgrammableWeb.com:

  • How many APIs are used by the mashups? That is, what's the distribution -- how many use 1, 2, 3, etc. APIs.
  • What's the most common pair of APIs being used? Most common threesome?
  • Is there any correlation between the popularity of an API and the popularity of mashups that use that API?
  • Are there broader correlations among usage patterns of APIs if we cluster them by categories? Are mashups likely to use more than one API in the same category or across categories?

As of the writing of this book, there is no formal API to programmableweb.com -- so answering these and allied questions require some other form of access to the data. I'm working with John to get such access.

Chapter 09
ProgrammableWeb
design patterns

Comments (0)

Permalink

Mashup tools to look at

In Chapters 9 and 11, I analyze service composition frameworks, tools that make it easier to create mashups, for "design patterns" among mashups. That is, if some tool offers a template, it's likely that there is a design pattern behind that template. If time allows, I'd like to study at the least the following frameworks. openkapow is one such system. I want to look at the ones highlighted in John Musser's recent presentation at Web 2.0 Expo. See Open APIs Talk at Web 2.0 Expo and specifically the quote from Digg floats API, phishing mashups to come:

    "The tool space is going to explode, both for developers and nondevelopers," Musser said. Of particular note were data mashup tools such as Yahoo Pipes, RSSBus, and Grazr; scraping tools for making structured data from unstructured data, such as Kapow and Dapper; and visual development tools, including JackBe, Teqlo, Bungee, and IBM's QEDWiki.

I'm already studying Yahoo! Pipes, Kapow, Dapper, and QEDWiki but have yet to look at:

How well do these tools work? We'll see.

Chapter 09
Chapter 11

Comments (0)

Permalink

At work on Chapter 9

I'm working today on the first draft of Chapter 9 "Dissecting mashups and remixes". Ideally, I'd like to create the equivalent of the Gang of Four's Design Patterns for mashups. Such a project is long-term effort. For this chapter, I suggest finding several emerging patterns from an analysis of a handful of specific mashups. I will also outline how we can look for patterns latent in the ProgrammableWeb database of mashups.

Where we can look for analyses of mashup-related patterns?

AJAX
Chapter 09
ProgrammableWeb
design patterns

Comments (0)

Permalink