Mashup Guide » screen scraping http://blog.mashupguide.net A blog about Raymond Yee's Book Pro Web 2.0 Mashups: Remixing Data and Web Services Wed, 23 Feb 2011 13:35:50 +0000 en hourly 1 http://wordpress.org/?v=3.3.1 http://blog.mashupguide.net http://blog.mashupguide.net/favicon.ico Mashup Guide Decay in the Amazon APIs http://blog.mashupguide.net/2011/02/23/decay-in-the-amazon-apis/ http://blog.mashupguide.net/2011/02/23/decay-in-the-amazon-apis/#comments Wed, 23 Feb 2011 13:35:50 +0000 raymondyee http://blog.mashupguide.net/?p=315 In Chapter 17 of Pro Web 2.0 Mashups, I created a mashup of an Amazon Wishlist and Google Spreadsheets. When I returned to examine my code last night, I learned that it no longer worked.  Why?  First, the  Amazon Ecommerce API morphed into the Amazon Product Advertising API; I was puzzled why the API wasn't [...]]]>

In Chapter 17 of Pro Web 2.0 Mashups, I created a mashup of an Amazon Wishlist and Google Spreadsheets. When I returned to examine my code last night, I learned that it no longer worked.  Why?  First, the  Amazon Ecommerce API morphed into the Amazon Product Advertising API; I was puzzled why the API wasn't listed where I expected it to be.  Unfortunately, Amazon, in its infinite and inscrutable wisdom,  also decided to kill the ListLookup operation, the one call that I depended on to retrieve the content of my Amazon wishlist.  (I'm not alone in having broken applications because of this change.)

So what to do now?  Interestingly enough, someone just announced a JSON feed service for a given wishlist, for example, Jeff Bezos' wishlist and mine (in JSON).  I hope it stays around.  How does it work given the demise of the ListLookup operation?  My guess is that some sort of screen-scraping is going on.

]]> http://blog.mashupguide.net/2011/02/23/decay-in-the-amazon-apis/feed/ 0 Screen-scraping references http://blog.mashupguide.net/2007/05/24/screen-scraping-references/ http://blog.mashupguide.net/2007/05/24/screen-scraping-references/#comments Thu, 24 May 2007 14:15:22 +0000 raymond.yee http://blog.mashupguide.net/2007/05/24/screen-scraping-references/ Even though my book focuses on the use of formal APIs for mashups, I'd like to provide guidance on screen-scraping and other forms of reverse engineering to my readers. There's plenty of mashup work that can be done even if you confine yourself to using only formal APIs. Sometimes, it's handy or even necessary to [...]]]>

Even though my book focuses on the use of formal APIs for mashups, I'd like to provide guidance on screen-scraping and other forms of reverse engineering to my readers. There's plenty of mashup work that can be done even if you confine yourself to using only formal APIs. Sometimes, it's handy or even necessary to supplement your use of formal APIs with other ways to get at the data, functionality, or user-interface elements that you want to recombine or mashup.

Some inter-related areas to cover (or at least to make reference to):

Some issues I want to address:

  • In the book, we seek to exploit as many of the structured information available designed for consumption by programs as we can before we move on to interpreting output meant primarily for human viewing. Screen-scraping brings up a lot of issues, technical and social, that we can get back to once you learn how to use APIs.
  • legal issues, terms of use -- see Web scraping - Wikipedia, the free encyclopedia

Some references:

]]> http://blog.mashupguide.net/2007/05/24/screen-scraping-references/feed/ 0