• You have to trust the API providers to keep that API up and running. Anyone that has done Twitter mashups know how much keeping an API up and running is worth.
• You have to trust the API providers to keep the API (somewhat) stable and to not remove features that you are depending on, or add features you really dont want. An example of this is all the mashups built on Google Maps, and we all know that there are way too many of those. Have the developers of those mashups really thought about what happens when Google puts ads on the Maps (and they will, they have already started in parts of the US)? I dont think so, the developers just trust Google to keep on going. If you have a commercial site using Google Maps you probably do not want ads for your competitors on those maps.
• If your mashup becomes a success you have to trust the API providers you are depending on not to change their terms of service so that your mashup suddenly becomes an illegal use of the API. Also, you have to trust the API provider not to use the fact that you are depending on their API for your business against you in a business negotiation.
• If you use web scraping you have to trust the site you scrape not to change to often (or at least trust your own ability to roll with the punches and update your scraping so it works with the new version of the site as well) etc. That is a lot of trust, and when mashups move from toys into real applications this becomes an issue.

All these things makes me a bit nervous, as I am a bit paranoid (note: this has not been clinically proven, I still think that THEY are out to get me). So how to mix a healthy bit of paranoia into my mashup building and get something good out of it all? What I do is that I always try to be aware of that I might have to switch API provider. Are you building a Twitter mashup? Why not also take a look at Jaiku’s API, or Pownce’s API (or Plurk or FriendFeed etc etc). You dont have to build your mashup so you can switch API provider in a matter of minutes, just be aware what else is out there so that you see the commonalities and don’t use to many features unique to one provider. This is the approach I am currently using when I am building mashups, at least I know that if shit hits the fan I can always go with somebody else. It will hurt a bit and take some effot, but I am not dead in the water. For the Google Maps example this would mean looking at Yahoo Maps and see what features the Yahoo Maps API have in common with the Google Maps API, and just use those common features. This can also come in handy if you hit the maximum number of requests on Google Maps, then it would be nice switch to Yahoo Maps automatically.

The risk with all this is of course to spend to much time preparing for something that won’t happen. It is the same situation as developers spending so much time making their code perfectly scalable and optimized that they acctually never ship anything in time. So dont go too far, but be aware of the situation. Trust is nice, but trusting several API providers to always do a good job and to not be evil in order for you to survive is quite risky.

Forrester divides mashups into three types:

• Presentation layer mashups – merge content from seperate sources into one view, the simplest type of mashups.
• Data mashups – more complex data driven mashups that get data from several sources and present them in one view
• Process mashups – mixes business processes and users with data from several data sources.

Presentation mashups and data mashups sound very much similar to me, but then again I dont get payed by Forrester… But Forrester has a lot of influence over this so unless Gartner comes up with another definition this is the ones we have to live with.

I am glad to see that Forrester also realized that enterprise mashups will be huge. It is kind of a self realizing profecy – there will be a lot of men in ties reading this report so it is going to help Enterprise Mashups grow. It is really the next wave in enterprise software. And if you are reading my humble blog you are already years ahead of the mashup wave

For more info about this report see Forrester: Enterprise Mashups to Hit $700 Million by 2013 on ReadWriteWeb.

The API Provider
You have this great service that lifts humanity to a new level, makes the sun shine brighter, makes TV sucks less and give the gift of limitless bandwidth to the people (or at least it is really cool). You have even added this great API, now what? How do you get developers to start using the API and spread the word of your great service to everyone and their grandmother?

Well, let’s back up a bit. First of all, do you really have a great service? If you do, then do you really have a great API? Without a product people want to use there is no need to go through the hassle of promoting it. Make sure that the API actually is usefull for developers, that it will enable them to do cool and usefull stuff easier than if they would just hack it all together from scratch. Also make sure that there are plenty of documentation, examples, code snippets etc for the developers to get their hands on to minimize the barrier to entry. Hack together some mashups yourself with your API included in the mix, to give people and idea of what can be done. The key to get an API used by developers is to get the developers excited about the possibilities and get them talking. So give them something to be excited about and something to talk about.

Once all that hard work is done then you can promote your API via directories such as programmableweb and webmashup so that developers can find you. If you have made your own example mashups, then go through the steps below to market that, that is a good way of getting some recognition.

Last, but not at all least, show some love for the developers that has taken their time and built something using your API. Have an example gallery where they can list their creations. Blog about them. Talk about them at conferences. “Link love shall be bestowed upon those who link love showeth”.

The Mashup Developer
For the developer of the mashup there is Google AdSense money on the line, or maybe just recognition from peers. Most mashups result in web pages anyway, so make sure to do all the SEO stuff – have good page titles, have a good copy, have validating HTML, have a sitemap available etc. If there is money down the line for you then also throw some money at advertising (Google & Facebook makes this a walk in the park). All this is standard, but as there are differences between mashups and a regular web page you should also use that to your advantage.

What APIs do you use? What tools have you used to piece things together? Explain how you made your mashup, what the moving parts are. If you used Yahoo! Pipes, then link to the pipes used and explain how they were done. If you used Google Maps (and if you are a mashup newbie then I guarantee that you have, just admit it… “my name is Andreas, and I am a Google Maps addict”) then explain how. If you used openkapow robots, then explain how you developed them. Since API providers are suckers for traffic, just as everyone else, it is not unlikely that they would be interested in adding your mashup (assuming it kicks-ass, which of course it does) to their example gallery. All this creates more link love, more Google baits and really increases the chances of your mashup being found and appreciated by fellow developers. Another plus is that all this also increases the chances to be blogged about, do not forget that bloggers are suckers both for traffic and content.

There’s both money and recognition in entering your mashup in a contest, see programmableweb for a good list of what you can enter right now. You might not have to redo the mashup from the ground up, just add another API to the already great mashup you have made and you could already be a winner. If you go to Mashup Camp you could enter the traditional Speed Geeking (like speed dating for mashups basically) and go home with a shiny new Macbook.

Of course also list your mashups in directories such as programmableweb and webmashup , but by now you should know that already

Thanks for everyone that discussed this with me at Mashup Camp! For the notes from this session check out the Mashup Camp wiki.

• 7 % had very good knowledge of Mashups
• 13% had good knowledge of Mashups
• 12% had some knowledge of Mashups

Even people that knew nothing about Mashups apparently follows what happens, since 41% are following Mashups. On the other end of the spectrum only 1% of the people responding had acctually implemented and evaluated Mashups (they didn’t ask me, so I have no part of that lonely percentage). I am looking forward to seeing next years Web Service Award numbers, hopefully Mashups have become more mainstream by then.
Richard Gatarski on weconverse.com has a more detailed report on the different questions and answers in the poll.

• Do not be shocked when somebody mashes up your site, it will happen if it hasn’t already. It is a great possibility for marketing your services and not a threat!
• Use mashups to quickly create unique and usefull user experiences to enrich your web sites.

Here is my presentation (in Swedish only):

| View | Upload your own

The presentation went very well and I got a great response to it. It seems like I did at least tickle the interest of the audience and at least some of them will play with mashups and hopefully all of them will see mashups as an opportunity and not something weird and dangerous. I think that Sweden is getting ready to embrase the Mashup wave.

You can find all the presentations from the Web Service Awards here, I especially recommend the usability presentation from Johnny Mellgren at Vinnovera.

Thank you to Richard Gatarski for inviting me to do the presentation and thanks to all of the people I queued with for the food yesterday

• Microsoft Live Search will be integrated into Facebook and the Microsoft-Facebook allience will start to seriously compete with Google for the search market.
• Enterprise Mashups will be on the top of the hype cycle in the end of the year and big companies will start to seriously invest in it (and not just talk about it).
• The Web 2.0 bubble will burst, but that really just means that some big famous companies will go bancrupt (bye bye Technorati). It won’t really have any similarities with the .com bubble, instead it will mostly be something created in the buzz obsessed blogosphere and then be writen about in mainstream media.
• Microformats will be integrated into WordPress, Blogger, Flickr och other major sites, and be the basis for many interesting integrations and mashups.

Some predictions specific for Sweden:

• There will be a lot of talk about mashups in Sweden, but not much action.
• Schibstedt will try to make a Swedish social network – “a Swedish Facebook”. Be prepared for a major ad campaign.

You can find more Swedish predictions in the latest episode of the whatsnext podcast (sorry, swedish only).My last prediction is that Digitalistic soon will have a new design and a section all in Swedish, I really hope that I am not wrong about this one!

The potential of HTML

All new interesting applications (Skype being the exception that proves the rule) has an HTML interface. And this is true not just for the consumer facing applications, but for Enterprise level applications as well. So with the millions and millions of HTML pages in existance today it is not unlikely that HTML is one of the worlds most common data formats (I wonder how it compares to printed text and audio for example). The great thing with HTML is that it does not just contain data, it also is the interface to a whole lot of functionality (when you search Google you do that via HTML don’t you?). What if we could use HTML as one big API? That would make HTML the worlds most widespread API and that would give mashup developers and programmers access to more data and more functionality than ever before.

The problem with HTML

Almost not sites on the web today are following the HTML 4 standards. So todays browsers are very good at interpreting the tag soup that most pages consists of (ie broken HTML, missing end tags etc). Furthermore HTML is used to both mark up data in a document, for example with the <title> tag, and to mark up how the data should be presented, for example the <b> tag. All this together makes HTML documents unstructured documents (by implementation, not by nature) with data in very application specific formats (microformats will help here, but there will be some time before that is widespread enough to be really usefull).

Another problem is of course that there is fewer and fewer pages on the web that uses pure (albeit broken) HTML, there are more and more Javascript around. Especially in the Web 2.0 applications most of the really interesting functionality is available via AJAX. So it is not only HTML, but also Javascript that has to be taken into consideration when one wants to get to a web applications functionality.

Parsing

So we have huge amounts of data and functionality in HTML and we want to use it to make our latest funky Mashup. The good old approach is to try to parse the page in question using Perl, now it can be done pretty well using almost any modern programming language. There are several problems with parsing though:

• It is damn complicated to get to work on serious web pages and once it is done it breaks easily
• Good luck handling a real tag soup, already that breaks most parsers (since using XML parsers for this means that the parser simply stops at the first error it encounters)
• It is boring to program those parsers (if you havent tried then lucky you)
• Can not access functionality that uses javascript and AJAX
• It is hard to handle things like login into a web application (ie session handling) and to navigate over several pages

Still this is a very usual approach to get to data and functionality in HTML. But there is a much easier way…

Web Scraping

I bet that a fair portion of the people reading the word “Web Scraping” think of old mainframe terminals and “Screen Scraping” and frown. Don’t worry, technology has moved forward lightyears since the days of mainframes. Web Scraping is to interact with HTML (including Javascript if it is a good scraper) and to either extract data from the HTML or repackage the functionality in the HTML. The data can be saved into a database or a file for example, and the functionality can be made available as a REST service, a programming language API or whatever else makes sense. Suddenly HTML is wide open. Just imagine that you wanted to get data from Digg (before the Digg API that came out a few weeks ago) for some reason, without an API that would be hard. But using a web scraper you could for example build a REST service out of the search on Digg only by accessing the HTML. Web Scrapers are used more and more for doing things like collecting large amounts of job ads or flight information and then repackage that data into sites that then allow users to search for a job or a cheap flight.

Openkapow

The web scraper of my choice is the one supplied on openkapow.com (disclaimer: I am working for Kapow Technologies, the company behind openkapow.com, but trust me in that I am not plugging openkapow to make my boss happy – it is really a great product). Using openkapow one can access data and functionality on any web page and access it as a REST service or and RSS/Atom feed. Of course JavaScript is handled automatically, it is possible to navigate multiple pages, login to restricted pages and have full control over the process flow with conditions and error handling. I recommend that anybody that is interested in how to use HTML as an API takes a look at openkapow.

An Eye Opener…

Thinking of HTML as an API does significantly expand your horizons as a developer. I have literaly seen a light go on in fellow geeks eyes when they realize the potential. Suddenly the web is really yours to use in your programs and mashups. When suddenly APIs and services are abundant then you can start using the other cool mashup tools around (Teqlo, jMaki etc).