Mashups are all the rage, but what exactly are they and should we be creating and enabling them? This article series explores mashups, providing the fundamentals as well as the practice behind the concept. This first article provides an introduction to the core concepts founds in mashups, examining the background, techniques and consequences of using, creating and enabling mashups in your IT infrastructure.

The second article in this series presents several concrete techniques for building mashups. By the end of the second article you will be in a position to comfortably create your own mashups on the Force.com platform.

Introduction

An influential article defining the Web 2.0 movement talks about cooperation, instead of control. This is something to keep in mind when approaching the concepts of mashups. In a nutshell, a mashup is created from several data sources and services, mashing up (combining, stitching together) the sources and services to create something new, or add value in some way.

That's a rather broad definition. As an example consider the ubiquitous Google Maps mashup. You could take Google Maps functionality and data (the ability to show and navigate maps) and mash it up with data about your accounts to map account locations, providing popup detail bubbles as you mouse over their locations. In effect you would be mashing up the location data of your client with the mapping data and services provided by Google Maps, using Google Maps to display the result on your own site—for example embedded on the Accounts page in Force.com.

Mashup chicago part1.jpg
Figure 1: An example of a mashup from ChicagoCrime.org

Figure 1 provides an example of this kind of mashup--a snapshot taken from the web site ChicagoCrime.org. What this web site does is mash up Google Maps with crime data in the Chicago area. This new hybrid Web application allows you to immediately see where the crime is occurring. It's mashing up data from the crime authorities with visualization functionality from Google.

As another example consider mashing up a business process with a phone SMS service. At some point in your interaction with a client you may want to send him an SMS to notify him of the current process state. SMS services are now available, and in this case you could compose the appropriate data and send it to the external SMS service using a web service protocol for broadcast.

Of course, I can't give a nice snapshot of this functionality because there is nothing visual about it--which makes the point. Not all mashups are visual or "on the web," though there are groups of developers out there who only use the term mashup for Web applications.

Creating mashups then, is about finding services and data and functionality and mixing it with other services and data and functionality. So what exactly are we mashing up, and where are we doing it? As these examples demonstrate we could do it at the browser level, perhaps with only some HTML or JavaScript. Alternatively, we could do it on some data services level using heavy protocols such as SOAP and web services.

It's helpful to classify mashups into a taxonomy using these rules of where the mashup takes place and just what is mashed up; doing this will let you understand the development effort and tooling needed to create them, and other implications such as security and auditing.

Categorizing Mashups

Knowing what is mashed up and where, will help you understand the technology, tools and skills needed to create or use a mashup. The following sections define three broad categories of mashups, two on the client and one on the server. Each is illustrated with an example and an indication of what it takes to produce a mashup in the category.

Client Presentation Mashups

Let's think a little about combining data and services. Say we wanted to spice up the Accounts page on Force.com. If we were feeling particularly lazy we could simply find some interesting functionality out there on the web, cut and paste the HTML/JavaScript code that was used to produce it, and embed that on our Accounts page. For example, the code could display a live clock or an external RSS feed.

There is little integration going on here, but it is a visual presentation mashup of sorts. Live data and functionality is simply embedded on a web page. We call these types of mashups client presentation mashups.

Technologies that you'd typically find here for the presentation include HTML, Ajax or Flash, usually prepackaged. The chunks of code here are sometimes called widgets, gadgets or badges too. Remember, we are describing something that displays information or adds functionality to a page rendered on a browser.

You can imagine this functionality could allow you to build portals by composing individual portlets--each portlet providing some useful bit of information. This kind of mashup is sometimes called an enterprise mashups when the portal provides a dashboard of activity relevant to your managerial status for example. At its heart, it's a simple syndication of content from some provider. They're easy to use, and let non-technical folk get started very easily--cut and paste mashups.

Mashup gadgets part1.jpg
Figure 2: Google Gadgets - easy to use, packaged functionality

The Google Gadgets site provides some great examples of these. Browsing around should give you an idea of what you can embed. Examples include instant messengers, weather forecasts, stock charts and even Pac-Man! The code for each gadget is already written, packaged and hosted, and including such functionality on a Web page is as simple as pasting a line or two into a custom S-Control. S-Controls, the key to embedding client mashups, will be covered in detail in a later article.

Client Service Mashups

With browsers as powerful as they are these days, we can really do a lot better than simply pasting some packaged code into our Web pages. Ajax allows you to make web service calls right from the browser itself. It also allows you to dynamically modify the Web page being presented. In effect, you can fetch, mashup and display right there on the browser. This is an example of a client service mashup.

It's important to distinguish this category from the simpler client presentation mashups. In the client presentation mashups, it's all about taking packaged widgets and adding them to a page with minimal integration. It's also about giving a name to a growing widget industry. In contrast, client service mashups are actually about services; perhaps calling web services or pinging web sites and feeds, or invoking the Force.com platform from the browser and processing the result.

Mashup sfmash part1.jpg
Figure 3: A modified Force.com Account details page, with a client presentation mashup on the left (the Google Gadget) and a client service mashup on the right (Yahoo! Maps)

Figure 3 shows an example of both types of client mashups. The map is an interesting example of a client service mashup because:

  • It requires advanced Ajax to allow the map to be scrolled and panned, and for the operation of the popups.
  • Much of the mapping functionality and data is external to our Force.com application.
  • It requires data from the Force.com platform to know where to place the popups and what information to put into them, and this information is internal to our Force.com application (yet we display it on a browser).

To display the map in Figure 3, JavaScript code is executing that makes dynamic calls (using a SOAP API) to the Force.com platform for data to display. It's a level up from the simply cut-and-paste mashup presentation mashups of the previous section—the code displaying the map is interacting with the Force.com platform using web services.

It's obvious that to create these types of mashups from scratch, a little more technology and skills are needed. Technologies that you'd typically use here include JavaScript, Flash or Applets, while data protocols often include RSS, XML and JSON. Luckily even these types of mashups are relatively easy to build on the Force.com platform, especially with the dedicated Ajax Toolkit.

Custom S-Controls that contain the code to produce such a map can make full use of the Force.com API using JavaScript. The JavaScript, in turn, can also interact with the other functionality on the page producing the map. Here's an excerpt from some code that is fairly typical:

1. <script language="javascript" src="/soap/ajax/8.0/connection.js"
2.         type="text/javascript">
3. <script>
4. function fetchAndDisplayData() {
5.    var result = *sforce.connection.query("Select Id, Name, Industry From Account")*
6.    output = processAccounts(result);
7.    document.getElementById("showme").innerHTML = output;
8. }
9.  <body onload="fetchAndDisplayData();">
10.     <div id="showme"></div>
11. </body>

Lines 1,2 and 3 include the Ajax toolkit. Lines 9,10,11 create an empty page, but call the fetchAndDisplayData() method. The method makes a query into our Salesforce data (line 5), processes it in some way to produce HTML and displays it (line 7). Line 5 disguises the power of this form of mashup: that's a straight Sforce Object Query Language (SOQL) statement embedded in the JavaScript.

Though the code may seem a little bewildering to JavaScript newbies, the key is that Salesforce provides an extensive and simple to use toolkit that lets you interact with your Salesforce data from your browser. This in turn means that you can extend the Force.com platform in a number of ways, taking that data and mashing in up with other data sources before visualizing the result for example.

External Service Mashups

Not all mashups happen on the client. At the heart of mashups is accessing and combining data sources in interesting ways, and these data sources can be external. For example, your intranet could be hosting several dynamic services itself using Ruby, PHP or Perl. The Force.com platform provides an extensive web services API that allows you to build applications that live outside of the Force.com platform to access your Salesforce data. In fact, there are a number of libraries built for Ruby, Perl, PHP, Flex and other programming languages that makes it very easy to use this API.

In this case we are in effect treating the Force.com platform as a client of some external system where the mashup takes place. For example, we could write a Ruby on Rails application that runs in our internal IT department that mashes up local relational data stored in a decrepit mainframe with the externally hosted Salesforce data. In this case, these types of mashups start looking a lot like data integration.

Taking this one step further, we could expose an internal web service that exposes some of our Salesforce-hosted data. Other divisions within our company could use this web service to compose richer applications in turn, mashing it up with other internal services or data. This is the start of service-oriented architectures.

Technologies that are used here commonly include web service APIs, WSDL and client languages such as Java or Ruby. As these technologies indicate, more technical skill is required to create these types of mashups.

Why Consider Mashups

Now that we've taken a look at what mashups are, let's consider whether they're even worth pursuing. What's the point of mashups? Why should we create or enable them?

Using Mashups

A good vote in favor of using client presentation mashups is that they provide immediate benefit at little cost. For example, enhancing the Account detail page with a stock chart for the associated ticker symbol is trivial and provides obvious value. Yes, you can still use the default click through on the ticker symbol to the Yahoo! finance page, but here's some of the important data right there on that page (in one line of code).

Mashups have a strong flavor of reuse. Why reinvent the wheel when someone else has implemented it and provided an API? For example, take a look at the wonderful Timeline system. This is very useful functionality that I no longer have to implement. All I need to do is mashup my Salesforce data with the Timeline and I have an awesome new piece of functionality. How about viewing and editing resume's of potential job applicants? You could simply mashup your applicant data with Google Docs and get this for free. For an example of these mashups in action, check out the excellent Winter '07 Release Video Tour video (Slide 6).

All of this means a faster time-to-market when building new applications, which is perhaps the most compelling reason for using mashups. Though we've only concentrated on some of the well-known types of mashups, take a look at ProgrammableWeb for a list of thousands of APIs available right now.

Enabling Mashups

With so many people using mashups, it makes sense for you to make it easy for other groups to mashup your data and services--when that's a viable option. Take for example the Google Gadget showing the Yahoo! stock chart. That's promoting Google and Yahoo!--even though it's providing value (the stock prices of a company). You can have your data and services similarly used and promoted by making it easy for others to access your services. In effect, you will be making it easy for others to write applications for you.

As we indicated earlier this is easily done on the Force.com platform using the SOAP API. This API lets you easily expose all your Salesforce objects and data as WS-I compliant web services. This means that you can access the services using almost any programming language out there today.

It's Not All Fun and Games

Who controls the API? What about stability considerations and licensing?

There are a few caveats to using mashups that are worth bearing in mind. Most of these only appear when you mashup externally hosted services. For example, some services provide limits to the number of calls that you can make to an exposed web service, and almost all require some sort of registration. Mature services, such as the Google Maps API, have a well-defined terms of use. Unfortunately, many newer APIs don't, or they are overly restrictive. Another item to consider is API stability. If you are mashing up an external API, it would be good to know that their service is stable and reliable. These kinds of issues may make life a little tricky for IT managers.

Having said that, many mashups are created using services exposed by some very credibly companies such as Amazon, Google and Yahoo!. Figure 4 shows the top APIs used for mashups for March 2007.

Mashup topapis part1.jpg
Figure 4: Top APIs as listed on programmableweb.com/apis"

Another tenet of Web 2.0 is that "Data is the Next Intel Inside." The data is often the valuable part of any service, and who owns the data? Obviously if you expose your own services you will need to think about this issue quite carefully. What data do you want to provide and how much of it? How do you regulate the usage of an API that you expose? What security controls do you have in place? For example, what's to prevent a competitor from using your web service to build their own database from your data? What are the license issues surrounding the data that you expose? Is the data that you record on behalf of someone else portable?

Summary

Mashups let you take part in the reuse software culture. Instead of coding up some functionality that already exists, client mashups let you embed that functionality into your web pages. Many mashups require little integration, especially the presentation mashups where hundreds of widgets are readily available. The Ajax Toolkit supplied by Salesforce makes creating client service mashups a walk in the park, and the web services API enables external service mashups too.

Mashups provide immediate benefit at little cost. The number of APIs that can be used in a mashup grows daily—providing a rich ecosystem of data and services that can be used to extend your Force.com applications. This all means a faster time to market with exciting new functionality that helps your workforce accomplish their daily tasks.

References

  • What is Web 2.0 (O'Reilly Network, 2005) - a great article making a good stab at defining Web 2.0
  • programmableweb - the one stop shop for finding new APIs to use in your mashups
  • SOAP API Developer's Guide