There are a number of different APIs that developers can use to integrate with a Salesforce organization. But sometimes it’s difficult to know what each API is for and when to use it. To that end, I sat down with Pat Patterson, Principal Developer Evangelist here at Salesforce, to talk about the different APIs that are available.
In our discussion, we wanted to give you an overview of all the API options and also impart some information about when you would want to use each of them.
D: Thanks for joining me today.
P: You bet. I’m always happy to talk about development and integration with our data.
D: To start off, tell us in general about the Salesforce APIs and what developers can do with them.
P: Sure. To start with, we’ve got a series of APIs, about eight of them now. While they all achieve a similar purpose – in one sense – of accessing data from Force.com or Database.com, there are some key differences among them which are worth explaining. First, it’s important to note that all of these APIs operate identically across Force.com and Database.com.
SOAP API
P: Probably the most well-known API is the SOAP API. As its name implies, it uses SOAP as a wrapper for API operations. Probably the bulk of our API traffic is from the SOAP API because it’s been there a long time.
D: So this is the API that’s been around the longest?
P: Yes, although it’s evolved over the years, it’s still the “granddaddy” of the APIs. The SOAP API is typically used in the enterprise setting because you need tooling to create SOAP messages. While it’s possible to concatenate strings to build a SOAP envelope, it’s pretty laborious. So we tend to see people using tools like Visual Studio or Eclipse and either using those tools to import the WSDL, or just using one of our toolkits. For instance, we offer a Java toolkit called WSC whereby you just import a .jar file and it gives you a nice programming interface to create records, run queries, etc.
Metadata API
P: Historically, the next API is the Metadata API. Where the SOAP API is oriented around accessing data and manipulating records, the Metadata API’s focus – as its name implies – is metadata. So it gives you an API to manipulate layouts, Visualforce pages, Apex triggers and classes – pretty much everything you can do to your org from the browser interface you can also do from the Metadata API. Not just retrieving them either – tools such as the Force.com IDE use the Metadata API to make changes in your org.
D: That’s good to know. So it’s not just for querying information about your organization but also for updating or creating objects?
P: Right. You have full create, read, update and delete with the metadata API. And this is another SOAP-based API so, again, you’ll need a library on the client side or you can use the tooling to import the WSDL.
Apex API
P: Now, the Metadata API allows you to upload Apex classes, but tools such as the Force.com IDE need a bit more functionality. The Apex API exposes API objects and calls that allow you to execute an anonymous block of code, compile classes, and run tests. Most developers won’t ever need to use the Apex API directly, but many benefit from its use in the IDE.
D: This is another SOAP API, right?
P: Right – you would download the WSDL to get started.
Apex SOAP
P: Staying with the SOAP theme, there’s a third API, which isn’t strictly speaking an API, that enables you to expose Apex methods via SOAP.
So you might write your own Apex method that takes parameters of a person name, company name and phone number and then creates an account and a contact in one operation as far as the client is concerned. You can then expose that method as a service operation by using the @WebService annotation on the method. So you can expose it via SOAP and download the WSDL and that all works very well and people have been using it for a long time.
But one of the qualities – although some might call it a drawback – of SOAP is that it’s pretty heavyweight. It was designed for a world in which people imagined that we would be taking these messages and passing them through intermediary systems.
REST
P: Now, if you just need to make a point-to-point call, in some ways SOAP can be overkill. You need an envelope and headers and a whole spec that explains how to process these messages. But if you’re just sending a message from a client to a server and sending back a response, you can use something much simpler which is REST.
REST stands for Representational State Transfer, and it takes a much simpler view of operations.
D: I understand you have recommended reading on this subject?
P: Yes! The original paper on REST by Roy Fielding is required reading for anyone working with this technology.
The idea here is that every object in the system is associated with a URL. When you perform operations on that URL, you’re accessing a representation of the state of that object. So if you do a GET on a URL representing an account record, you just get back XML or JSON that is the state of that account. You can use POST to create a new account, DELETE to delete that account. There’s a relatively new addition to the HTTP methods called PATCH that you can use to update that account. What we’re doing is passing XML or JSON back and forth and leveraging HTTP operations. This makes things much simpler for the programmer.
SOAP messages are all formatted as XML with angle brackets and tags and so on. JSON – which stands for JavaScript Object Notation – is a much more concise data representation than XML. In JSON, the data is represented as objects defined in JavaScript. You have property names, property values, and you can have nested objects and arrays.
For example, if you use the REST API to do a query which would be a GET on a URL that ends in “query?q=soql query” you get back an array of records formatted as JSON.
The nice thing about JSON is that it’s a bit more concise and lightweight, and there’s less data on the wire so it can be a bit faster. But from the point of view of the programmer, if you’re working in JavaScript or Java or PHP or any modern language, there are good, high-performance libraries available for parsing JSON. It’s very easy to work with. So you don’t have to download a WSDL that’s particular to the Salesforce API. You can just get the name of a single account.
As I mentioned, a really nice thing about using URLs is when you have a foreign key relationship. For example, if you have an account and a contact, if you retrieve just the contact, it will contain the URL to the account. Then you only need to do a GET on that URL to retrieve the account. You can navigate around the object hierarchy very efficiently.
REST API
D: For REST though, the programmer does need to know what objects are available to query, correct?
P: Yes, absolutely. So before I get to that, I’ll just introduce the REST API. The REST API went into early access about a year ago, and has been generally available (GA) since Spring ’11, and it does encompass some metadata operations. But in the REST API the metadata is read only. So a developer can perform a similar operation from REST, for example, you could retrieve a list of all the objects that are available in the system. Then for a given object, you could retrieve the list of its fields. So you can actually build dynamic interfaces with the REST API.
The REST API really shines for things like JavaScript where you have JavaScript running in the browser and you’re manipulating low numbers of records and you have a rich interface.
For example, I wrote a small sample that lets you query accounts, and it does a describe operation on the Account object to get back the list of fields on the object. Then it populates a dropdown with all the string fields so that you can filter on any of those fields. The program doesn’t know in advance that the Account object has fields like Name and Website and so on. So the REST API proper really brings together read-only metadata plus create, read, update and delete on the records themselves.
Typically, the REST API operates on smaller numbers of records. You can GET a single record using its URL and you can also run a query and bring back a set of records that match that query.
D: Right, and it’s synchronous, so that’s a consideration.
P: Right, you wouldn’t want to send a request that returns a lot of data or you’ll just timeout.
Bulk API
P: This brings us to the Bulk API which has been around longer than the REST API. It’s an asynchronous API, and it has the ability to work with large data sets like 500,000 records. But you wouldn’t want to use this API if you just need a handful of records and you want to create a dynamic UI that’s easy to program. It’s more of a special purpose API for loading lots of data.
D: With the Bulk API, does the developer need to know about the objects in their organization? It seems they would because if you’re using it to load data, you would have to format the data based on your schema.
P: Right, the data set that you send has to match the schema. So you would know what objects you’re loading data into or you might use another API to get the schema.
Apex REST
P: Rounding out that basic set of APIs is Apex REST. In the same way that you can write an Apex method and expose it via a SOAP interface, you can now have an Apex method exposed via REST. That’s really useful because one of the properties of the REST API is that you can do updates, but they’re on individual records. For example, let’s say you want to do an atomic update where you want to update an account and a contact and you want them to both succeed or both fail. It’s not possible to do that with the REST API. You would need to do a PATCH on the account and a PATCH on the contact.
D: So you can’t have a transaction?
P: Right, the programmer is in for a lot of pain if he wants to try and simulate that in the REST API. So what you would do in that case is define an Apex method that performed both of those updates and expose the method via REST. You would call the method, send the account and contact data, and then make the updates in that method. The way that Apex methods work is that if the method has one successful operation and the next operation results in an error, the method will roll back any changes that were made.
So you can define an Apex method that does multiple updates and it’s effectively one transaction. That’s a really, really useful thing when you want to work with more complex data scenarios other than just record-by-record.
D: So Apex REST and Apex SOAP are not strictly speaking APIs, but they’re ways that you can expose your Apex objects – create your own APIs, so to speak?
P: Exactly. They’re a mechanism for you to create your own API.
So far we’ve covered about six of the APIs. These are the APIs used in the classic data access scenarios like accessing data and metadata with various granularities: from individual records or handfuls of records in the REST API up to hundreds of thousands of records in the Bulk API.
Streaming API
P: The next API to discuss is the Streaming API, which is currently in pilot. Where the rest of the APIs give you the ability to make a call and you get back data or you update data, the Streaming API allows you to create a push topic based on a query and get updates on an ongoing basis when the results of that query change. So you can say ‘select name from some object’ and then subscribe to the topic channel.
In the absence of any changes, you get nothing – your app sits there and patiently waits. But as soon as someone creates or updates a record that matches your push topic query, you’ll get a notification that contains the fields that are in the SELECT statement of the topic query.
D: So the notification of the data change is being pushed to the client?
P: Exactly. You set up a subscription and you get updates when data changes. You can craft your SOQL query to match whatever constraints you like down to changes on an individual record. This is a tool that you can use to build very dynamic user interfaces.
Now, there’s really two ways you can use this API. The first is for user interfaces. Maybe you have a Web page that you want to update dynamically when data changes. So you can have lists that grow as new objects are created in the back end. We describe it as near real-time, so you’re going to get notification within about 5 seconds of the change happening. So you have a way of creating quite rich, dynamic interfaces.
The other way we see this API being used is for synchronization. So if you want to keep some legacy system in sync with your data in Salesforce, you can subscribe to changes and then you’ll be able to update your legacy system in near real-time. Rather than, say, running a batch job every night, you can get those changes pretty much as they happen.
D: And you don’t have to use up API calls by continually checking for data changes.
P: Yes. The Streaming API is still in developer preview, so we’re on the bleeding edge here a bit. But what people have done up until now is polling. You would run the same query say every 30 seconds, 10 seconds, whatever, and compare the results from query to query. But it’s very expensive in terms of API calls. The observations we’ve done show that about 70% of the time, there’s no change in the data. So you’re spending a lot of API calls and getting back something interesting about a third of the time. So the Streaming API doesn’t cost API calls in that way.
There are limits on the number of updates you can get in a day, but you’re not using up API calls in an ongoing basis as you are with polling.
Chatter API
P: The last, but not least, of the APIs is the Chatter REST API. Now this is interesting, because with the exception of the Apex API, all the APIs so far have been very general purpose. They apply to any object in the database. In fact, you can use all of these APIs to interact with Chatter if you know the underlying Chatter data model. And we do document it and the various standard objects in the system. But accessing Chatter in that way is a bit laborious for the developer because you’ve got to thread these objects together yourself. There are different objects for links and updates and files that you post to your feed.
D: And you’ve got to figure that all out yourself?
P: Yes, in addition it’s pretty costly in terms of round trips. You might have to make several round trips using the REST API or the SOAP API to assemble the data you want to show to the user.
So what the Chatter API does is present a higher-level abstraction of that data. Instead of dealing with the underlying objects, you’re dealing with a slightly abstract representation of a user’s Chatter feed and the entries in it. It’s designed so that you can do one call and get pretty much everything you need in the 80/20 case. You might need to drill down a little bit, but certainly most applications that are manipulating Chatter are making pretty standard requests. Like the Chatter desktop, for example. Your application might be showing all the posts on a user’s Chatter feed and with the Chatter API, you can get that all in one request.
D: That’s interesting. I can really see the integration possibilities, especially with social media.
P: It’s really up to the developer’s imagination what she wants to build. But this API is a huge time-save over manipulating the objects directly.
D: Just to confirm then, the Chatter API specifically exposes Chatter objects only?
P: Yes, it’s a special purpose API designed for Chatter. Basically, what’s happening behind the scenes when you make a call to the Chatter API is that the API implementation gathers together the data from the various objects and composes it into a response. So you can get the data in one round trip and get a much more performant application.
Summary
D: Very interesting. To wrap up, do you have any advice for developers that you’ve gleaned from working with customers implementing these various technologies?
P: I would say don’t work too hard – there’s sample code and libraries out there. If you keep up with the Developer Force blog and keep up with the articles, toolkits, and much more that we have on GitHub, you can save yourself a lot of work. There’s the Javascript REST toolkit and the PHP SOAP toolkit.
It’s like Abraham Lincoln said: if he had 8 hours to chop down a tree, he’d spend 6 hours sharpening his axe. It’s worth spending the time to get out there and learn about the tools. The investment of time is worth it.
D: Yes, and I think just this conversation will really help developers understand what the vision is for the different APIs.
P: Absolutely. The other thing I would say is to engage with the community. We have a great developer community out there and we have a great developer evangelism team that makes itself available.
If you run into a problem or see where things can be improved, let us know. We’re all on Twitter – I’m at @metadaddy – or, you can leave comments on the blog. You can also post your ideas on the IdeaExchange website. That’s how we improve by getting that feedback. If there’s a corner of one of the APIs that’s a bit clumsy or could be improved, tell us about it. We really do implement this stuff. You can go on the IdeaExchange website and see where ideas have been implemented in specific releases.
D: Yup, we listen!
P: We get our best ideas from customers. [laughs]
You’ll also find a lot of great supporting information on our developer documentation page. There are some good webinars available on YouTube talking about the various APIs. What’s good about the webinars is that there’s about 50 minutes of detailed discussion and demos and then the rest of the time is Q&A. So those can be a good resource.
D: Great. Anything else you want to mention?
P: [laughing] When in doubt, email Alex Toussaint [the Salesforce PM for the API team]. He’s the king of APIs. If you need help, he’ll sort you out.
D: [laughing] Right, I’ve seen that to be true. Pat, thanks so much for your time.
P: Thank you!