Developing Offline Apps with Salesforce Mobile Services

Mobile apps and offline access go hand in hand. If a sales rep has five minutes with a doctor and an iPad in the basement of a hospital, or a service rep needs to complete a mobile inspection report in a remote location, they might not have a strong data signal when they need it most. Worldwide cell coverage improves every year, but when it comes right down to it, if you’re on the move, your signal strength and data bandwidth will vary greatly from place to place. Apps are no longer just “online” or “offline” -- there’s a huge gray area in between where your device might technically be online, but the connection just isn’t usable enough for mission critical tasks.

The Salesforce Mobile Services does an excellent job of providing the tools needed to build enterprise mobile apps that allow you to securely transfer and store data on your mobile device for highly performant offline access. This post explores how to use the Salesforce Mobile SDK SmartStore to store encrypted data in a NoSQL-style database on both iOS and Android devices. The code shown throughout is written in Javascript, which would be used for hybrid mobile apps, but all of the same Mobile SDK functionality exists in Objective-c for iOS native apps and Java for Android native apps.

Offline SmartStore Figure1.jpg

The first thing we should talk about is how to architect a mobile app for online/offline use. One of the principles I focus on is that the app’s connectivity should be transparent to the user, and switching between online and offline should be seamless. This allows users to focus on using the app to do their job rather than mucking about with online / offline settings. To achieve this, the app should use locally stored data whenever possible because it will improve performance when signal strength is weak and bandwidth is low. As you can see in this flow, the app will first attempt to load records from SmartStore. If none exist, it will check if the device is online, and if it is, finally it will attempt to load the records from Salesforce.com.

This takes care of Reading data from Salesforce, but what if the user needs to be able to Create, Update, or Delete data? That’s where offline data queues come into play. Any time a record is upserted or deleted in the app, the local copy of that record is upserted or deleted and a queue record is created too. If the app is online, the record can be upserted or deleted in Salesforce.com immediately and the queue record can be deleted. Since the queue record isn’t deleted until a successful response is received from the Salesforce API, this ensures that no data will be lost regardless of whether the device is online, offline, or has a bad connection and a synchronization error occurred during data transfer. Since Salesforce can also calculate values for some fields by using formula fields, workflow rules, or triggers, it’s a good practice to query for any recently upserted records just to make sure the data is in sync across Salesforce and the mobile device.

Offline SmartStore Figure2.jpg

SmartStore Terminology

So what is this SmartStore, anyway? SmartStore is an encrypted NoSQL-style JSON document data store. In fact, it is the only cross-platform encrypted NoSQL mobile database on the market that works with both hybrid and native development models. But what does all that mean? “NoSQL” databases are a class of database that, unlike their “relational” brethren, are designed to be inherently schemaless. They take care of storing data, and let the application worry about how to structure it. Some typical benefits of this style of database are their raw speed, a dynamic schema where fields and tables can be added by the application at will, and ease of use for the developer. JSON (Javascript Object Notation) is a lightweight industry-standard way to encode data for transfer between systems. Some NoSQL databases are designed to be simple key/value stores, but more advanced systems like SmartStore allow for storing and indexing full JSON documents. This is important, because the Force.com REST API and Chatter API both return JSON documents from the Salesforce.com server. So, the code you wrote to process the data you’re getting from the REST API is the same code you’ll use to process the data you’re getting from SmartStore, and you don’t have to set up a schema offline. You’re already halfway done with your offline app, and you haven’t even started!

Let’s talk about some terminology -- with a typical relational database, data is stored in “tables”, where data is segmented across many tables in order to avoid things like duplicating data. With SmartStore, data is stored in “soups”. Soups are then stored in Stores, which are essentially separate database files on disk. Unless you have some strong need for multiple database files, most apps will have only one Store and any number of Soups. This terminology was first used with the Apple Newton, but it’s an apt naming system because SOQL queries can span multiple tables and thus return JSON documents that contain data from a soupy (get it?) mixture of different tables.

Thing you can do with your soup

Now that you know what SmartStore is, what can you do with it?

Register a Soup

You can register a new soup with an IndexSpec. An IndexSpec is a simple JSON document that tells SmartStore which fields need to be indexed for searching, upserting, deleting, and sorting. For instance, imagine you queried the Force.com REST API, and got back a response that looks like this:

Raw Response
HTTP/1.1 200 OK
Date: Fri, 05 Apr 2013 19:35:49 GMT
Content-Type: application/json;charset=UTF-8
Content-Encoding: gzip
Transfer-Encoding: chunked


{
  "totalSize" : 2,
  "done" : true,
  "records" : [ {
    "attributes" : {
      "type" : "Password__c",
      "url" : "/services/data/v27.0/sobjects/Password__c/a02d0000003H1scAAC"
    },
    "Id" : "a02d0000003H1scAAC",
    "Name" : "Reddit",
    "Username__c" : "tomgersic",
    "Password__c" : "notmypassword",
    "URL__c" : "http://www.reddit.com"
  }, {
    "attributes" : {
      "type" : "Password__c",
      "url" : "/services/data/v27.0/sobjects/Password__c/a02d0000003HFSaAAO"
    },
    "Id" : "a02d0000003HFSaAAO",
    "Name" : "Google",
    "Username__c" : "tomgersic",
    "Password__c" : "notmypassword",
    "URL__c" : "http://www.google.com"
  } ]
}

You can see we’ve gotten two records back from the API (“totalSize” is 2), and we have the Id, Name, Username__c, Password__c, and URL__c field for each record. Since you probably want to be able to query, upsert, and delete by the Id and Name fields, you need to create an IndexSpec that specifies that indexes be created for these two fields. Types can be specified as either “string” or “number”.

var indexSpec=[{"path":"Id","type":"string"},{"path":"Name","type":"string"}];

The indexSpec variable can now be used in the registerSoup method call to create the actual soup:

navigator.smartstore.registerSoup('Queue',indexSpec,success,error);

The first parameter of the registerSoup method is the name of the soup you want to create. The second parameter is the indexSpec. The last two are the success and error callback methods you want to use. I recommend using one common error method throughout an app so that error message formatting is consistent for the user.

Query a Soup

Now that you’ve created a Soup, let’s query it. Since this is a NoSQL database, it should be pretty obvious that we won’t be using SQL to query it. So, what will we be using? QuerySpecs!

You can choose from four different types of QuerySpec to query your data:

buildAllQuerySpec = function(path, order, pageSize)

This QuerySpec is used to query all records from the object in the given sort order (or null order). This is the one to use if you want to load all records from a Soup into app memory and work with it there. This isn’t the best option if you have a lot of data in a soup, though the pageSize parameter can help to split the results up into workable chunks of data if, for instance, you want to display 10 records per page in a listview and use an “infinite scrolling” component to quickly show more records as a user scrolls down the page.

buildExactQuerySpec = function(path, matchKey, pageSize)

Use this QuerySpec if you know exactly what record you want. The path parameter should specify one of the fields that you indexed int he IndexSpec. Given the example above, you would want to specify “Id” or “Name”, and then for the matchKey parameter, the Id or Name you’re looking for.

buildRangeQuerySpec = function(path, beginKey, endKey, order, pageSize)

This QuerySpec is similar to the ExactQuerySpec, but it allows you to specify a range of values with the beginKey and endKey parameters. For instance, if you wanted to query all Names between “Adams” and “Zaphod”, you would want to use this QuerySpec.

buildLikeQuerySpec = function(path, likeKey, order, pageSize)

If you’re familiar with SQL, this QuerySpec shouldn’t be too difficult to figure out. The query is similar to the LIKE clause, and allows you to search for a substring within string indexes. This is useful if a user is entering a search string because you can return, for instance, all names starting with “Arthur.”

Upsert Data

You Created and Update data with the upsertSoupEntries method. This is pretty easy to use. You just need to specify which soup you want to write to, and pass in a JSON document for the entries parameter. It will be automatically indexed using the indexes you specified when you registered the soup earlier.

upsertSoupEntries = function (soupName, entries, successCB, errorCB)

Delete Data

No database would be complete without some way to delete data, and SmartStore provides the removeFromSoup method for just that reason. Specify the soup you want to delete data from, and the Ids you want to remove.

removeFromSoup = function (soupName, entryIds, successCB, errorCB) 

Mobile App Example

How can we put all of this into practice? Take a look at my HazyPassword app on GitHub. Starting from index.html, it walks you through everything you need to know to build a simple offline-capable hybrid password management app using the Salesforce Mobile SDK.

Offline SmartStore Figure4.jpg

Here’s an overview of the files within:

Index.html

About what you’d expect from something named index.html, this is the starting point for the app. As you can see within, I’m using JQuery Mobile for the UI. Pages in JQuery Mobile are defined with
tags, and here we have two pages defined: home shows a list of passwords and edit shows the editable detail view of a Password record when one is selected.

hazypassword.js

The code in here is just for some setup work. It could have been included in the index.html file, but I chose to put it in a separate file to keep things cleaner. Inside, we instantiate a new instance of the Password class that we’ll talk about in a moment, push any queued records left over from a previous session to SFDC using the OfflineQueue class, and set up some event listeners for page change operations.

Password.js

Manages the handling of Password records, both in Salesforce, and in Smartstore. You’ll notice that the loadRecords method checks to see if the Soup exists (as shown in the Flow above), and if it does, loads records from it. If it doesn’t, SFDC is queried instead, and the records are stored in Smartstore using OfflineQueue.StoreRecords prior populating the listview. This class also includes methods to register the Password Soup to begin with, and upsert records changed by the user in the UI.

OfflineQueue.js

This class handles the queue functions that are written to any time a record is saved using OfflineQueue.StoreRecords. The OfflineQueue.UploadQueue method takes care of emptying the queue and upserting the records to SFDC. Delete methods are left as an exercise for the reader.

About the Author

Tom Gersic works as Enterprise Mobility Technical Director for Salesforce.com Services. He leads a delivery team that architects and develops commercial and enterprise deployments of mobile apps for many Salesforce.com customers. He is a frequent speaker at Salesforce.com events such as Dreamforce and Cloudforce, and his work has been featured in keynotes at these events. Tom has also developed simulation software for the US Naval Postgraduate School, the US Army, the Department of Homeland Security, and FEMA, and has taught at Northwestern University as a member of the adjunct faculty.

References