Today we released v1.1 of the Salesforce Mobile SDK. This latest version of the SDK has several enhancements, but the most significant change is the support for secure offline storage. You can now develop hybrid and native iOS and Android applications that (securely) store data on the device and allow the application to function even when the user is offline. This new feature of the Mobile SDK will help our customers and partners develop mobile applications that address complex use cases while still meeting the stringent security requirements that are typical of an Enterprise mobile application (e.g. what happens to my company’s sensitive data if the mobile device is lost or stolen?).

Lets dive into how you can use this new feature in mobile applications that are built using the SDK. To illustrate the offline feature, I’m going to use the same Cloud Tunes example that is part of the Mobile SDK Workbook. You can of course go through the workbook and build the Cloud Tunes application from scratch, or you can simply look at the application code on GitHub. As a quick refresher, the Cloud Tunes application lets users view records from a simple ‘Albums’ Custom Object in a classic ‘list view’ format. Users can then navigate to a list of child (Master-Detail relationship) Track records by clicking on any Album. We’ll be working on a hybrid version of this application (Tutorial 7 in the workbook) that uses Visualforce pages (plus JQuery Mobile) for the basic Album and Track list views and then ‘wraps’ these pages in the PhoneGap based SDK container to generate native Android or iOS applications. This hybrid approach is ideal for cases where you want to use web technologies like HTML5, JavaScript and CCS3 for your mobile development (vs Objective-C/iOS or Java/Android) but also want to access device features like the camera, address book, microphone etc.

So what would it take to support offline access for our Cloud Tunes application? Funny that  you should ask….

 

Setup

The Mobile SDK Workbook includes steps on how to download and install the Mobile SDK and so lets just focus on the setup of the new offline feature.

The hybrid Mobile SDK exposes device functionality (like camera, device storage etc.) as JavaScript and therefore the SDK’s offline function, aka ‘SmartStore’, is also implemented in JavaScript. Internally, the SDK maps the JavaScript calls to the appropriate device OS  (i.e. Android or iOS) APIs and features (like SQLite) to store your application data securely on the device.  In order to enable offline access in a hybrid mobile application you need to include a couple of JavaScript and CSS files in your Visualforce or HTML page. You can look at the cloudtunes Visualforce page to see an example of these includes, but here’s a brief rundown:

  • SFSmartStorePlugin.js –  contains the core implementation of the SDK’s offline functionality.
  • SFHybridApp.js – for useful utility methods (like determining if you’re offline or not).
  • SFHybridApp.css
  • You also need to include the JS file for PhoneGap since the hybrid SDK extends that library.

 

Initializing the offline database

SmartStore is inspired by the Apple Newton OS Soup/Store model. It also bears strong resemblance to NoSQL databases like CouchDB (e.g. the ability to store JSON values, simple key->value based storage etc.). Resisting the urge to think about the Seinfeld Soup Nazi episode, lets look at the SmartStore architecture.

You store your offline data in SmartStore in one or more ‘soups’. A soup, conceptually speaking, is a logical collection of data records—represented as JSON objects—that you want to store and query offline. In the Force.com world, a soup will typically map to a Standard or Custom Object that you wish to store offline, but that is not a hard and fast rule. You can store as many soups as you want in an application, but remember that soups are meant to be self-contained data sets; there is no direct correlation between them. In addition to storing the data itself, you can also specify indices that map to fields within the data, for greater ease and customization of data queries.

Lets take a look at the soup schema for the Cloud Tunes application to make this more tangible. You’ll find this code in the cloudtunes_smartstore.js file.

var ALBUMS_SOUP_NAME = "ct__albumsSoup";
var TRACKS_SOUP_NAME = "ct__tracksSoup";

function hasSmartstore() {
    if (PhoneGap.hasResource("smartstore") && navigator.smartstore) {
        return true;
    }
    return false;
}

function regOfflineSoups() {

    if (hasSmartstore()) {
        //Registering soup 1 for storing albums
        var indexesAlbums = [
            {path:"Name",type:"string"},
            {path:"Id",type:"string"}
        ];

        navigator.smartstore.registerSoup(ALBUMS_SOUP_NAME,
                                        indexesAlbums,
                                        onSuccessRegSoup,
                                        onErrorRegSoup);

        //Registering soup 2 for storing tracks
        var indexesTracks = [
            {path:"Name",type:"string"},
            {path:"Id",type:"string"},
            {path:"Album__c",type:"string"}
        ];
        navigator.smartstore.registerSoup(TRACKS_SOUP_NAME,
                                        indexesTracks,
                                        onSuccessRegSoup,
                                        onErrorRegSoup);
    }
}

The snippet above sets up two soups – one for storing Album records, and another for storing Track records. Lines 15 and 26 show how we’ve defined indices for the respective soups. You can have as many indices in a soup as you like, but they must all be defined when you create the soup.

An index consists of a ‘path’ and a ‘type’:

  • Path – Index paths map to the JSON field that should be used to index the soup data. Paths can be specified as compound hierarchies too, such as Account.Name, as long as the JSON records you save in the soup follow the same hierarchy.
  • Type – is the data type, currently only string and integer types are supported.

For the Album soup, we’ve chosen to index by ‘Name’ and ‘Id’ while for the Track soup, we’ve chosen to index on ‘Name’, ‘Id’ and ‘Album__c’. With these indices defined, we can later query the offline database (i.e. soup) on these specific fields. But more on that later.

Note the calls on line 20 and 31 to the ‘registerSoup’ function of SmartStore.  You need to register all the soups that your application will use to store offline data. As with other functions of the SmartStore library, ‘registerSoup’ requires success and error callback functions that get invoked by the SDK asynchronously.

 

Saving offline data

With the appropriate soup schema and associated indices defined, lets now look at how you can store offline data in these soups. The snippet below shows how the Cloud Tunes application stores Album records for offline use.

CloudtunesController.queryAlbums(function(records, e) {
    showAlbums(records, callback);
    addOfflineAlbums(records, onSuccess, onError);
}, {escape:true});

function addOfflineAlbums(entries, success, error) {
    if (hasSmartstore()) {
        navigator.smartstore.upsertSoupEntries(ALBUMS_SOUP_NAME,entries,
                                               success,
                                               error);
    }
}

The first line is a JS Remoting call to the Apex Controller to retrieve Album records. We then invoke the ‘upsertSoupEntries’ method of SmartStore (line 8 ) to save those records in the Album soup. Data records stored in a soup are simply JSON objects. The beauty of using JS Remoting is that the response is already in JSON format and so you don’t have to do any further manipulation to store the data in SmartStore. As the name implies, ‘upsertSoupEntries’ will insert or update the data records (i.e. JSON objects) based on whether or not they already exist in the offline database. Each new record that is added to a soup automatically gets assigned a unique ‘_soupEntryId’ (akin to the ‘Id’ field of a Force.com record). The SDK uses this field to determine whether a record should be inserted or updated during the invocation of ’upsertSoupEntries’.

 

Retrieving offline data

Next, let’s look at how you can retrieve data from the SmartStore offline database. This is typically done once you determine that the user is offline and you need the application to display previously saved offline data. The code snippet below shows how the Cloud Tunes application retrieves offline Album records.

function fetchOfflineAlbums(success, error) {
    if (hasSmartstore()) {
        var querySpec = navigator.smartstore.buildAllQuerySpec("Name", null, 20);

        navigator.smartstore.querySoup(ALBUMS_SOUP_NAME,querySpec,
                                       function(cursor) {
                                           success(onSuccessQuerySoup(cursor));},
                                       error);
    }
}

SmartStore supports various types of queries that you can perform against a soup. Each query type has a corresponding method – e.g. ‘buildAllQuerySpec’ on line 3. The ‘All’ query type returns all records in a particular soup in random order (equivalent to a ‘select name from Account’ SOQL query). You also need to pass in the index (e.g. ‘Name’) and page size (e.g. 20) to use with the query. In addition to the ‘All’ query, SmartStore has other query types that you can use to retrieve offline data. For example, here is an ‘Exact’ query to retrieve all Tracks for a specific Album record.

var querySpec = navigator.smartstore.buildExactQuerySpec("Album__c", albumId, 20);

The ‘querySoup’ method requires a callback function that is asynchronously invoked by SmartStore with a Cursor object that contains the result set. A Cursor provides a way to page through a potentially long list of query results. Your JavaScript code can move forward and back through the pages in the Cursor (using methods like ‘moveCursorToNextPage’, ‘moveCursorToPreviousPage’ ) to retrieve the entire result set. For a detailed example of how to navigate through a Cursor, you can look at the ‘onSuccessQuerySoup’ method in the cloudtunes_smartstore.js file.

In addition to ‘querySoup’, SmartStore also provides a ‘retrieveSoupEntries’ method to retrieve specific soup entries – i.e. those with a specific ‘_soupEntryId’ value.

 

SmartStore vs HTML5

The SmartStore functionality of the Mobile SDK provides a way for your application to store dynamic business data for offline use. What about application artifacts like the Visualforce markup, JS and CSS files, images etc.? How do you cache such static application resources on the mobile device so that they’re available in offline mode? For that you need a feature of HTML5 called Application Cache. Though the Application Cache has nothing to do with SmartStore or the Mobile SDK in general, I thought it was worth mentioning here since you’ll typically combine the SmartStore feature of the SDK with an Application Cache to implement a fully offline-enabled app. You can find more details about Application Cache and how you can use Visualforce to implement it in this blog post and the recently recorded Mobile SDK webinar.

Note also that HTML5 has other offline feature like local storage and Web SQL Database that could be used in lieu of SmartStore to persist offline data. The advantage of using the SDK’s SmartStore is that it stores all data securely on the device (using PhoneGap plugins to access native device features like SQLite) whereas HTML5 offline storage is not secure.

 

Implementing offline in native mobile apps

This blog post has been focused on how to use the SmartStore feature of the SDK in hybrid mobile applications. It is important to remember however that the JavaScript implementation of SmartStore is merely a wrapper around a core native (iOS or Android) implementation that persists data on the device. You can therefore also use the SmartStore functionality from a native Android or iOS application built using the Mobile SDK.

 

Hopefully this blog post provides you with enough information to implement offline functionality in your own mobile applications. The Mobile SDK team is constantly working on improving the SDK and adding additional functionality and so this offline feature is just a start. Its a social and mobile world out there and the Salesforce Mobile SDK is a mighty handy tool to have in such a world.

tagged , , , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • John Bowen

    Thanks for the blog. One question, is there a way to sync offline data back to Salesforce?

    • Anonymous

      Good question John. While the Offline feature of the Mobile SDK itself does not have any built-in sync capabilities, we fully expect our customers to implement one on top of the SDK to meet their specific sync requirements. As you know, the requirements for syncing offline data are very specific to the use case (e.g. ready only sync or read + edit sync) and so you can implement your custom sync logic on top of the Mobile SDK. As part of the sync. implementation, you would simply use our REST APIs to update data in Salesforce from the mobile device.

    • http://www.facebook.com/marcin.perlak Marcin Perłak

      John, syncing back and forth is in fact a complicated problem. Maybe a solution for you would be have a look at mobeelizer.com with native SDK for mobile platforms?

  • Ian Hulme

    Thanks for this it has helped a lot, although I do have one problem.
    I did have the CloudTunes_Offline example working, then for no real reason I start to get the error:
    Uncaught TypeError: Cannot read property ‘connection’ of undefined ———– SFHybridAppJs:109
    I have tried everything I can think of, including starting from scratch, but I still get the above error.
    Don’t suppose you can point me in the right direction.

    Many thanks,
    Ian

    • Anonymous

      Hi Ian,

      There’s probably some more comprehensive type checking that needs to happen here. Can I ask, are you getting this error when you’re running the app in a hybrid container, or in the browser?

      Thanks,
      Kevin

      • Ian Hulme

        Hi Kevin,

        Thanks for the quick response, I’ll download the code now and have a go.

        It seemed happened in both browser and ios hybrid container using the simulator.

        Will let you know how I get on.

        Thank,
        Ian

      • Ian Hulme

        Hi Kevin,

        Worked a treat in the browser and iOS Hybrid using the simulator.

        Thanks again,
        Ian

    • Anonymous

      I was able to reproduce this issue in a web browser (not the container—this is the behavior I would have expected, the way it was coded). I’ve uploaded a fix to the Samples repo, so if you reinstall the CloudTunes-force.zip package, the problem should (hopefully) go away.

      One note is that, given that the artifacts have been cached via HTML5 cache manifest, you may need to clear your cache to see the fix. For Firefox, you can go to https://developer.mozilla.org/en/Using_Application_Cache and jump to “Storage location and clearing the offline cache”, removing the cache files at the suggested location.

      Hope this helps,
      Kevin

  • Nigel Street

    Hi,

    I would like to use SmartStore as part of a native iOS app.

    I just did a fresh install of the Salesforce Mobile SDK for iOS and this is fine except I don’t see any reference to SmartStore in the Xcode project (I’ve searched and browsed). I can, however, see SmartStore in the source directories from where I ran the ./install.sh.

    Is there an extra step or parameter required to include SmartStore in the install into Xcode so it can be used in a native project?

    Thanks.

    • Anonymous

      Hi Nigel,

      We had primarily envisioned the SmartStore functionality being used by hybrid developers, with the idea that native developers have a hundred different ways to store data at their disposal. As such, the primary code base lives in the SalesforceHybridSDK framework project.

      However, in theory, there’s no reason you couldn’t use that framework in your native project, and leverage the SFSmartStore bits (located in SalesforceHybridSDK/Plugins/SFSmartStore) in your native code. We haven’t really played with this concept in native apps, ourselves.

      • Nigel Street

        Thanks for the clarification, that’s saved me some time, and is appreciated.

  • Ian Hulme

    Hi Again,

    I’m doing quite well on my app now, but I have come across a strange error that I just can’t seem to figure out.

    On Subsequent logins to the app, the manifest fails to load with the following error:

    Application Cache Error event: Manifest fetch failed (-1) https://c.na14.visual.force.com/apex/assManifest

    When I included the script from:
    http://www.html5rocks.com/en/tutorials/appcache/beginner/
    this is the function thats fired:
    // The manifest returns 404 or 410, the download failed,
    // or the manifest changed while the download was in progress.
    appCache.addEventListener(‘error’, handleCacheError, false);

    Once the cache is cleared and everything is downloaded again it all seems happy until another login is required.

    Your help with his problem is greatly appreciated as this is my final hurdle for the app to be properly demoed.

    Thank,
    Ian

  • Ian Hulme

    Also, this is setup on a free developer account, could this be causing problems?
    Thanks

    • Anonymous

      Ian – are you testing the iOS or Android hybrid version? Also, are you testing in the Emulator/Simulator, or an actual device?

      • Ian Hulme

        Hi Bhanots,

        Sorry for the late reply, been off on my holidays….
        I’m testing on iOS with the Simulator, as well as through a standard browser.

        The app is going to be offline, but there seems to be a problem once the session has been refreshed?

        Thanks for you help.
        Ian.

        • Anonymous

          Ian – I believe that the Mobile SDK team has made a couple of bug fixes to the Hybrid SDK in the recent weeks. Do you mind pulling the latest SDK code from GitHub and testing again to see if this problem has been addressed? Thanks
          Sandeep

  • Mahesh Anantharaman

    I am trying to use the sample application VFConnector. The application works well for sometime and after a while it prompts me for username and password even though the
    function loginSuccess(oauthCredentials)
    is called. It loadstheurl but salesforce prompts me with username and password.
    please clarify
    thanks
    regards
    mahesh

    • Anonymous

      Mahesh – how does this issue relate to the secure offline feature of the SDK? Are you just trying to get the default hybrid VFConnector sample app to work (with no offline reqs), and getting login issues?

  • http://profile.yahoo.com/AZLGDK37HJGSXCG6ULG4ICPMCQ Maurizio Capobianco

    Hi,

    Sorry if this is not the right Blog (if not please can you point me to the right one?) I have gone through the Tutorial to configure my cloud tunes Hybrid application to use the offline smart store database. Up to know I have not been able to use the offline functionality since I get the following error when starting the IPAD simulator on XCODE:
    ERROR: Plugin ‘com.salesforce.smartstore’ not found, or is not a PGPlugin. Check your plugin mapping in PhoneGap.plist.
    I tried to modify manually the PhoneGap.plist file without success. Still the SmartStore Plug in is not usable… can you please help me identifying which could be the issue? I think I followed all the steps in the tutorial (the only difference I found is that on the tutorial ppt is refers to a StartDate variable in the bootconfig.js, while I think it should be StartPage)

    Tx a lot

    Maurizio.

  • kishore

    Hi,
    Do we have Offline Data Save support for Native apps ?
    I tried to apply SFSmartStore to Navtive app, but i didnt found SFSmartStore.h in SDK.

    Thanks

  • Ian Hulme

    Take a look at the following video:
    //www.youtube.com/watch?v=ERIrKTussUw

    it gives you all the basic information to start using offline support in iOS, which is supposed to be the same (or similar) in android if your using that.

    Also came across this today which should be of use to people developing for iOS.
    //www.youtube.com/watch?v=PntLl4mWBX4

  • http://twitter.com/carloseiroa Carlos Eiroa

    Regarding the statement “HTML5 offline storage is not secure” it might be useful for users to refer to https://developers.google.com/web-toolkit/doc/latest/DevGuideHtml5Storage, where it says:

    “HTML5 local storage saves data unencrypted in string form in the regular browser cache. It is not secure storage. It should not be used for sensitive data, such as social security numbers, credit card numbers, logon credentials, and so forth.”

    I imagine we would not want to store Salesforce data like Account IDs, etc, either.