Introducing Lightning Connect Custom Adapters

Lightning Connect came out in Winter ’15 and introduced a new way to integrate Salesforce with external data. It allows you to create a new kind of object that provides a live view of data residing in external systems. Until now, you were limited to integrating with OData web services, forcing you to build or buy OData producers hosted outside of Salesforce.

Summer ’15 is rolling out an exciting new feature that lets you create your own Lightning Connect adapters in Apex. Now, with the Apex Connector Framework, you can integrate with anything—well, anything that we can get to from Salesforce by using HTTP callouts. The Apex code runs within Salesforce, enabling you to develop seamless integration solutions entirely within the Force.com platform.

DataSource.DataSourceProvider

It’s quite simple to create your custom adapter for Lightning Connect. First, create a new Apex class that extends DataSource.Provider. This class is interrogated by the system to learn the set of capabilities that the external system support. It is an abstract class with the following methods:

List<Capability> getCapabilities();
List<AuthenticationCapability> getAuthenticationCapabilities();
Connection getConnection(ConnectionParams connectionsParams);

Once you implement these methods, you can create an external data source. When you do so from Setup, you will see a new Type option that displays the name of your Apex Provider class, listed alongside the standard Lightning Connect adapters that Salesforce provides (an OData v2 and Salesforce adapter).

getCapabilities

The initial release of the Apex Connector Framework offers a rather small list of capabilities.

  • ROW_QUERY indicates that you can execute SOQL or view the external data in list views and detail pages.
  • SEARCH indicates that this external data source should be used in Salesforce global searches and SOSL queries.
  • REQUIRE_ENDPOINT indicates that a field to supply a URL should appear on the External Data Source setup UI.

getAuthenticationCapabilities

Your Provider class needs to declare a set of authentication capabilities. These are the types of authentication you support: ANONYMOUS (no authentication), BASIC (for Basic authentication), OAUTH, and CERTIFICATE. The declared authentication capabilities determine what options appear in the Authentication Protocol field on the External Data Source page in Setup.

getConnection

Lastly, you need to instantiate a Connection class, which we’ll define below. The getConnection method is called every time you execute a SOQL or SOSL query. You shouldn’t do anything expensive, such as callouts, while instantiating the Connection class.

The getConnection method receives connection parameters as an argument (these are supplied by the system). The connection parameters come from whatever the admin configures on the External Data Source page in Setup.

You can pass the connection parameters on to the Connection class. Note that anyone who can view debug logs can see the heap allocations for the object being instantiated and passed into the Connection class. For better security, I recommend that you use named credentials instead of supplying the credentials on the External Data Source page in Setup.

The following Provider class doesn’t pass along any credentials because authentication isn’t required in this example.

Example

You will need to save the Connection class, defined below, before this class can be saved.

global class LoopbackDataSourceProvider extends DataSource.Provider {
      override global List getAuthenticationCapabilities() {
          List capabilities = new List();
          capabilities.add(DataSource.AuthenticationCapability.ANONYMOUS);
          return capabilities;
      }
      override global List getCapabilities() {
          List capabilities = new List();
          capabilities.add(DataSource.Capability.ROW_QUERY);
          capabilities.add(DataSource.Capability.SEARCH);
          return capabilities;
      }
      override global DataSource.Connection getConnection(DataSource.ConnectionParams connectionParams) {
          return new LoopbackDataSourceConnection();
      }
}

DataSource.DataSourceConnection

The Connection class is where the real work happens. (The Provider class just takes all the credit.) Your Connection class needs to extend DataSource.Connection, which is an abstract class that requires you to declare the following methods:

List<DataSource.Table> sync();
DataSource.TableResult query(DataSource.QueryContext c);
List<DataSource.TableResult> search(DataSource.SearchContext c);

sync

The sync method is called when you click the Validate and Sync button on the External Data Source page in Setup. The method returns the table schema as a DataSource.Table object that your Lightning Connect adapter can handle. An organization that uses your custom adapter might not create an external object for every single DataSource.Table. An organization might also have more than one external object backed by the same external DataSource.Table (though there aren’t many reasons to do so).

Each DataSource.Table declares a set of DataSource.Column objects that correspond to the columns on the external data store. If the DataSource.Table is synced, the columns become fields on the external object in Salesforce. Each column has a data type and a name.

query

The query method is called when users execute SOQL queries, and when the system executes SOQL queries as users browse external object list views and detail pages. The method receives an object that represents the query context, and that has the following properties:

    
global List metadata {get;set;} 
global Integer offset {get;set;} 
global Integer maxResults {get;set;} 
global TableSelection tableSelection {get;set;}

The TableSelection object has properties that describe the table we are querying against, the columns we want to return, the criteria to determine what rows we should receive, and how we want the results ordered.

/**
 * FROM Clause - name of table
 **/
global String tableSelected {get; set;}

/**
 * SELECT Clause
 **/
global List columnsSelected {get; set;}

/**
 * WHERE Clause
 **/
global Filter filter {get; set;} 

/**
 * ORDER BY Clause
 **/
global List order {get; set;}

The Filter object is the WHERE clause from the query.

For a very simple comparative filter (such as “meaningOfLife = 42”), only the tableName, type (EQUALS), columnName (meaningOfLife), and columnValue (42) will be set. As a developer, you would be expected to ensure that you return only the rows that match this criteria.

We can also group filters. This will happen if the SOQL has AND, NOT, or OR statements. In this case, the type will be AND_, NOT_, or OR_, and the subfilters property will be set to a List of child Filters.

/**
 * EQUALS, NOT_EQUALS, LESS_THAN, GREATER_THAN,
 * LESS_THAN_OR_EQUAL_TO, GREATER_THAN_OR_EQUAL_TO,
 * STARTS_WITH, ENDS_WITH, CONTAINS, LIKE_
 * NOT_, AND_, OR_
 **/
global DataSource.FilterType type {get;set;}

/**
 * List of subfilters for compound filter types (NOT_, AND_, OR_)
 **/
global List subfilters {get;set;}

/**
 * For simple comparative filter types, the table name where the column resides
 **/
global String tableName {get;set;}

/**
 * For simple comparative filter types, the name of the column that is being compared.
 **/
global String columnName {get;set;}

/**
 * For simple comparative filter types, the value we use to evaluate against each record
 * in the data set in the table we are querying against
 **/
global Object columnValue {get;set;}

As a developer, it would be your job to acquire the data from wherever it comes from and provide the list of records that should be returned for the query. The records are returned as a List<Map<String,Object>>; the List is the collection of records, and each record is a Map where the key is one of the selected columns, and the Object is the value for that column on that record). The data can be sourced from a Salesforce organization, a CSV or other file, a callout response from a remote web service, or even generated entirely in Apex.

search

The search method is executed when you run a SOSL query or use the global search in Salesforce.

For simplicity, in the example below, we convert the search to a query that filters rows on the object’s name field. (External objects don’t have a standard Name field. Instead, you can designate any single text field as the name field for an external object.) The example isn’t complete, but we will explore search further in future blogs.

Example

That’s a lot of information for now. Let’s see some code in action with a sample custom adapter that handles some basic queries. The example will source its data by executing SOQL against the Account standard object. While that isn’t very practical, it illustrates how to handle filters.

This sample will not work with SELECT COUNT() queries, which is a requirement for this to work in Salesforce1, nor will it support ORDER BY clauses. It doesn’t support queries that use the CONTAINS clause, so this object doesn’t work with Global Search.

We’ll enhance this example to cover those cases in subsequent blog posts.

 global class LoopbackDataSourceConnection extends DataSource.Connection {
      global LoopbackDataSourceConnection(DataSource.ConnectionParams connectionParams) {
      }

      global LoopbackDataSourceConnection() {}

      override global List sync() {
          List tables = new List();        
          List columns;
          columns = new List();

          // Always declare these two fields.
          columns.add(DataSource.Column.text('ExternalId', 255));
          columns.add(DataSource.Column.url('DisplayUrl'));

          // These are custom fields for our external object.
          // 'Name' and 'NumberOfEmployees' are our internal names within this
          // Apex class. It doesn’t have to correspond to what the columns are called anywhere
          // else, but for convenience, we keep it the same as the columns in the Account
          // object we read from.
          // They will be exposed as Name__c and NumberOfEmployees__c in the external
          // objects that are created when synced from this adapter.
          columns.add(DataSource.Column.text('Name', 255));
          columns.add(DataSource.Column.number('NumberOfEmployees', 18, 0));
          tables.add(DataSource.Table.get('Looper', 'Name', columns));
          return tables;
      }

      // This example handles only simple SOQL. It doesn’t process
      // LIMIT, OFFSET, or ORDER BY clauses, and it doesn’t handle
      // COUNT() queries.
      override global DataSource.TableResult query(DataSource.QueryContext c) {
          List<Map<String,Object>> rows = execQuery(getSoqlQuery(c));
          return DataSource.TableResult.get(c,rows);
      }

      override global List search(DataSource.SearchContext c) {        
          return DataSource.SearchUtils.searchByName(c, this);
      }

      private List<Map<String,Object>> execQuery(string soqlQuery) {
          List objs = Database.query(soqlQuery);
          List<Map<String,Object>> rows = new List<Map<String,Object>>();
          for (Account obj : objs) {
            Map<String,Object> row = new Map<String,Object>();
            row.put('Name', obj.Name);
            row.put('NumberOfEmployees', obj.NumberOfEmployees);
            row.put('ExternalId', obj.Id);
            row.put('DisplayUrl', URL.getSalesforceBaseUrl().toExternalForm() + obj.Id);
            rows.add(row);
          }
          return rows;
      }

      private string getSoqlQuery(DataSource.QueryContext c) {
          string baseQuery = 'SELECT Id,Name,NumberOfEmployees FROM Account';
          string filter = getSoqlFilter('', c.tableSelection.filter);
          if (filter.length() > 0)
          	return baseQuery + ' WHERE ' + filter;
          return baseQuery;
      }

      private string getSoqlFilter(string query, DataSource.Filter filter) {
          if (filter == null) {
          	return query;
          }
          string append;
          DataSource.FilterType type = filter.type;
          List<Map<String,Object>> retainedRows = new List<Map<String,Object>>();
          if (type == DataSource.FilterType.NOT_) {
          	DataSource.Filter subfilter = filter.subfilters.get(0);
          	append = getSoqlFilter('NOT', subfilter);
          } else if (type == DataSource.FilterType.AND_) {
          	append =  getSoqlFilterCompound('AND', filter.subfilters);
          } else if (type == DataSource.FilterType.OR_) {
          	append =  getSoqlFilterCompound('OR', filter.subfilters);
          } else {
          	append = getSoqlFilterExpression(filter);
          }
          return query + ' ' + append;
      }

      private string getSoqlFilterCompound(string op, List subfilters) {
          string expression = ' (';
          boolean first = true;
          for (DataSource.Filter subfilter : subfilters) {
            if (first)
                first = false;
            else
                expression += ' ' + op + ' ';
            expression += getSoqlFilter('', subfilter);
          }
          expression += ') ';
          return expression;
      }

      private string getSoqlFilterExpression(DataSource.Filter filter) {
          string columnName = filter.columnName;
          string op;
          object expectedValue = filter.columnValue;
          if (filter.type == DataSource.FilterType.EQUALS) {
          	op = '=';
          } else if (filter.type == DataSource.FilterType.NOT_EQUALS) {
          	op = '<>';
          } else if (filter.type == DataSource.FilterType.LESS_THAN) {
          	op = '<';           
          } else if (filter.type == DataSource.FilterType.GREATER_THAN) {           	
                op = '>';
          } else if (filter.type == DataSource.FilterType.LESS_THAN_OR_EQUAL_TO) {
                op = '<=';           
          } else if (filter.type == DataSource.FilterType.GREATER_THAN_OR_EQUAL_TO) {           	
                op = '>=';
          } else if (filter.type == DataSource.FilterType.STARTS_WITH) {
          	return mapColumnName(columnName) + ' LIKE \'' +
                    String.valueOf(expectedValue) + '%\'';
          } else if (filter.type == DataSource.FilterType.ENDS_WITH) {
          	return mapColumnName(columnName) + ' LIKE \'%' + 
                    String.valueOf(expectedValue) + '\'';
          } else {
          	throwException('Implementing other filter types is left as an exercise for the reader: ' + filter.type);
          }	
          return mapColumnName(columnName) + ' ' + op + ' ' + wrapValue(expectedValue);
      }

      // The standard fields ExternalId and DisplayUrl don’t exist in the
      // Account object that we are querying against, but we can generate
      // their values from the Account record ID.
      private string mapColumnName(string apexName) {
          if (apexName.equalsIgnoreCase('ExternalId'))
          	return 'Id';
          if (apexName.equalsIgnoreCase('DisplayUrl'))
          	return 'Id';
          return apexName;
      }

      // Put strings in quotes when generating SOQL queries.
      private string wrapValue(object foundValue) {
          if (foundValue instanceof string)
          	return '\'' + string.valueOf(foundValue) + '\'';
          return string.valueOf(foundValue);
      }
  }

And that’s it!  Now you can create a data source, sync it to make an external object called Looper__x, make a custom tab for that object, browse list and detail views in Salesforce, and execute SOQL against it using the REST and SOAP APIs. Instead of taking the data from the Account object on the same org, you could modify your code to take the data from another org, or replace the SOQL queries with REST callouts to an external web service. The possibilities are endless.

Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • Neal Hobert

    Hi Lawrence. I really liked this article. Thanks for publishing it.

    Is there any documentation on the various DataSource.Column types that are available. Your article uses Text, URL and Number but am curious what others are available and what the parameters are for other types.

    Thanks Neal

    • The official documentation should come out soon, but some of the other factory methods are:

      boolean(name)
      lookup(name, domain)
      externalLookup(name, domain)
      indirectLookup(name, domain, targetField)
      get(String name, String label, String description, Boolean isSortable, Boolean isFilterable, DataSource.DataType type, integer length, integer decimalPlaces, String referenceTo, String targetField)

      type is a DataSource.DataType instance (System.debug(DataSource.DataType.values()) for a list of values)

      or you can use an empty constructor (new DataSource.Column()) and set the properties (name, label, description, filterable, sortable, type, referenceTo, referenceTargetField, length, and decimalPlaces) by hand.

      • Neal Hobert

        Thanks Lawrence. This info is extremely helpful.

  • Dan Fowlie

    Great write up. What are the licensing requirements for using custom adapters?

    • It’s part of Lightning Connect, which is sold per external data source. Each data source can be of any of the following connector types: OData, Salesforce, or custom.

  • Enrico Murru

    Awesome article! I’m trying to replicate this on my own DE (summer 15 enabled) but it seems that the DataSource.Connection is not hyet available. Does anybody knows if Custom Adapters have to be enabled in some other ways (e.g. asking to support)?

    • Enrico Murru

      They’ve added this feature automatically in the latest patch

    • Ankush Somani

      @enricomurru:disqus I am not able to use DataSource.Connection classes by today also. How to enable this classes. anything i have to take care of. I am on way to create a new custom adapter. so at first step i am stucked.

      • Enrico Murru

        It just worked the same day I wrote le last comment. Have you tried creating a brand new developer org?

        • Ankush Somani

          Yes It worked . In lightning dev org. Thanks for reminding.

  • Sean Canny

    When I paste the above LoopBackDataSourceConnection class into Eclipse I’m getting errors on a lot of the List(s) – I don’t recognize the format – there are no types specified, e.g., I’m seeing List tables = new List(); when I would expect something like List tables = new List();