Lightning Connect is now available for distribution by the ISV partners. We have seen a lot of interest in Lightning Connect, so this is welcome news for the partner ecosystem. Lightning Connect is a very powerful enabling technology that brings to bear a whole new set of integration patterns which were either not possible or very cumbersome to accomplish before. This article will cover some of the potential benefits of Lightning Connect technology and describe how it can be packaged and included in your ISV application for distribution.
What is Lightning Connect?
Lightning Connect is a standards-based data virtualization and data federation technology that is part of force.com platform. It allows to render, view and operate on external data sources by reference while leveraging the power of the platform. Please see the resources below to learn more about the Lightning Connect:
Webinar Recording: Access External Data in Real-time with Lightning Connect
OData presentation by Pat Peterson from API Strategy conference last year
Benefits to ISV Partners
There are many potential benefits that Lightning Connect can bring to various ISV apps. Below, are just some of these benefits. As it always happens, our partner and developer community will certainly come up with many interesting ideas and uses for this technology that we may not have even envisioned at the outset.
Increase the Breadth and Scope of Available Data. There is a class of ISV apps on the AppExchange that specialize on delivering rich external data into Salesforce. Historically, the datasets that were integrated into the platform have been limited to a small number of objects and attributes typically around Account and Contact objects. The overhead of having to replicate the data, maintain the data, deal with synchronization scenarios and storage consumption made it impractical to bring more data into the platform. Now, all these constraints can be avoided and data providers are able to make more datasets available to the platform users with little development effort.
Address Concerns About Losing Control over Proprietary Data. ISVs are sometimes concerned about replicating proprietary datasets to all of their subscribers. With Lighting Connect, the data is not copied but exposed to subscribers in real-time, by reference. An ISV can now retain much better visibility into how the data is used.
Create New and Flexible Distribution and Pricing Models. Ability to serve the data in real-time creates an opportunity to develop a lot more flexible and/or consumption-based pricing models. Lightning Connect provides ISVs with a real-time understanding of the number of and size of the queries and the real-time usage patterns across organization and at each individual user level. You can build comprehensive metrics and analytics and gain real intelligence on how the data is being consumed. This can help managing adoption and can open the possibility to offer new go-to-market, pricing and monetization models such as freemium or trial offerings. Data Providers can now expose parts of the entire breadth of their available data to customers and up-sell additional data sources or even additional attributes of the same table/object without friction or sales/technical overhead. There are scenarios to consider for how to tailor and segment the access to external data for any given customer in a very precise and flexible manner.
- Integrate with On-premise Data Sets and Address Data Residency Requirements. Lightning Connect can remove constraints for use cases where local data residency is important. A certain dataset can be hosted in a local data repository and brought into Salesforce on the fly. While the data will be passed through Salesforce, it is never persisted there. If data persistence outside of mandated geographical or legal domain is important, this integration pattern can combine the best of both worlds. The data stays local, yet it is fully available inside Salesforce for modeling the desired application behavior to benefit the end user.
Easily Interface with On-premise Systems. No app is an island and many of ISV apps have to integrate with various on-premise or hosted systems such as ERP or HCM. If you run into such integrations often, it may make sense to package a connector as part of your product or as an add-on and build additional product features tightly incorporating the external data from these connected sources into your offering.
Simplify Third-Party Service Integration. As the landscape of business applications has become more and more interconnected, we are seeing a trend of partners integrating more 3rd-party components and services into their applications. With data virtualization using Lightning Connect, such integrations can be much richer and easier to build and distribute.
Distributing Lightning Connect
Packaging Lightning Connect works much in the same way as all the other components available for packaging. If you need to refresh your knowledge of packaging ISV products on force.com, please review this section of the ISVforce Packaging Guide. To demonstrate the packaging process for Lightning Connect, I created a simple set of components that connect sample Northwind data source from OData.org website.
Description of the Sample Package
In my sample package, I linked three data tables from Northwind: Orders, Order Details, and Invoices. Orders and Invoices are top level objects and have corresponding Tabs. Order Detail is a second-level object and is a child of Order that I linked using an External Lookup Relationship Field. I also linked Order to the standard Account object by creating an Indirect Lookup Relationship to a custom field on Account designated as External Id. This is what the standard page layout looks like on a sample External Order record.
* An experienced ISV developer will probably raise a hand here and notice that including ExternalId fields on a standard object in a Managed Package is not a good practice. This is true, because limit of 3 ExternalIds per object is not namespaced and you do not want to face failing package installations if any given customer already has 3 ExternalIds fields defined. Do not do this in your production managed package. This is for illustration only.
I also added a custom app. You can navigate to it via the app picker.
All of the UI is, of course, available in the Lightning Experience both mobile, tablet and desktop.
I did not add a custom page layout for Account, but if you add a CustomerId field to your existing Account layout and add Orders related list, you can see how the external data can be linked to the standard Account object.
Populate the Customer Id field on any Account record with a value matching Customer Id field on any of the External Order records and you will see the related list with External Order populate on the fly.
With this very basic and simple setup, I get a lot of functionality out of the box. I can view the data in the standard Salesforce user interface (desktop as well as mobile), customize the page layouts using drag and drop editor, search external data from the global search, enable chatter and collaborate on records and leverage all the power of platform on this external data. There are some current limitations and considerations that you can review here.
To package your Lightning Connect metadata for distribution, go to your package main page, click Add Components and include External Data Source, External Objects and any other components you created. Of course, if you start adding components from the top level, the platform will take care of adding all the dependencies. In my case, I just added the custom app which contains tabs, external objects, and everything was pulled into the package automatically. Now that my package is ready, I uploaded it first as beta, tested and then changed to Managed Released making it ready for distribution.
Below, are the sample unmanaged metadata package as well as the managed package that I created. You can use the unmanaged version if you want to deploy in your own DE org and package for distribution with your own test namespace. You can download the managed package to test out the customer installation experience.
Unmanaged Package Example – download and install unpackaged version if you want to try packaging it yourself
Managed Package Example – install if you want to see the customer installation experience
* Please note that I enabled chatter feeds on some of the objects. If chatter is not enabled in your org, the package should still install successfully, but the feed will not be available. If Chatter is enabled, you will see the Chatter feed available on the Order and Invoice objects.
** Also, the target install org needs to have a Lightning Connect license. If it does not, the package should still install, but you will get an error message when attempting to view the external data. Any developer edition org will have Lightning Connect enabled, so you can spin up a regular DE or a Partner DE org to try it out if you do not have a test or sandbox instance with LC enabled.
Authentication and Access Control
In my example above, I used an unauthenticated OData provider. I did this because it makes it very easy for anyone to replicate and try out my sample application without having to worry about setting up an account with a data provider that requires authentication. Please keep in mind, however, that Lightning Connect supports two authenticated Identity Types – Named Principal and Per User and two authentication mechanisms – Username and Password or OAuth.
If you select Named Principal Identity Type, the subscriber organization will use one login account on the external system. This model can be used if you want to license access to your external data on a per org basis. It is also useful if you do not need to provide different levels of access to your service based on which user is accessing the data.
If you select Per User Identity Type, the subscriber organization will use multiple login accounts on the external system. Each of the users can have a unique set of credentials, or you can group the subscriber users—for example, by function or business unit—and have each group share a set of credentials. After the admin grants user access to the external data source through permission sets or profiles, users can set up and manage their own authentication settings for the external system. This authentication model provides flexibility for a very granular management and metering of how your subscribers interact with the external data source down to individual user level.
When you package the External Datasource, the authentication credentials will need to be entered by each subscriber after the package is installed. Please keep in mind that there is a post-installation step for your customers to complete. As always, make sure you fully test the end user experience for setting up and configuring your managed package post installation and make sure there are no unexpected and avoidable friction points. This is especially important if you decide to use more advanced access control and authentication methods such as Per User and OAuth. OAuth flows vary depending on the implementation and you will want to make sure there are no unnecessary configuration complexities for the subscriber admin and no unnecessary redirects in the authentication flow for the end users.
This article just scratches the surface of the capabilities of Lightning Connect. We have an exciting roadmap ahead of us. Below are just some of the things our R&D team has been working on:
Available in Summer ’15 is the ability to create custom adapters. If your external data source does not support OData, fear not. As long as you can expose it via HTTP (REST or even SOAP) which can be accessed by an Apex callout, you can create a custom adapter and make the data accessible as an External Data Source. See this blog post by Lawrence McAlpin with more on Custom Adapters for Lightning Connect. Custom Adapters are enabled by default in new Developer or Partner Developer Edition org. See the documentation for the brand new Apex Namespace: DataSource if you want dig into the details.
Lightning Connect Read/Write is in Pilot in Summer ‘15. This functionality allows you to update external data right from inside Salesforce UI.
Targeted for Winter ‘16 is full CRUD settings support for External Objects, an ability for admins to define object and field level access control policy on External Objects similarly to how it is done for regular standard and custom objects.
Further ahead is the support for Workflow and Triggers for External Objects. When I heard about this one it took a few seconds to for me to believe this was even possible, but it will be. With the upcoming support for OData 4.0 which introduces protocol level support for asynchronous notification callbacks on resource change, Lightning Connect will allow to execute workflow and apex triggers in Salesforce when records in the external data source change state. I can’t even begin to imagine how many possibilities this design pattern creates.
- Support for Custom Report Types for External Objects is also on the roadmap.
Enjoy exploring Lightning Connect and please comment here or reach out on Partner Community to share your feedback, ideas or questions. Work with your ISV account team if you would like to add Lightning Connect to your product catalog and to be able to distribute it to customers as part of your offering.