Event-Driven Architecture with Salesforce and MuleSoft Anypoint Platform
In my last blog post we looked at how to use MuleSoft Anypoint Platform for your Extract-Transform-Load (ETL) needs. Now we’re focusing on how to combine Anypoint Platform with the event-driven capabilities of the Salesforce Platform to build near-real time integrations with any system and device.
Event-driven capabilities within Salesforce
Looking back in history, event-driven architectures (also known as EDA) are not a new paradigm. In fact they have been around for a while in the forms of different implementations and tools. Particularly, because of the tremendous success of Apache Kafka, many companies started to embrace this kind of architecture. For Salesforce this has resulted in the implementation of new platform capabilities that are exposed to customers, namely platform events and Change Data Capture. My colleague Katia Hage wrote some great blog posts about Change Data Capture as well as where to use which kind of event.
Why use Anypoint Platform for event-driven applications
Data comes in different shapes and systems talk in different protocols. Connecting all these systems into an event-driven architecture poses many challenges. Mostly they are about additional development efforts that are necessary to make the different systems talk to each other.
Let’s look at an example: You have an on-premise SQL-based datastore that needs to get updated in near-realtime with data changes within Salesforce. You would either have to
- Craft some Apex Triggers to call an API (if it exists) that then writes to the SQL database.
- Create an (on-premise) app that listens to platform events or change cata capture events, and then writes these changes to the SQL database.
If more parts of the business would need a similar integration, but just with another target datastore, you’d have to go through the same decisions and processes again. This kind of integration doesn’t scale as it’s not really re-usable.
For solving this challenge, Anypoint Platform offers a rich set of features that enable you to actually build out an event-driven architecture – without changing the code base of your existing applications. It can listen or publish to any of the Salesforce provided events, as well as connect to any other system.
So let’s see what an implementation would look like.
Setting up Change Data Capture
For this blog post we’re going to focus on change data capture events. In case you want to leverage platform events, I recommend to check out the capabilities by walking through the Platform Events Basics module on Trailhead. For the purposes of this post, you only need to know that there is no technical difference in terms how you would set up Anypoint Platform for consuming Change Data Capture or platform events.
As you see, enabling Change Data Capture is as simple as selecting the object(s) that you want to enable for it within Setup. Right after it is enabled, any CRUD changes to the selected object type(s) are getting published and any subscriber can now pick up these changes. So let’s configure that for Anypoint Platform.
Connecting Anypoint Platform to Change Data Capture
As in the ETL blog post, we’re going to use Anypoint flow designer. But instead of creating a new Mule Application that runs on a schedule, we’re using the Salesforce Connector as starting point. For connecting to Change Data Capture or platform events we have to choose the Replay Channel operation.
After you’ve created the initial configuration, you need to specify which channel the Salesforce Connector should listen to. Change Data Capture offers a variety of options, like listening to all change events for all enabled objects, or to only listen to specific objects. We only want to get notified about change events for Account records.
It’s important to set the Replay Option to the value of -1. The connector manages and stores the replay IDs of the events. In case you restart the Mule Application, the connector can — with that setting — pick up any messages since its last run.
In case you want to use platform events you would specify the related URI, like
Working with events
After the connector is set up and the Mule Application is running, you can directly experience how it works. It’s as simple as changing a record value in Salesforce.
Note the green line in the flow. This is another cool feature of flow designer. It shows you the progress and success of data flows in real-time, and you can explore the actual payload for a change data capture event in the Output tab.
The JSON data that you receive from Change Data Capture can be used within your flow as needed. For example you can use the choice connector to execute different actions, like receiving an update vs. a created event for an Account record.
And with the many connectors that are available on Anypoint Exchange, you can then integrate your Account change data capture events from Salesforce with any available system — whether that is posting to a third-party API, storing data directly in a database, or somewhere else.
Anypoint Platform offers a variety of options to connect your Salesforce events with any system. That way you can start building out an event-driven architecture without increasing your code base. That reduces your time-to-market, as well as any future development-related maintenance. And if you have external event-driven applications, you can leverage other connectors, like the Kafka connector, to ingest that data into your Salesforce org.
About the author
René Winkelmeyer works as Principal Developer Evangelist at Salesforce. He focuses on enterprise integrations, Lightning, and all the other cool stuff that you can do with the Salesforce Platform. You can follow him on Twitter @muenzpraeger.