Platform Events and Change Data Capture events allow you to send custom notifications or monitor record changes in Salesforce and to communicate these changes to other systems. However, Platform Events or Changed Data Capture events are currently bound by a couple of important limits:
- 2,000 concurrent subscribers limit, which means only 2000 clients at a time can subscribe to Salesforce data changes
- Event delivery limits (allocation based on license type), which is the maximum number of event notifications that can be delivered in 24 hours. The key thing to note here is that this limit is shared by all CometD clients.
The above limits are in place because High Volume Platform Events (HVPE) were initially for system-to-system interactions. Over time, however, we have seen customers adopting HVPE for B2C apps.
In this blog post, we will discuss how developers can combine Salesforce and Heroku to efficiently leverage Platform Event delivery limits and still use Platform Events for B2C use cases and apps.
Understanding the Platform Events delivery limit
Let’s consider a simple example to help illustrate how one can easily hit these limits in the real world on a large-scale enterprise implementation.
Let’s imagine that a fictitious company, “Electric Motors,” wants to provide their 5m+ customer base with a live update on their support cases without requiring customers to refresh the page. The company uses Salesforce Experience Cloud to provide an online digital portal experience to its customers.
The Electric Motors development team has designed a custom Lightning Web Component for their Experience Cloud site that provides notifications to customers about their case status in real-time. To achieve this, developers leveraged Change Data Capture on the case object, and they used the base component lightning-emp-api.
However, during scale testing, developers found that they would quickly hit the allocated Platform Events delivery limits due to the company’s large volume of customers. This is known as a “fan-out” issue and is very common.
Here’s what’s happening. The lightning-emp-api component creates a unique CometD connection for every user session. In this scenario, if there are 5K active customers subscribing to events via the component, the Experience Cloud site would reach the 2000 concurrent subscribers limit and would error out for the rest of them. Also assuming that there are 10 notifications published, the site would consume 50K (5K*10) of the event delivery count, causing errors to all other processes that use Platform Events.
In the next section, we will explore how to use a fan-out messaging pattern that combines Salesforce and Heroku to efficiently use these limits.
Fan-out messaging pattern to improve events delivery limits
In a fan-out messaging pattern, a publisher pushes messages to some or all of its channels, so that various subscribers in separate channels can receive the same message at the same time.
You can achieve this pattern on the Salesforce Platform by combining Heroku and Salesforce as shown in the below diagram.
For the implementation shown above, we used a Node.js microservice on Heroku to create a connection to the Salesforce environment and a WebSocket server.
This approach comprises the following sequence of operations:
- A change in the Salesforce record (case status field change) is published using a Change Data Capture event.
- A Node.js service on Heroku subscribes to the Change Data Capture event. The service uses the OAuth 2.0 JWT Bearer Flow for authentication to Salesforce.
- A WebSocket server (Node.js service) hosted on Heroku broadcasts the messages with the relevant payload (comprised of Case ID and Case Status).
- A Lightning Web Component (LWC) on the Experience Cloud Site page subscribes to the WebSocket connection on Heroku to display the notifications.
Follow our step-by-step guide to set up the demo.
Show me the code
To establish the connection to the Salesforce environment from Heroku, we leveraged the OAuth 2.0 JWT Bearer Flow for authentication to Salesforce.
Connect Heroku and Salesforce
We used the open-source JSforce Node.js module and salesforce-jwt-bearer-token-flow Node module to authenticate. Note that one needs to create a Connected App in Salesforce to authorize the Heroku app to connect to Salesforce with the right OAuth Scopes.
Subscribe and listen to Change Data Capture
The JSforce module provides functions to subscribe to Change Data Capture or Platform Events. You will need to input the channel URL. The below code snippet shows how to create a streaming channel and subscribe to the Change Data Capture event.
Create a WebSocket server on Heroku
To create a WebSocket server on Heroku, we leveraged an open-source library called socket.io. The code snippet below shows how to use express server and WebSocket to send messages.
Subscribe to messages from a Heroku WebSocket connection using LWC
Lightning Web Components use the socket.io client-side library (loaded from the static resource as a third-party JavaScript library) to subscribe to the WebSocket connection. The code snippets below for the Lightning Web Component JavaScript controller show how this is achieved.
To learn more, get the complete sample code for the Node.js microservice. Note: this is a sample implementation to show you how to build a Node.js microservice with websocket.io. Be sure to follow WebSocket security best practices for your production application.
Filter your stream of Platform Events with channels (pilot)
The Platform Events product team has been working on a new feature that filters Platform Event streams with channels. It is currently in pilot and addresses the type of fan-out issues that we’ve been discussing here. With this new feature, customers can set up virtual channels using filter criteria and then subscribe to these channels to get a filtered list of all of their Platform Events, instead of all the events, thus reducing their event delivery limit consumption.
To access this feature, contact your account executive or reach out to Product Director Raj Advani via his Twitter handle: @RajSFDC.
Conclusion
The solution shared in this blog post enables anywhere from 10 thousand to millions of customers to receive events with the least possible consumption of event delivery limits. Instead of all subscribers counting as individual clients for Change Data Capture events, we now have one client, Heroku, that distributes the messages to all subscribers. This shows how you can combine the power of Heroku and Salesforce together to achieve scale.
References
Fanout Pattern Setup Document – Latest (JWT Auth)
Github repo for Node.js microservice for Platform Events subscription and WebSocket implementation
Github repo for LWC subscribing to Platform Events
About the authors
Leela Mohan is a Lead Developer at Salesforce. He has a decade of experience in product development and a strong ambition to learn, build solutions, and solve problems.
Sam Mittal has been in the IT industry for over two decades. He has worked as a Solution Architect and a top-tier Sales Engineer providing technical guidance to various C-level executives at Fortune 500 companies. Currently, Sam is CTO of Integration – AMER COMMERCIAL at Salesforce.
John VanSant is a veteran of enterprise systems with over 30 years of software architecture and development experience. As a Systems Engineer at MuleSoft (a Salesforce company), John helps customers understand how to best utilize MuleSoft’s Anypoint Platform to solve their business problems.
Mohith Shrivastava works as a Lead Developer Advocate at Salesforce. He is currently focusing on Slack and Salesforce Integration, Heroku, and Developer Tooling. You can follow him on Twitter @msrivastav13.