High-Level Replication Steps

Each change event contains a transaction key, which uniquely identifies the transaction that the change is part of. Each change event also contains a sequence number, which identifies the sequence of the change within a transaction. The sequence number is useful for operations that include multiple steps, such as lead conversion. We recommend that you replicate all the changes in one transaction as a single commit in your system. One approach is to buffer all changes related to a transaction and commit them all at once.

If you choose not to use a transaction-based replication process, your replicated data might not be complete if your subscription stops. For example, if your subscription stops in the middle of an event stream for one transaction, only part of the transaction’s changes are replicated in your system.

A transaction-based replication process involves these high-level steps.

1. In your subscribed client, allocate a transaction buffer for each transaction key. For example, create a map (Map<String, List<ChangeEvent>>) where the key is the transactionKey value.
2. Open a CometD subscription to the general /data/ChangeEvents channel that captures all events.
3. For each change event received over the channel, deserialize it and add it to the appropriate map entry for the transaction key.
4. When an event with an isTransactionEnd value of true is encountered, commit all changes from the map entry for the transaction key and purge the map entry.
5. If a gap event is encountered, make a Salesforce API call using the included record ID to retrieve the corresponding record and overwrite it in your system. A gap event is not strongly ordered, so you can buffer a number of gap events together and make a bulk API call to retrieve multiple records.