Event Deserialization Considerations

The Pub/Sub API delivers events in the raw Avro binary format without modifying the fields in the event message. As a result, some fields in the deserialized event aren’t readable when you print them for debugging, and they must be decoded before they can be processed in the subscriber. Also, the fields received in change data capture events with Pub/Sub API and Streaming API (CometD) have differences.

Change data capture events contain bitmap fields whose contents aren’t readable when printed, and they must be decoded for processing in the subscriber app logic. The Pub/Sub API delivers the events in the raw Avro binary format without converting these fields. The bitmap fields are:

changedFields

Contains the fields that were changed.

diffFields

Contains the fields whose values were sent as a data diff.

nulledFields

Contains the fields that were set to null.

When you print these fields, you get a hexadecimal value and not the field names. For example: 'changedFields': ['0x650004E0']

A bitmap field uses a bitmap, encoded as a hexadecimal string, to represent the fields. This method is more space efficient than using a list of field names. For example, a bit set to 1 indicates that the field at the corresponding position in the Avro schema was changed for changedFields.

A bitmap field is an array of strings. The first element of the array contains a bitmap for the individual fields. Compound fields are placed in additional elements of the array, with bitmaps indicating nested fields. The format for the additional array elements is “{ParentFieldPosition}-{NestedFieldBitmap}”.

To decode the bitmap, match it against the fields in the schema. The GitHub repository contains a code example that shows how to decode bitmap fields in Python. See ChangeEventHeaderUtility.py in Pub/Sub API Examples - Utility Code. In the SalesforceListener.py example, the process_confirmation function shows how to get and convert the changedFields bitmap field by calling the process_bitmap function.

Byte fields in protocol buffer messages map to specific field types in various programming languages. See Scalar Value Types in the gRPC documentation. For example, bitmap fields in ChangeEventHeader are in bytes and so is the replay_id field. You can store byte values in a variable that holds bytes in your programming language.

After you deserialize a received platform event or change event in your client, Date and Date/Time fields are in Epoch time. As a result, when you print them, you get a number. For example: 'CreatedDate': 1632858587281 If you want to make the Date field readable for debugging purposes, convert the Epoch format into another date format. For more information, see Unix time.

Change data capture events that are delivered with the Pub/Sub API contain some fields that aren’t present on the change events received with Streaming API (CometD). This difference is because Streaming API uses JSON encoding for delivered events while Pub/Sub API uses the Avro binary format.

• Change events received with Pub/Sub API contain all the record fields, including the unchanged fields. Unchanged fields have an empty value in the change event, for example, 'Description': None, even if they have a value in the Salesforce record. The changedFields field indicates which fields changed. The nulledFields field indicates which fields were set to null. In contrast, change events received with Streaming API (CometD) contain only the changed fields and exclude unchanged fields.
• Change events received with Pub/Sub API contain header fields that aren’t included in Streaming API events. The header fields are: diffFields and nulledFields. These fields are present on events received in Apex triggers. For more information, see Change Event Triggers in the Change Data Capture Developer Guide.