Event Deserialization Considerations

The Pub/Sub API delivers events in the raw Avro binary format without modifying the fields in the event message. As a result, some fields in the deserialized event aren’t readable when you print them for debugging, and they must be decoded before they can be processed in the subscriber. Also, the fields received in change data capture events with Pub/Sub API and Streaming API (CometD) have differences.

Change data capture events contain bitmap fields whose contents aren’t readable when printed, and they must be decoded for processing in the subscriber app logic. The Pub/Sub API delivers the events in the raw Avro binary format without converting these fields. The bitmap fields are:

changedFields

Contains the fields that were changed.

diffFields

Contains the fields whose values were sent as a data diff.

nulledFields

Contains the fields that were set to null.

When you print these bitmap fields, you get a list of hexadecimal values and not the field names. For example: 'changedFields': ['0x650004E0','4-0x10','7-0x06']

A bitmap field uses a bitmap, encoded as a hexadecimal string, to represent the fields. This method is more space efficient than using a list of field names. For example, a bit set to 1 indicates that the field at the corresponding position in the Avro schema was changed for changedFields.

A bitmap field is an array of strings. The first element of the array contains a bitmap for the individual non-compound fields. Compound fields are placed in additional elements of the array, with bitmaps indicating nested fields. The format for the additional array elements is '{ParentFieldPosition}-{NestedFieldBitmap}'. In the previous example,'4-0x10' represents a compound field. 4 indicates that the compound field is at the fourth position in the Avro schema and 0x10 is the bitmap for the nested fields it contains.

For nulledFields, if the only change when updating a record is to set the compound nested fields to null, the first element of the nulledFields array doesn’t contain the individual non-compound fields. In that case, the array contains one entry, which is the compound field specification in the '{ParentFieldPosition}-{NestedFieldBitmap}' format. For example, the array value can be ['4-0x08'].

The fields referenced in the field bitmap respect the field permissions that the subscriber has. Any bits in the field bitmap for fields that the subscriber doesn’t have permission for are cleared to 0. For compound field bitmaps, '{ParentFieldPosition}-{NestedFieldBitmap}', the nested compound field bits aren’t cleared. Instead, the entry for the compound field is excluded from the field bitmap array. The reason is because field permissions are checked at the compound field level and not at the nested field level.

To decode the bitmap, match it against the fields in the event schema. You can get the event schema by calling the GetSchema RPC Method. The GitHub repository contains code examples that show how to decode bitmap fields. In the Java code example, EventParser.java expands the bitmap fields. CommonContext.java uses the getFieldListFromBitmap() method from EventParser to handle bitmap fields for change events. In the Python code example, ChangeEventHeaderUtility.py in Pub/Sub API Examples - Utility Code expands the bitmap fields. In the SalesforceListener.py example, the process_confirmation function shows how to get and convert the changedFields bitmap field by calling the process_bitmap function.

Byte fields in protocol buffer messages map to specific field types in various programming languages. See Scalar Value Types in the gRPC documentation. For example, bitmap fields in ChangeEventHeader are in bytes and so is the replay_id field. You can store byte values in a variable that holds bytes in your programming language.

If you use ByteBuffer in Java or JavaScript to store byte values for the bitmap fields in ChangeEventHeader or replay_id, reverse the bytes. ByteBuffer puts the bytes in big-endian order, but Pub/Sub API expects those fields to be in little-endian order.

After you deserialize a received platform event or change event in your client, Date and Date/Time fields are in Epoch time. As a result, when you print them, you get a number. For example: 'CreatedDate': 1632858587281 If you want to make the Date field readable for debugging purposes, convert the Epoch format into another date format. For more information, see Unix time.

Change data capture events that are delivered with the Pub/Sub API contain some fields that aren’t present on the change events received with Streaming API (CometD). This difference is because Streaming API uses JSON encoding for delivered events while Pub/Sub API uses the Avro binary format.

  • Change events received with Pub/Sub API contain all the record fields, including the unchanged fields. Unchanged fields have an empty value in the change event, for example, 'Description': None, even if they have a value in the Salesforce record. The changedFields field indicates which fields changed. The nulledFields field indicates which fields were set to null. In contrast, change events received with Streaming API (CometD) contain only the changed fields and exclude unchanged fields.
  • Change events received with Pub/Sub API contain header fields that aren’t included in Streaming API events. The header fields are diffFields and nulledFields. These fields are present on events received in Apex triggers. For more information, see Change Event Triggers in the Change Data Capture Developer Guide.

See Also