Token Streaming

Feed partial responses in real time as they become available, engaging users immediately and allowing them to consume content sooner. This feature is available from CapabilitiesVersion 63. Streaming tokens have 2 sets of payloads: a Messaging streaming tokens payload and a streaming tokens validation payload.

Message streaming tokens have to be grouped and ordered in the client.

1. The id field contains a unique identifier (uuid) for each token.
2. All tokens in a streaming set have the same targetMessageIdentifier value. This field value is the same as the id value of the final message.
3. Each token in a streaming set has a unique, increasing integer value in the sequenceNumber field to denote the order of that particular token in the token set. The tokens in the streaming set can come out of order, and you use the sequenceNumber value to arrange the tokens in the right order.

Salesforce's trust layer can't validate for hallucinations until the entire response is complete. Although tokens start to stream, the StreamingTokenValidation signal notifies you when there's an issue like hallucination with the tokens streamed. You can remove the token streamed upon getting this signal. Another set of tokens will stream after the validation token, you can choose to display those tokens.