Pull Subscription and Flow Control

Unlike push-based subscriptions in which a client waits to receive new events pushed from the server, with a pull subscription, a client requests events from the server. To request events, a client calls the Subscribe RPC Method or the ManagedSubscribe RPC Method (Beta) and passes in a FetchRequest or ManagedFetchRequest, respectively. The client sets the number of events that it has capacity to process in the num_requested field of FetchRequest or ManagedFetchRequest. This event flow control ensures that the client doesn’t get overwhelmed with more events that it can handle if there’s a spike in event publishing.

A client can send one or more FetchRequests as part of the same Subscribe call, and it can receive one or more FetchResponses in return. Similarly, a client can send one or more ManagedFetchRequests as part of the same ManagedSubscribe call, and it can receive one or more ManagedFetchResponses in return.

The maximum number of events that can be requested in a Subscribe or ManagedSubscribe call across all requests is 100. If a client requests more than 100 events in a Subscribe or ManagedSubscribe call, the server considers the number of requested events to be 100. The client sets the number of events to request in the num_requested field of FetchRequest or ManagedFetchRequest. The server adjusts the total number of requested events based on the requested events sent in one or more requests and how many events were already delivered. If the client requests more events before the server finishes the last requested amount, the server appends the new amount to the current batch of events it still needs to deliver.

These sections describe ways to control event flow.

In this subscription flow, the client can request one event at a time. It waits for the event to be received, processes the event, and then requests a new event.

This diagram shows a basic sequence of requesting and processing one event at a time. The sequence is:

1. A client requests an event in a request to Salesforce.
2. When an event is available in the event bus, Salesforce delivers the event to the client.
3. After receiving the event, the client initiates a new request requesting one more event.
4. When a new event is available in the event bus, Salesforce delivers the event to the client.

The server can send the requested events in one or more responses. If there are enough events in the event bus, the requested events can be delivered in one response, or in multiple responses if the total size of events exceeds 3 MB.

If there aren’t enough events in the event bus, the client receives a partial number of events in one response. As new events are published to the event bus, they’re sent immediately to the client in a new response as long as there are still pending events to be delivered to the client.

The response contains the number of pending events not yet delivered in the pending_num_requested field. The client can use this number to decide how many new events to request, if it has the capacity to continue processing events. For example, a client’s processing capacity is 100 events and it requests 100 events. Then the client receives 60 events in a response with a pending_num_requested field value of 40. The client can request 60 events again, which corresponds to 100 minus the pending_num_requested value of 40.

In this subscription flow, the client initiates a request and waits until all requested events are delivered before it sends a new request to ask for more events. In this example, the client determines that it can process 100 events and sends a request for 100 events, which corresponds to the maximum number of events that can be requested from Salesforce. The client then waits for all events to be delivered before initiating a new request to ask for more events. The events can be delivered all at once, in one response, or in multiple responses. In this example, the events are delivered in multiple responses.

The sequence in this diagram is:

1. A client requests 100 events in a request to Salesforce.
2. As events become available in the event bus, Salesforce delivers the events to the client in multiple responses.
3. After receiving all the events, the client initiates a new request for 100 more events.

In this subscription flow, the client initiates a request, and while events are delivered, sends a new request to ask for more events. In this example, the client determines how many events to ask for in the next request based on the number of events received so far. The server delivers events as they become available in the event bus in multiple responses.

The sequence in this diagram is:

1. A client requests 100 events in a request to Salesforce.
2. The first response returns 60 events.
3. The client sends another request for 60 new events.
4. The requested events from the first request and the second request are delivered in multiple responses.
5. After receiving all the events, the client initiates a new request for 100 more events.

The Subscribe or ManagedSubscribe call stream stays open as long as there are pending events to be sent to the client. The number of pending events is based on the number of events the client requested and the number of events delivered.

If the server sends all requested events to the client, there are no more pending events to be delivered and pending_num_requested in response is equal to zero. In this case, to keep the subscription stream open, the client must send a new request within 60 seconds from when the last response was sent. If the client fails to do so, the subscription stream closes and the client must call Subscribe again to open a new stream.

If there are pending events to be sent to the client (pending_num_requested in FetchResponse or ManagedFetchResponse is greater than zero), but no new events are available in the event bus, the server keeps the stream alive by sending keepalive messages within 270 seconds. For more information, see Subscribe Keepalive Behavior and ManagedSubscribe Keepalive Behavior.

To process events from multiple topics in separate subscription streams, a client can make multiple Subscribe calls or multiple ManagedSubscribe calls for different managed subscription IDs. The number of events requested and the maximum requested value in a request are per Subscribe or ManagedSubscribe call. If you make multiple calls in one client, event flow is controlled separately for each subscription. For example, you can subscribe to one topic with a low batch size of 10 events and to another topic with a high batch size of 100 events.

To stay within your event allocations, you can use methods that reduce event consumption for one topic.

• Lower the batch size of requested events.
• Send requests less frequently.
• Pause a subscription by stopping it and making another Subscribe or ManagedSubscribe call.

The received event rate depends on your client implementation and whether you’re processing events from each subscription in parallel or sequentially.