Pull Subscription and Flow Control

Unlike push-based subscriptions in which a client waits to receive new events pushed from the server, with a pull subscription, a client requests events from the server. To request events, a client calls the Subscribe RPC Method and passes in a FetchRequest. The client sets the number of events that it has capacity to process in the num_requested field of FetchRequest. This event flow control ensures that the client doesn’t get overwhelmed with more events that it can handle if there is a spike in event publishing.

A client can send one or more FetchRequests as part of the same Subscribe call, and it can receive one or more FetchResponses in return.

The maximum number of events that can be requested in a Subscribe call across all FetchRequests is 100. If a client requests more than 100 events in a Subscribe call, the server considers the number of requested events to be 100. The client sets the number of events to request in the num_requested field of the FetchRequest. The server adjusts the total number of requested events based on the requested events sent in one or more FetchRequests and how many events were already delivered. If the client requests more events before the server finishes the last requested amount, the server appends the new amount to the current batch of events it still needs to deliver.

These sections describe ways to control event flow.

In this subscription flow, the client can request one event at a time. It waits for the event to be received, processes the event, and then requests a new event.

This diagram shows a basic sequence of requesting and processing one event at a time. The sequence is:

1. Client requests an event in a FetchRequest to Salesforce.
2. When an event is available in the event bus, Salesforce delivers the event to the client.
3. After receiving the event, the client initiates a new FetchRequest requesting one more event.
4. When a new event is available in the event bus, Salesforce delivers the event to the client.

The server can send the requested events in one or more FetchResponses. If there are enough events in the event bus, the requested events can be delivered in one FetchResponse, or in multiple FetchResponses if the total size of events exceeds 3 MB.

If there aren’t enough events in the event bus, the client receives a partial number of events in one FetchResponse. As new events are published to the event bus, they’re sent immediately to the client in a new FetchResponse as long as there are still pending events to be delivered to the client.

The FetchResponse contains the number of pending events not yet delivered in the pending_num_requested field. The client can use this number to decide how many new events to request, if it has capacity to continue processing events. For example, a client’s processing capacity is 100 events and it requests 100 events. Then the client receives 60 events in a FetchResponse with a pending_num_requested field value of 40. The client can request 60 events again, which corresponds to 100 minus the pending_num_requested value of 40.

In this subscription flow, the client initiates a FetchRequest and waits until all requested events are delivered before it sends a new FetchRequest to ask for more events. In this example, the client determines that it can process 100 events and sends a FetchRequest for 100 events, which corresponds to the maximum number of events that can be requested from Salesforce. The client then waits for all events to be delivered before initiating a new FetchRequest to ask for more events. The events can be delivered all at once, in one FetchResponse, or in multiple FetchResponses. In this example, the events are delivered in multiple FetchResponses.

The sequence in this diagram is:

1. Client requests 100 events in a FetchRequest to Salesforce.
2. As events become available in the event bus, Salesforce delivers the events to the client in multiple FetchResponses.
3. After receiving all the events, the client initiates a new FetchRequest requesting 100 more events.

In this subscription flow, the client initiates a FetchRequest, and while events are delivered, sends a new FetchRequest to ask for more events. In this example, the client determines how many events to ask for in the next FetchRequest based on the number of events received so far. The server delivers events as they become available in the event bus in multiple FetchResponses.

The sequence in this diagram is:

1. Client requests 100 events in a FetchRequest to Salesforce.
2. The first FetchResponse returns 60 events.
3. Client sends another FetchRequest for 60 new events.
4. The requested events from the first FetchRequest and the second FetchRequest are delivered in multiple FetchResponses.
5. After receiving all the events, the client initiates a new FetchRequest requesting 100 more events.

The Subscribe call stream stays open as long as there are pending events to be sent to the client. The number of pending events is based on the number of events the client requested in FetchRequests and the number of events delivered in FetchResponses.

If the server sends all requested events to the client, there are no more pending events to be delivered and pending_num_requested in FetchResponse is equal to zero. In this case, to keep the subscription stream open, the client must send a new FetchRequest within 60 seconds from when the last FetchResponse was sent. If the client fails to do so, the subscription stream closes and the client must call Subscribe again to open a new stream.

If there are pending events to be sent to the client (pending_num_requested in FetchResponse is greater than zero), but no new events are available in the event bus, the server keeps the stream alive by sending keepalive messages within 270 seconds. For more information, see Subscribe Keepalive Behavior.

To process events from multiple topics in separate subscription streams, a client can make multiple Subscribe calls. The number of events requested and the maximum requested value in a FetchRequest are per Subscribe call. If you make multiple Subscribe calls in one client, event flow is controlled separately for each subscription. For example, you can subscribe to one topic with a low batch size of 10 events and to another topic with a high batch size of 100 events.

To stay within your event allocations, you can use methods that reduce event consumption for one topic.

• Lower the batch size of requested events.
• Send FetchRequests less frequently.
• Pause a subscription by stopping it and making another Subscribe call.

The received event rate depends on your client implementation and whether you’re processing events from each subscription in parallel or sequentially.