Pull Subscription and Flow Control

Unlike push-based subscriptions in which a client waits to receive new events pushed from the server, with a pull subscription, a client requests events from the server. To request events, a client calls the Subscribe RPC Method and passes in a FetchRequest. The client sets the number of events that it has capacity to process in the num_requested field of FetchRequest. This event flow control ensures that the client doesn’t get overwhelmed with more events that it can handle if there is a spike in event publishing.

A client can send one or more FetchRequests as part of the same Subscribe call, and it can receive one or more FetchResponses in return.

Subscribe RPC requests and responses

The maximum number of events that can be requested across all FetchRequests in a Subscribe call is 100. If a client requests more than 100 events, the server considers the number of requested events to be 100. The client sets the number of events to request in the num_requested field of the FetchRequest. The server adjusts the total number of requested events based on the requested events sent in one or more FetchRequests, and how many events were already delivered.

In a basic subscription flow, the client can request one event at a time. It waits for the event to be received, processes the event, and then requests a new event.

This diagram shows a basic sequence of requesting and processing one event at a time. The sequence is:

  1. Client requests an event in a FetchRequest to Salesforce.
  2. When an event is available in the event bus, Salesforce delivers the event to the client.
  3. After receiving the event, the client initiates a new FetchRequest requesting one more event.
  4. When a new event is available in the event bus, Salesforce delivers the event to the client.

Basic flow of one event requested at a time

The server can send the requested events in one or more FetchResponses. If there are enough events in the event bus, the requested events can be delivered in one FetchResponse, or in multiple FetchResponses if the total size of events exceeds 3 MB.

If there aren’t enough events in the event bus, the client receives a partial number of events in one FetchResponse. As new events are published to the event bus, they’re sent immediately to the client in a new FetchResponse as long as there are still pending events to be delivered to the client.

The FetchResponse contains the number of pending events not yet delivered in the pending_num_requested field. The client can use this number to decide how many new events to request, if it has capacity to continue processing events. For example, a client’s processing capacity is 100 events and it requests 100 events. Then the client receives 60 events in a FetchResponse with a pending_num_requested field value of 40. The client can request 60 events again, which corresponds to 100 minus the pending_num_requested value of 40.

In this subscription flow, the client initiates a FetchRequest and waits until all events are delivered before it sends a new FetchRequest to ask for more events. In this example, the client determines that it can process 100 events and sends a FetchRequest for 100 events, which corresponds to the maximum number of events that can be requested from Salesforce. The client then waits for all events to be delivered before initiating a new FetchRequest to ask for more events. The events can be delivered all at once in one FetchResponse, or in multiple FetchResponses. In this example, the events are delivered in multiple FetchResponses.

The sequence in this diagram is:

  1. Client requests 100 events in a FetchRequest to Salesforce.
  2. As events become available in the event bus, Salesforce delivers the events to the client in multiple FetchResponses.
  3. After receiving all the events, the client initiates a new FetchRequest requesting 100 more events.

A flow with multiple responses

Complex Flow: Requesting New Events While Processing Received Events

In this subscription flow, the client initiates a FetchRequest, and while events are delivered, sends a new FetchRequest to ask for more events. In this example, the client determines how many events to ask for in the next FetchRequest based on the number of events received so far. The server delivers events as they become available in the event bus in multiple FetchResponses.

The sequence in this diagram is:

  1. Client requests 100 events in a FetchRequest to Salesforce.
  2. The first FetchResponse returns 60 events.
  3. Client sends another FetchRequest for 60 new events.
  4. The requested events from the first FetchRequest and the second FetchRequest are delivered in multiple FetchResponses.
  5. After receiving all the events, the client initiates a new FetchRequest requesting 100 more events.

A complex flow with multiple requests and responses

The Subscribe call stream stays open as long as there are pending events to be sent to the client. The number of pending events is based on the number of events the client requested in FetchRequests and the number of events delivered in FetchResponses.

If the server sends all requested events to the client, there are no more pending events to be delivered and pending_num_requested in FetchResponse is equal to zero. In this case, to keep the subscription stream open, the client must send a new FetchRequest within 60 seconds from when the last FetchResponse was sent. If the client fails to do so, the subscription stream closes and the client must call Subscribe again to open a new stream.

If there are pending events to be sent to the client (pending_num_requested in FetchResponse is greater than zero), but no new events are available in the event bus, the server keeps the stream alive by sending keepalive messages within 270 seconds. For more information, see Subscribe Keepalive Behavior.