This paper is for experienced technical architects who work with Salesforce deployments who want to have a better understanding of Force.com asynchronous processing. This will help an architect build effective design patterns around asynchronous processing.
An asynchronous process is a process or function which does not require immediate interaction with a user. An asynchronous process can execute a task "in the background" without the user having to wait for the task to finish. Force.com features such as asynchronous Apex, Bulk API, and Reports and Dashboards use asynchronous processing to efficiently process requests.
Asynchronous processing provides a number of benefits including:
There are many different types of asynchronous requests on Force.com. Some are user initiated and some are internal housekeeping functions. Asynchronous requests users will be familiar with include:
Force.com asynchronous processing makes a best effort to complete requests as quickly as possible, however there are no guarantees on wait or processing time.
Asynchronous processing, in a multi-tenant environment, presents some challenges:
The following diagram provides a high level overview of Force.com's asynchronous processing technology:
Salesforce.com uses a queue-based asynchronous processing framework. This framework is used to manage asynchronous requests for multiple organizations within each instance. The request lifecycle is made up of three parts:
Each request is processed by a handler. The handler is the code that performs functions for a specific request type.
Handlers are executed by worker threads on each of the application servers that make up an instance. Each application server supports a finite amount of threads. Each thread can execute any type of handler and Salesforce determines how many threads are available to process requests for a given request type. The threads request work from the queuing framework and when received, start a specific handler to do the work. The following diagram shows the asynchronous processing in action:
Notice that the queue contains requests from multiple organizations. Each request can be associated with a job that could vary in complexity and running time. In the diagram, longer running jobs are represented with larger boxes.
As one request is completed, another request is removed from the queue and processed. Error handling and failure recovery is built in (via request persistence) so the requests are not lost if a queue failure or handler failure occurs.
An organization can have many requests outstanding. For example, a single organization could queue 250,000 @future Apex requests in a 24-hour period, depending on Salesforce license type. If one organization adds a large number of requests to the queue, it could prevent other customers from getting access to the worker threads. To avoid this, the queuing framework implements flow control which prevents a single customer from using all of the available threads.
When a worker thread is available to process a request, the queuing framework will determine if the maximum number of worker threads (as determined by the handler) is being used by a single organization. If so, the framework will "peek" into the queue to see if other organizations have requests waiting. The set of requests is called the peek set and is limited to a fixed number of requests at the front of the queue (currently set at 2,000 requests). The framework will look for the requests for a different organization and process those (as long as that organization isn’t currently consuming all of its allocated threads for a given handler).
For example, assume organization 1 creates 13 @future requests that are at the head and adjacent in the queue as shown in the diagram below:
Organization 2 adds two @future requests to the queue:
And two more organization 1 @future requests are en-queued. At this point, the queue looks like this:
For this example, assume that a maximum of 12 threads can process requests from a single organization, and that our peek set size is 15 requests. If 13 total threads are available and no other requests are being processed for organization 1 or organization 2, the processing will be as follows (see diagram above):
What happens when requests for a particular organization occupy the entire peek set when the queue is scanned in step 3 above?
Again, assume 12 threads are processing requests from organization 1. This time, organization 1 has 15 requests remaining in the queue and organization 2 has two requests in the queue as shown in this diagram:
Since all of the requests in the peek set are from a single organization (organization 1), those 15 requests will be moved to the back of the queue with a specific delay. This is called an extended delay.
The delay is different for each message. For example, for @future requests, the delay is 5 minutes. That means a minimum of 5 minutes must elapse before those requests are eligible for processing.
When delayed requests become eligible for processing, it's possible for these requests to be acted upon by flow control and again get moved to the back of the queue and delayed. Therefore requests can be delayed several times before they're completed. Additionally, when those requests are moved, they will be put back into the queue in the same logical order and they may have other requests intermingled with them.
Asynchronous processing in Force.com is very important but has lower priority over real-time interaction via the browser and API. Message handlers run on the same application servers that process interactive requests, and it's possible that asynchronous processing or increased interactive usage can cause a sudden increase in usage of computing resources.
To ensure there are sufficient resources to handle a sudden increase, the queuing framework will monitor system resources such as server memory and CPU usage and reduce asynchronous processing when thresholds are exceeded. If necessary, under heavy load, Salesforce will delay long running jobs in the queue to give resource priority to synchronous requests. Once the resources fall below thresholds, normal asynchronous processing will continue.
Apex supports batch Apex and @future Apex methods. Both of these features add requests to the asynchronous queue. Keep the following best practices in mind when planning out development work that will use asynchronous Apex.
Every @future invocation adds one request to the asynchronous queue. Design patterns that would add large numbers of @future requests over a short period of time should be avoided unless absolutely needed. Best practices include:
Extended delay time is 5 minutes.
Some example scenarios:
Ensure that the Batch Apex process executes efficiently as possible and minimize the batches submitted at one time. Like @future requests, batch Apex needs to execute as fast as possible. Best practices include:
Extended delay is not applicable to batch Apex.
For more best practices that ensure your asynchronous Apex requests are handled properly, see the Force.com Apex Code Developer's Guide.
The Bulk API lets you create jobs and batches to load large volumes of data asynchronously. Bulk API batches are added to the asynchronous queue. If too many batches are submitted at one time, they may be subject to flow control therefore minimize the number of batches if possible.
If more than 2,000 unprocessed requests from a single organization are in the queue, any additional requests from the same organization will be delayed while the queue handles requests from other organizations. Minimize the number of batches submitted at one time to ensure that your batches are not delayed in the queue.
For more best practices that ensure your asynchronous Bulk API batches are handled properly, see the Bulk API Developer's Guide.