Avoiding Apex Speeding Tickets (Concurrent Request Limits via Synchronous Callouts)

Oh, that error….

Perhaps you’ve seen it before: “Unable to process request: Concurrent requests limit exceeded”. This is the Apex concurrency limiter giving you a “speeding ticket”.

Unable to Process Request: Concurrent Requests Limit ExceededBelieve it or not, the Apex concurrency limit is your friend. It isolates several failure scenarios and contains them from propagation across the Salesforce App Cloud, which might otherwise simultaneously impact thousands of customers; including you.

While you may not have seen the warning screen, many have, and all too often the “go to” response is to suspect the problem lies in the platform.  Not so fast!  If I’m being honest (I am, promise), the limit is a necessary evil, and, when an org gets hit with the error, it’s often because of factors that lie outside of the platform.

Think of it like this: We don’t like issuing speeding tickets and you don’t like receiving them. So, the intent of this document is to dig into one common factor and equip you with knowledge to insulate your org from further impact.

Sound familiar?

In this case, I’m referring to synchronous callouts. When it’s necessary to integrate with an external service, synchronous callouts are widely used because the implementation is simple; you send the request, block the thread, and wait for the response. Furthermore, synchronous callouts often go to services which make additional callouts in a chain. Callouts of this nature are notoriously fragile due to the unpredictable nature of WAN latency, remote systems and their subsequent dependency’s performance profiles. Consequently, one of the key risks in this style of integration is that of cascading failure.

Additionally, all the response times in the chain are cumulative, which can quickly grow and degrade the user experience!

For example, a callout from a Visualforce page to your middleware app A to backend B to cache C to database D can, at any point during the chain, hang waiting for a response to the next downstream service because of <insert any of a million reasons here>.

Synchronous Call Chain

Naturally, these are highly transactional services, so those stuck threads pile up fast and exhaust the service resources. At that point, subsequent requests begin to see “Server too busy” errors or hang, waiting for an available request handler thread. Worse still, assuming this started with the database, D, the error then moves upwards from C, to B, to A and finally, the platform. Without defensive engineering, everything clogs up along the chain.

DB Cascading Failure

 

Defensive strategies exist for this scenario, and others like it! Owners of these disparate services should implement some mechanism to prevent cascading failure such as “Fail Fast”, “Circuit Breakers”, etc (outlined later on this post). However, this is often not done. Thus, the failure cascades into the platform and we “pull you over” to stop accepting further requests from your org until the number of waiting requests is reduced.

Hmph, familiar.  So now what?

Whether or not you can build resiliency into those downstream services, (you should if you can!) there are steps that should be taken to minimize the effects of slow synchronous callouts in your Salesforce application. The platform has callout features you can adapt to your use case, and reduce hitting the Apex concurrency limit error.

As discussed elsewhere, the Apex concurrency limit is a fixed number of concurrent requests that an org is allowed to have running past a certain duration threshold. For the purposes of this doc, we’ll assume it’s 10 requests at 5 seconds each.

Doesn’t sound like much? Perhaps it would help to know that there are customers with millions of requests with callouts that, every day, somehow managed to avoid hitting the limit! I will show you how to do just that.

Why so synchronous? The suboptimal approach

To better illustrate how best to implement callout behavior, let’s take a closer look at the suboptimal approach, which we can then juxtapose to the alternatives. Take, for example, a simple Visualforce page that makes a callout to
an external service which is either slow or susceptible to the aforementioned millions of performance fluctuations.Death by Waiting

This is risky! If the service takes longer than 5 seconds (someday it will) to respond and there are more than 10 of these page requests in-flight, you are susceptible to Apex concurrency limit errors from all your Visualforce pages, SOAP requests, REST requests, etc., not just those pages that are currently long-running!

 

Synchronous callout example

To illustrate, I’m using a homegrown Node.js app on Heroku which takes a duration parameter and essentially ‘sleeps’ before returning the response. As you can see in the code examples below, the page makes a synchronous callout to an HTTP URL and waits for the response which is then displayed in the page messages.

Does the following look familiar? Granted, many of you have wrapped the HttpRequest bits into a helper class, but it’s essentially the same pattern:

Synchronous Visualforce page code

<apex:page controller="SyncCalloutPageController">
  <apex:messages id="results"/>
   <apex:form >
    <apex:pageBlock title="Sync Callout Page">
     <p>
      Enter the duration of the service call:
     </p>
     <apex:inputText id="duration" value="{!duration}"/>
     <apex:commandButton action="{!submit}" value="submit" title="Submit"/>
    </apex:pageBlock>
   </apex:form>
 </apex:page>

Synchronous controller code

public class SyncCalloutPageController {
  
  // Duration value
  private static final Integer defaultDuration = 1000;
  public Integer duration = 0;
  public Integer getduration () {
    return duration;
  }
  public void setduration (Integer value) {
    duration = value;
  }
 
  
  public PageReference submit() {
    if ( duration <= 0 ) {
      duration = defaultDuration;
    }
 
    Http h = new Http();
    HttpRequest request = new HttpRequest();
    request.setEndpoint('http://durationservice.herokuapp.com/long?duration=' + duration);
    request.setMethod('GET');
    request.setTimeout(120000);
    HttpResponse response = h.send(request);
    Integer statusCode = response.getStatusCode();
 
    if ( statusCode != 200 ) {
      ApexPages.addmessage(new ApexPages.message(ApexPages.severity.FATAL, response.getBody()));
    } else {
      ApexPages.addmessage(new ApexPages.message(ApexPages.severity.CONFIRM, response.getBody()));
    }
    return null;
  }
 }

But I learned this by watching you!

Fine, there is plenty of documentation that models this approach yet doesn’t subject you to reams of diatribe on how to implement safe, synchronous callouts.  Also, other sites <cough>Sourceforge</ cough> have sprinklings of good advice which usually falls prey to the TL;DR filter.  Ultimately, there is too much adoption of “the wrong way”, which is why I wrote this document.

Thus, I submit to you: “Making a synchronous callout the right way”:

Single continuation example: an optimized approach

This case uses the asynchronous callout framework in the platform to perform long running (up to 60 seconds each) callouts without tying up the application server thread and prevents being counted against the Apex concurrency limit.Spiraling Timepiece

You’ll notice three key changes from the Synchronous example:

  1. Added JavaScript to the Visualforce page to disable the submit button and update the label to ‘Submitted…’ and
  2. Moved the response handling logic to a separate callback handler method.
  3. Added a ‘reRender’ target to the submit button.

The reason for #1 is because, once submitted, the browser shows no indication of a page load.  No spinny icons, no animations, etc. This is due to how continuation works under the hood;  The page request is submitted to the server while JavaScript in the page polls for a response.

The reason, of course, for #2 is because without a callback handler method we wouldn’t be able to update the page with success/error results.

Third, #3 is added because without it, the callback method will not be invoked and your results will not be displayed in the page.

Here’s the code:

Single continuation Visualforce page code

<apex:page controller="ContinuationCalloutPageController">
     <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js" type="text/javascript"></script>
     <script>
         var $ = jQuery.noConflict();
         function submitClick() {
             $('input[id*=submitBtn]').attr('value','Submitted...');
             $('input[id*=submitBtn]').attr('class', 'btnDisabled');
             return true;
         }
         function submitComplete(){
             $('input[id*=submitBtn]').attr('value','Submit');
             $('input[id*=submitBtn]').attr('class', 'btn');
             return true;
         }
     </script>
     <apex:outputPanel id="wholePanel">
         <apex:messages id="results"/>
         <apex:form >
             <apex:pageBlock title="Continuation Callout Page">
                 <p>
                     Enter the duration of the service call:
                 </p>
                 <apex:inputText id="duration" value="{!duration}"/>
                 <apex:commandButton id="submitBtn" action="{!submit}" value="Submit" title="Submit" reRender="wholePanel" onClick="submitClick()" onComplete="submitComplete()"/>
             </apex:pageBlock>
         </apex:form>
     </apex:outputPanel>
 </apex:page>

Single continuation controller code

public with sharing class ContinuationCalloutPageController {
     private static String URL = 'http://durationservice.herokuapp.com/long?duration=';
     public String result {get;set;}
     public integer callbackCount {get;set;}
     public ContinuationCalloutPageController() {
         this.callbackCount = 0;
     }    
     // Duration value
     Integer defaultDuration = 1000;
     public Integer duration = 0;
     public Integer getduration () {
         return duration;
     }
     public void setduration (Integer value) {
         duration = value;
     }
     // Continuation request label
     public String requestLabel;
     
     public Object submit() {
         if ( duration <= 0 ) {
             duration = defaultDuration;
         }
         HttpRequest request = new HttpRequest();
         request.setEndpoint(URL + duration);
         request.setMethod('GET');
         request.setTimeout(60000);
         
         Continuation cont = new Continuation(60);
         // The requestLabel is the key to getting the Response object later, hang on to it!
         this.requestLabel = cont.addHttpRequest(request);
         cont.continuationMethod = 'processContinuationResponse';
         
        // Can add more HttpRequest objects here - up to three, they will all execute in parallel.
         return cont;
     }
     public Object processContinuationResponse() {
         System.debug('Callback called');
         HttpResponse response = Continuation.getResponse(this.requestLabel);
         System.debug('Got response: ' + response.getBody());
         Integer statusCode = response.getStatusCode();
         System.debug('ResponseCode: ' + statusCode);
         if ( statusCode != 200 ) {
             ApexPages.addmessage(new ApexPages.message(ApexPages.severity.FATAL, response.getBody()));
         } else {
             ApexPages.addmessage(new ApexPages.message(ApexPages.severity.CONFIRM, response.getBody()));
         }
         return null;
     }
  }

I want to affirm some of the key take aways from this pattern:

  1. It effectively eliminates the need for synchronous callouts in Visualforce pages.  Stop using them or use “Fail Fast” or both (see below).
  2. You can bundle up to three requests in a single continuation object and they will all be processed in parallel! (Note: if they need to be done in order, use “chain continuation” – see below).

Also, you should read the “Know thy limits” section further down as they apply to both single and chained continuation implementations.

Please, sir, may I have some more?

While replacing synchronous callouts with the continuation pattern will solve a lot of the world’s problems (the world of sync callouts hitting concurrency limits, that is), it doesn’t solve them all.

Take for example the ubiquitous case of making multiple requests in serial. Say you need to authenticate, request data, and submit changes in that order. In this case, we have orchestration of three individual requests that need to be made where the subsequent requests are dependent on the results of the previous.

Good news! We support this and it is called “continuation chaining”:

Chain continuation: another optimized approach

This example describes how to chain three sets of synchronous service calls together and includes one set where three are executed in parallel.
ChainsSome of the key differences between this example versus the previous, Single continuation example are:

  1. The aggregate wait time across all three continuation batches must not exceed 180 seconds.
  2. Tracking the response label, a unique identifier, is crucial to retrieving the response object for a particular callout from the Continuation object.
  3. Instead of returning ‘null’ or a new PageReference from the response handler method, you return the next Continuation object.
  4. Messages added to the page are displayed from the last response handler method only. To avoid this, the messages are stored on the controller.
  5. The ApexPages.message object is not serializable. The type and value, however, are serializable.  To maintain simplicity, a separate class was created to store type and message in a list.

Chain Continuation Visualforce page code

<apex:page controller="ChainContinuationCalloutPageController">
     <script src="https://ajax.googleapis.com/ajax/libs/jquery/1.7.1/jquery.min.js" type="text/javascript"></script>
     <script>
         var $ = jQuery.noConflict();
         function submitClick() {
             $('input[id*=submitBtn]').attr('value','Submitted...');
             $('input[id*=submitBtn]').attr('class', 'btnDisabled');
             return true;
         }
         function submitComplete(){
             $('input[id*=submitBtn]').attr('value','Submit');
             $('input[id*=submitBtn]').attr('class', 'btn');
             return true;
         }
     </script>
     <apex:outputPanel id="wholePanel">
         <apex:messages id="results"/>
         <apex:form >
             <apex:pageBlock title="Chain Continuation Callout Page">
                 <p>
                     Enter the duration of the service call:
                 </p>
                 <apex:inputText id="duration" value="{!duration}"/>
                 <apex:commandButton id="submitBtn" action="{!submit}" value="Submit" title="Submit" reRender="wholePanel" onClick="submitClick()" onComplete="submitComplete()"/>
             </apex:pageBlock>
         </apex:form>
     </apex:outputPanel>
 </apex:page>

Chain continuation controller code

public with sharing class ChainContinuationCalloutPageController {

    private static String endpointUrl = 'https://na16.salesforce.com/services/apexrest/longrunningrestcallout';
    private static String authURL = 'https://login.salesforce.com/services/oauth2/token';
    private static String authPostText = SFDC_OAuth_Content__c.getOrgDefaults().value__c;
    private static String cacheToken = 'local.auth.accesstoken';
    private static Integer DEBUG = 1;

    // Duration value
    Integer defaultDuration = 1000;
    public Integer duration = 0;
    public Integer getduration () {
        return duration;
    }
    public void setduration (Integer value) {
        duration = value;
    }


    public class SerializableMessage {

        private String message;
        public String getmessage() {
            return this.message;
        }
        public void setmessage(String value) {
            this.message = value;
        }

        private ApexPages.severity severity;
        public ApexPages.severity getseverity() {
            return this.severity;
        }
        public void setseverity(ApexPages.severity value) {
            this.severity = value;
        }

        public SerializableMessage(ApexPages.severity severity, String message) {
            this.message = message;
            this.severity = severity;
        }
    }

    public class ContinuationHelper {

        // Helper method to handle request maps above
        public Continuation createContinuation(Map<String, Map<String, String>> requestsMap, Map<String, String> headersMap) {
            Continuation cont = new Continuation(60);

            for (String mapKey : requestsMap.keySet()) {
                if (mapKey.startsWith('REQUEST_')) {
                    Map<String, String> requestMap = requestsMap.get(mapKey);
                    HttpRequest request = new HttpRequest();
                    if ( requestMap.get('METHOD') == 'GET' && requestMap.get('QUERY_STRING') != null ) {
                        request.setEndpoint(requestMap.get('ENDPOINT') + '?' + requestMap.get('QUERY_STRING'));
                    } else {
                        request.setEndpoint(requestMap.get('ENDPOINT'));
                    }
                    if ( requestMap.get('METHOD') == 'POST' && requestMap.get('REQUEST_TEXT') != null ) {
                        request.setBody(requestMap.get('REQUEST_TEXT'));
                    }
                    request.setMethod(requestMap.get('METHOD'));
                    request.setTimeout(Integer.valueOf(requestMap.get('TIMEOUT')));
                    if ( headersMap != null && headersMap.size() > 0 ) {
                        for ( String headerMapKey : headersMap.keySet() ) {
                            request.setHeader(headerMapKey, headersMap.get(headerMapKey));
                        }
                    }
                    requestMap.put('RESPONSE_LABEL', cont.addHttpRequest(request));
                }
            }
            cont.continuationMethod = requestsMap.get('CALLBACK').get('METHOD_NAME');
            return cont;
        }

    }

    private ChainContinuationCalloutPageController.SerializableMessage [] messages = new List<ChainContinuationCalloutPageController.SerializableMessage>();
    private ContinuationHelper ch = new ContinuationHelper();

    // Auth request behavior - could be an object
    private Map<String, Map<String,String>> authRequestMap = new Map<String, Map<String, String>> {
            'REQUEST_AUTH' => new Map<String, String> {
                    'METHOD' => 'POST',
                    'TIMEOUT' => '60000',
                    'ENDPOINT' => authURL,
                    'REQUEST_TEXT' => authPostText
            },
            'CALLBACK' => new Map<String, String> {
                    'METHOD_NAME' => 'processAuthResponse'
            }
    };

    // Request data behaviors - could be a custom object
    private Map<String, Map<String, String>> getDataRequestMap = new Map<String, Map<String, String>> {
            'REQUEST_GET_DATA' => new Map<String, String> {
                    'METHOD' => 'POST',
                    'TIMEOUT' => '60000',
                    'ENDPOINT' => endpointUrl,
                    'REQUEST_TEXT' => '{"duration":"' + duration + '"}'
            },
            'CALLBACK' => new Map<String, String> {
                    'METHOD_NAME' => 'processGetDataResponse'
            }
    };

    // Finally, submit changes behavior, could be a custom object
    private  Map<String, Map<String, String>> submitChangesRequestMap = new Map<String, Map<String, String>> {
            'REQUEST_SUBMIT_CHANGES' => new Map<String, String> {
                    'METHOD' => 'POST',
                    'TIMEOUT' => '60000',
                    'ENDPOINT' => endpointUrl,
                    'REQUEST_TEXT' => '{"duration":"' + duration + '"}'
            },
            'CALLBACK' => new Map<String, String> {
                    'METHOD_NAME' => 'processSubmitChangesResponse'
            }
    };


    private void populatePageMessages() {
        for (ChainContinuationCalloutPageController.SerializableMessage message : messages) {
            ApexPages.addmessage(new ApexPages.message(message.getseverity(), message.getmessage()));
        }
    }

    public Object submit() {
        Map<String, String> headersMap = new Map<String,String>();
        headersMap.put('Content-Type', 'application/x-www-form-urlencoded');
        Continuation cont = ch.createContinuation(authRequestMap, headersMap);
        return cont;
    }

    public Object processAuthResponse() {

        System.debug('processAuthResponse');

        // Process the oauth response
        HttpResponse response = Continuation.getResponse(authRequestMap.get('REQUEST_AUTH').get('RESPONSE_LABEL'));
        try {
            JSONParser parser = JSON.createParser(response.getBody());
            while ( parser.nextToken() != null ) {
                if ( parser.getCurrentName() == 'access_token') {
                    Cache.Org.put(cacheToken, parser.getText());
                }
            }

        } catch ( Exception e ) {
            System.debug('Exception while processing response: "' + response + '"');
            System.debug('Exception caught: ' + e);
        }

        // Possible Continuation error status codes:
        // 2000: The timeout was reached, and the server didn’t get a chance to respond.
        // 2001: There was a connection failure.
        // 2002: Exceptions occurred.
        // 2003: The response hasn’t arrived (which also means that the Apex asynchronous callout framework hasn’t resumed).
        // 2004: The response size is too large (greater than 1 MB).


        Integer statusCode = response.getStatusCode();
        if ( statusCode == 200 ) {
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.CONFIRM, 'Authenticated: ' + response.getBody()));
        } else {
            // Error returned from service, display to page and end flow here.
            messages.add(new CHainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'FAILED.  Status: ' + statusCode + ' Response: ' + response.getBody()));
            populatePageMessages();
            return null;
        }

        if ( Cache.Org.contains(cacheToken)) {
            // Use bearer token to make call to retrieve data from service
            System.debug('Bearer: ' + Cache.Org.get(cacheToken));
            Map<String, String> headersMap = new Map<String,String>();
            headersMap.put('Authorization', 'Bearer ' + Cache.Org.get(cacheToken));
            headersMap.put('Content-type', 'application/json');
            Continuation cont = ch.createContinuation(getDataRequestMap, headersMap);
            return cont;
        } else {
            // No bearer, display message(s) to page and end flow here.
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Auth FAILED. Status: ' + statusCode + ' Response: ' + response.getBody()));
            populatePageMessages();
            return null;
        }

        // Landing here is unexpected, display message(s) to page and end flow.
        messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Unexpected processAuthResponse state.'));
        return null;
    }

    public Object processGetDataResponse() {

        System.debug('processGetDataResponse');
        // Handle response
        HttpResponse response = Continuation.getResponse(getDataRequestMap.get('REQUEST_GET_DATA').get('RESPONSE_LABEL'));
        Integer statusCode = response.getStatusCode();

        if ( statusCode == 200 ) {
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.CONFIRM, 'Got Data: ' + response.getBody()));
        } else {
            // Error returned from service, display to page and end flow here.
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Get Data FAILED.  Status: ' + statusCode + ' Response: ' + response.getBody()));
            populatePageMessages();
            return null;
        }


        if ( Cache.Org.contains(cacheToken)) {
            // Use bearer token to make call to retrieve post data changes from service
            Map<String, String> headersMap = new Map<String,String>();
            headersMap.put('Authorization', 'Bearer ' + Cache.Org.get(cacheToken));
            headersMap.put('Content-type', 'application/json');
            return ch.createContinuation(submitChangesRequestMap, headersMap);
        } else {
            // No bearer, display message(s) to page and end flow here.
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Auth FAILED. Status: ' + statusCode + ' Response: ' + response.getBody()));
            populatePageMessages();
            return null;
        }


        // Landing here is unexpected, display message(s) to page and end flow.
        messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Unexpected processGetDataResponse state.'));
        populatePageMessages();
        return null;
    }

    public Object processSubmitChangesResponse() {

        System.debug('processSubmitChangesResponse');
        // Handle process submit response
        HttpResponse response = Continuation.getResponse(submitChangesRequestMap.get('REQUEST_SUBMIT_CHANGES').get('RESPONSE_LABEL'));
        Integer statusCode = response.getStatusCode();
        if (statusCode == 200) {
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.CONFIRM, 'Submitted Changes: ' + response.getBody()));
        } else {
            messages.add(new ChainContinuationCalloutPageController.SerializableMessage(ApexPages.severity.FATAL, 'Submit Changes FAILED.  Status: ' + statusCode + ' Response: ' + response.getBody()));
            populatePageMessages();
            return null;
        }

        // Completed flow - publish messages to page and render.
        populatePageMessages();

        // Done
        return null;
    }
}

 

Key takeaways for the chain callout example:

  1. You get up to three requests per Continuation and up to three Continuations can be chained.
  2. All requests in a Continuation are made in parallel but the response handler is called only when all requests in the Continuation are completed.
  3. Be sure to track response labels carefully to ensure retrieval of the intended response objects.

Know thy limits..

While Continuation gives you a lot more rope with which to play, it’s not a blank check. Try to keep the following details in mind:

The Book

  • As of the Summer 2016 release, Named Credentials work with Continuations (see the Named Credential Sample below for some sample code). Now that Continuations support Named Credentials, you can eliminate one step in the chain, and reducing aggregate wait time.
  • For previous releases, or cases where Named Credentials cannot be leveraged, continue considering alternative patterns. For example, you can use the Chained continuation example just covered, which uses Custom Settings and Platform Cache to orchestrate authentication, fetch, and push across several callouts.
  • You get up to sixty (120) seconds per callout in a Continuation regardless of the number of callouts in that Continuation;  i.e., each individual callout in the continuation can take up to 120 seconds to respond. The upside, of course, is that they are run in parallel which reduces the aggregate response time.
  • You can have up to three (3) Continuations in a chain. With each Continuation allowing for three parallel callouts. This allows for the potential total of 360 seconds in running time across all three Continuations. Note that VF pages using Continuation are not subject to the 120 second page timeout.

For example, a Continuation with three callouts where one takes 5 seconds, the second takes 3 seconds and the third taking 10 seconds would take 18 seconds if executed serially. However, Continuation callouts run in parallel, so the running time will be that of the longest running callout, or 10 seconds, leaving 50 seconds remaining of the Continuation timeout limit.

You forgot X, Y or Z!

By now, an astute reader such as yourself is asking “What about REST or SOAP services written in Apex that make callouts?” or “What if I need more than three chains or three requests in a chain?”.

Admittedly, continuation doesn’t handle every case and is currently limited to making calls from Visualforce page and Visualforce Remote controllers. For Continuation from SOAP/REST Apex services, there is an internal discussion about support but no current plans. And, while increasing the limits is possible in a future release, it would require making a strong case to the Development team to raise them for your org.

Fortunately, there are a few remaining suggestions to help with these and other cases:

Bundling

Making many (5+) service requests from any type of synchronous request/response framework is going to involve perceptible delays. Consider that each request has at least 100-150ms in round trip time (SSL,Ping,etc.) before adding in server runtime which is anywhere from 30-3000ms. When invoked in serial it’s impractical to expect aggregate response times less than 5 seconds and, when in the wild, they can take 10 seconds or more.

One suggestion is to find a way to bundle these requests into a single request. The Mediator pattern, from the Enterprise Integration Patterns repertoire, could take a combined payload and fan out the requests in parallel and return an aggregate response payload.

You could easily build one on Heroku but this is beyond the scope of this doc. Stay tuned!

Circuit Breakers

This is one of my favorite system integration pattern fail-safes. It’s a more intelligent version of “Fail Fast” (see below), but, essentially, circuit breakers are a software version of the same thing you have in your home. However, instead of tripping when you stick a fork in an outlet (do not try this!), it trips when certain runtime criteria are met.

One way to implement circuit breakers is similar to what the Salesforce App Cloud does with the Apex limiters: use the Platform Cache feature to keep track of callouts to certain remote systems or endpoints. And, when you start seeing N failures or timeouts, you block/queue/whatever subsequent calls to that endpoint and stop failure from cascading.

Similar to “Fail Fast”, one of the key elements of this defense pattern is communicating to end users and thinking through your use case. Make sure the handling is graceful, and, where possible, resume work as seamlessly as possible.

Not to be a tease, but building circuit breakers into your Salesforce application is out of scope for this doc.  Stay tuned.

When all else fails.. FAIL FAST!

Last but not least, consider that when you allow dynamic factors (such as internet latency) to impact your application’s stability, you are allowing for some amount of inevitable downtime.  Do not, then, be surprised (or upset!) when the inevitable happens.

It was planned, right? No? Read on.

There’s a mantra in integrated systems design which applies directly to this problem: take the problem head on and build it into the system’s behavior.  We know it’s going to fail/slow down/hiccup/barf, it’s just a matter of “when”, so Fail Fast.

The difference between taking this approach and leaving the problem to someone else (i.e. the Apex limiters) is that, when it does hit the fan, you can do it with style and panache that may make your users happy instead of leaving them reaching for their pitchforks.

So, in Apex, whenever you make a callout, take the extra effort to code in some limits of your own and tie it into the application experience.  It’s more work but it pays off:

Fail Fast example

Here we’re not doing anything fancy, really. We just make sure the request will never go longer than 5 seconds.  Alarm ClockWhen the timeout is hit, you catch the exception and notify the users that the remote server is not responding within your tight SLA’s and do one of many things:

  1. Give them control of the page after a few seconds (So they don’t spam the reload button!)
  2. Retry programmatically (with automated decay)
  3. Queue and poll
  4. etc.

Granted, some of these options can be complex to implement, but the key benefit is that you can explain to the users what the problem is (the Apex concurrency limit error is vague and indiscriminate!). Communicate what their next course of action should be and do it in a way that is integrated with your applications’ look-and-feel.

Fail Fast example Visualforce page code

<apex:page id="TimeoutCalloutPage" controller="TImeoutCalloutPageController">
     <apex:outputPanel id="wholePanel">
         <apex:messages id="results"/>
         <apex:form >
             <apex:pageBlock title="Sync Callout Page">
                 <p>
                     Enter the duration of the service call:
                 </p>
                 <apex:inputText id="duration" value="{!duration}"/>
                 <apex:commandButton action="{!submit}" value="submit" title="Submit"/>
             </apex:pageBlock>
         </apex:form>
     </apex:outputPanel>
 </apex:page>

 

Fail Fast example controller code

public with sharing class TImeoutCalloutPageController {
         // Duration value
         private static final Integer defaultDuration = 1000;
         public Integer duration = 0;
         public Integer getduration () {
             return duration;
         }
         public void setduration (Integer value) {
             duration = value;
         }
         public PageReference submit() {
             if ( duration <= 0 ) {
                 duration = defaultDuration;
             }
             Http h = new Http();
             HttpRequest request = new HttpRequest();
             request.setEndpoint('http://durationservice.herokuapp.com/long?duration=' + duration);
             request.setMethod('GET');
             request.setTimeout(5000); // stop at 5 seconds
             HttpResponse response;
             try {
                 response = h.send(request);
             } catch ( CalloutException ce ) {
                 // Handle timeout gracefully
                 if ( ce.getMessage().equals('Read timed out') ) {
                     ApexPages.addmessage(new ApexPages.message(ApexPages.severity.FATAL, 'WARNING: Remote system timed out. Please contact the administrator.'));
                 } else {
                     ApexPages.addmessage(new ApexPages.message(ApexPages.severity.FATAL, ce.getMessage()));
                 }
                 return null;
             }
             Integer statusCode = response.getStatusCode();
             if ( statusCode != 200 ) {
                     ApexPages.addmessage(new ApexPages.message(ApexPages.severity.FATAL, response.getBody()));
             } else {
                     ApexPages.addmessage(new ApexPages.message(ApexPages.severity.CONFIRM, response.getBody()));
             }
             return null;
         }
 }

 

The key difference between this example and the synchronous pattern is that we set the timeout to something ‘tight’, like 5 seconds. Then, catch the CalloutException, confirm it matches the “Read timed out” value, and add a message to the page.  Review of your use case should readily offer up one or more behaviors that occur when you start hitting timeouts. All of which should certainly be preferable to hitting the Apex concurrency limit.

Drive safe!

And Now, Named Credentials, Too!

As of the Summer 2016 release, Continuations support Named Credentials. With Named Credentials, the platform takes steps for authentication and caching of auth tokens, etc. This effectively eliminates the need to use one chain in a Chained Continuation to auth and store the token in the cache like we did in the example above.
While configuring a Named Credential is beyond the scope of this post, the code changes to the Continuation examples are relatively straightforward. Named Credentials are invoked by prefixing the endpoint URL in the format “callout:<NamedCredential>”. In this case, our Named Credential is called “NA16” and would thusly be invoked via the prefix “callout:NA16”.

The following shows the how you can change the Chain Continuation example presented above to leverage the NA16 Named Credential:

ChainContinuationCalloutPageController

Static Declarations

Before:
 private static String endpointUrl = 'https://na16.salesforce.com/services/apexrest/longrunningrestcallout';
 private static String authURL = 'https://login.salesforce.com/services/oauth2/token';
 private static String authPostText = SFDC_OAuth_Content__c.getOrgDefaults().value__c;
 private static String cacheToken = 'local.auth.accesstoken';
After:
private static String endpointUrl = 'callout:NA16/services/apexrest/longrunningrestcallout';

Submit Method

Before:
public Object submit() {
     Map<String, String> headersMap = new Map<String,String>();
     headersMap.put('Content-Type', 'application/x-www-form-urlencoded');
     Continuation cont = ch.createContinuation(authRequestMap, headersMap);
     return cont;
 }
After:
public Object submit() {
    System.debug('Submitting call to get data');
    Map<String, String> headersMap = new Map<String,String>();
    Continuation cont = ch.createContinuation(getDataRequestMap, headersMap);
    return cont;
}

Process Auth Response Method

Removed.

 

That’s it. This now frees the Chain Continuation for one more chain of up to three parallel service calls! Very useful!

About the Author

Scott Mikolaitis is a Technical Enablement Architect within the Technical Enablement team of the Salesforce Customer-Centric Engineering group.  He has 18 years of industry experience with focus on system architecture and software integration to curate and evangelize best practices.   The team’s mission is to help customers understand how to implement technically sound Salesforce solutions. Check out all of the resources that this team maintains on the Architect Core Resources page of Developer Force.

Related Resources

 

tagged , , , , Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • Great article – love the Continuation framework!

  • Daniel Ballinger

    At the time of writing with the Winter `16 release it isn’t (easily) possible to create test cases for continuations using SOAP based web services. See Asynchronous Callout from an Imported WSDL and Running Test for WSDL-based asynchronous callouts fails with Internal Salesforce.com Error

    At Dreamforce 15 Reggie Nair presented a work around (hack) to use a continuation outside of a typical Apex context. Basically create a wrapper VF page that serves to make the callout and return the response. See Introduction To Apex Asynchronous Callout Framework, aka, Continuation

  • Robert Strunk

    Thanks for posting, very thorough! There is a lot to digest here and I look forward to honing my Continuation

    skills.

  • Sanchit Dua

    Very informative.. Thanks for the share!

  • The apex concurrency limit helps in isolating several failure scenarios which might impact a lot of customers. The synchronous pattern catch the CalloutException, “Read timed out” value.. all of these can be be preferred for hitting the Apex concurrency limit. This helps a lot for the CRM salesforce consulting partner.

  • Scott Mikolaitis

    Just updated the post to include content for the newly added Named Credential support for Continuations in Summer ’16.

    • Andrew Taylor

      Thanks for posting this Scott; I noticed in the “Chain Continuation Visualforce page code” section, the Visualforce page references {!duration}, but there’s no duration in the controller class. Is one of these out of date from the updates you made?

      • Scott Mikolaitis

        @disqus_oxkNWSruey:disqus Good point! I had the functionality in place but didn’t pass it through the controller to the REST service. I went ahead and updated the controller code to support the duration pass through. Thanks for mentioning it.

  • Scott – Know Thy Limits bullet three: “You get up to sixty (120) seconds”. Nice deception 🙂