+ Start a Discussion
JonWuJonWu 

How do I diagnose sudden bursts of REQUEST_RUNNING_TOO_LONG errors since Winter '16?

For years I've had code that uses the SOAP API through the Java SDK to insert and update records. I also have some Apex triggers that run on the Salesforce side. However, this past weekend, I many REQUEST_RUNNING_TOO_LONG errors - mostly in 2 distinct time intervals (see screenshot below).

Error time windows (Pacific Time)

We're on NA10 but I see nothing posted in trust about issues this past weekend. However, it seems like slowness or an issue on the Salesforce end is the most likely cause here given that we haven't changed anything in months.

In addition, I've seem this 10's of other times (usually in bursts again) since the Winter '16 update but we've never gotten this error before. How can we get back to being reliable and has anybody else had this issue?
James LoghryJames Loghry
From what it sounds like, the integration and triggers have been close to this limit, but just haven't gone over the timeout for whatever reason.  I would take a look at your trigger and make sure it's bulkified and optimized for your integration first.  That being said, I came across another post here having a similar issue: ​http://salesforce.stackexchange.com/questions/97355/what-causes-request-running-too-long-test-errors (http://salesforce.stackexchange.com/questions/97355/what-causes-request-running-too-long-test-errors)

Might want to keep your eye out there and see if something comes up.  You could also try opening a support case to see if you get any traction there.
JonWuJonWu
Hi James,

Thanks for the response. It's interesting to see that somebody else was having an issue with REQUEST_RUNNING_TOO_LONG on deployment as well since Winter '16. All my triggers are bulkified and pretty optimized. In some sense, it doesn't even matter if my code is optimized, even though it is already. When I insert or update records, I only do a handful at a time. There's usually only a handful of SOQL queries and DML statements per API call.

What's tricky is actually getting the logs since the issue seems to come up in short bursts, then go away for days and come back. I just had another bunch of problems in the last few hours. While I wasn't able to capture logs at the time, I enabled debug logs to see how performance is overall. Most API calls take 200-700 ms and are pretty quick.

There are a few longer ones but nothing over 6000-7000 ms. What's weird in the longer ones is that the logs don't even show reflect the full time. For example, the logs say a request took 6,129 ms but the first statement is
05:41:16.041 (41922002)|EXECUTION_STARTED
and the last is
05:41:16.110 (110233378)|EXECUTION_FINISHED
Should that be 69 ms?

Do you have any tips on how to get a ticket in front of the right people without developer support? I feel so stuck with Salesforce because every time I've contact support over the past few years, it takes weeks to get my ticket to the right level. They usually start off telling me they won't support me since I don't have Premier Support or they insist on screen sharing which is usually irrelevant. And 95% of the time, I've found a bug in the platform so it's frustrating trying to convince them so hard even with a reproducible test case.

I understand the need to pay for general development help, but it feels like we should be supported when we find errors that are or seem to be part of the platform and out of our control.

Appreciate the help!
Jon
JonWuJonWu
Also, I just looked deeper into our server logs and we have just as many UNKNOWN_EXCEPTION errors from Salesforce as well. It defintiely seems like something's really wrong on the Salesforce side.

For example:
2015-11-03T04:33:02.895000Z - Exception Code: UNKNOWN_EXCEPTION, Exception Message: An unexpected error occurred. Please include this ErrorId if you contact support: 258740464-244376 (-585205383)
2015-11-03T04:30:02.154000Z - Exception Code: UNKNOWN_EXCEPTION, Exception Message: An unexpected error occurred. Please include this ErrorId if you contact support: 549837247-247302 (1241199627)
2015-11-03T04:29:58.386000Z - Exception Code: UNKNOWN_EXCEPTION, Exception Message: An unexpected error occurred. Please include this ErrorId if you contact support: 1622494075-37548 (-19788460)
2015-11-03T04:29:55.963000Z - Exception Code: UNKNOWN_EXCEPTION, Exception Message: An unexpected error occurred. Please include this ErrorId if you contact support: 642834536-240544 (-1813102819)
2015-11-03T04:29:52.962000Z - Exception Code: UNKNOWN_EXCEPTION, Exception Message: An unexpected error occurred. Please include this ErrorId if you contact support: 1080441985-227203 (303410781)

The problem, is... how do I contact support when it doesn't let me file a case for API issues?
Craig Steen 17Craig Steen 17
Hi Jon,
I'm experiencing something similar in another Production org with a process that's been running for 18months / year. Data volumes aren't the issue, and running the queries run in the Apex code directly from the Dev console return the results quickly (< 1s).

It would be good to understand what your resolution was here.
Cheers,
Craig
JonWuJonWu
Hi Craig,

Sorry to hear you're in hell too! Unfortunately, I have nothing but stories of pain to share. After countless hours back and forth with support and escalations to tier 3 and R&D, we finally got confirmation that they had a database connection bug in their release from back when I last posted. That issue has since been resolved, but it wasn't until our peak season had passed along with all those customers and orders. It was weeks until support would even acknowledge there was an issue, despite hundreds of detailed reports we gave them.

The best advice I can give you is to try and find a reproducible test case, and push to get to the highest tier of support that's possible. You can use your account manager to help speed up escalations. We didn't have Premier Success at the time when I last posted, and that made it really hard for support to talk to us as they kept pushing back saying they wouldn't help on a developer issue (despite that their database tier was clearly just not working). Since then, we've been paying the Premier Success ransom just to get the ability to talk to somebody without friction, but Salesforce support is rarely helpful, and is a pain to talk to unfortunately. I hope you have a better experience than us!

Best,
Jon