Concurrent Apex errors on the Salesforce Platform often arise from several common factors. These include prolonged database transaction times caused by inefficient SOQL queries or an extremely high number of concurrent DMLs taking more time on database layer or increased runtime due to poorly written apex code consuming high app cpu time and it could also be due to setup changes or Apex deployments that can lead to cache invalidation, among other factors.
To help address concurrent Apex errors, Scale Center has introduced a new approach for triaging and diagnosing them. Understanding the root causes of these errors is the first step, and in this post, we’ll walk you through how to use the Concurrent Apex Report in Scale Center to help you take appropriate action.
Overview of concurrent errors
The Agentforce platform uses governor limits to ensure fair resource distribution. Exceeding these limits, such as the one for concurrent requests (synchronous Apex requests longer than five seconds), results in unrecoverable runtime exceptions. The default limit for these long-running concurrent transactions, capped at 50, is now based on license type and count (minimum 10). This limit prevents service disruptions, enhances stability, improves resource allocation, and safeguards user experience by avoiding prolonged page load times (over five seconds). It’s important to consider these limits during application design for scalability.
How to analyze and action on concurrent Apex reports
Troubleshooting concurrent Apex errors in Scale Center has been improved with a redesigned investigation process. This redesign will provide users with details around potential causes and also provide clarity on the tech stack that’s primarily contributing to the concurrent errors, along with a list of entry points. Let’s deep dive into this.
Understanding potential causes
The widgets section highlights potential factors contributing to these errors.
- Throttling: During high system load, Salesforce may throttle requests, potentially causing concurrent Apex errors. Scale Center now helps identify the root cause of these throttling-induced errors by displaying the affected Uniform Resource Identifiers (URIs) and providing a detailed explanation of the throttling reason. This enhancement gives users more specific information to diagnose and resolve these errors, potentially optimizing their applications or reducing throttling triggers, thus improving troubleshooting and platform transparency. Contact Salesforce Support for further details.
- Setup or deployment invalidation: Schema and metadata changes, whether deployed or configured directly, invalidate cached metadata. This invalidation forces the system to regenerate data, increasing processing time for synchronous requests. The resulting temporary runtime increase can lead to concurrent Apex errors if multiple synchronous requests occur during cache rebuilding as prolonged Apex transaction times may cause overlap or exceed governor limits. Understanding this relationship is crucial for Salesforce application stability and performance. Careful planning of schema modifications and monitoring application performance can help prevent and resolve these errors.
- Error and User Counts: Remaining widgets provide the count and the total number of unique users impacted by these errors.
Identifying areas for optimization (tech stack analysis)
The report then focuses on the tech stack, showing where time is consumed during long-running transactions. This helps identify the layer that requires tuning. For instance, significant time spent on the database indicates that optimizing SOQL and DML statements within the identified entry points is the recommended next step to mitigating concurrent errors.
Pinpointing problematic entry points
Below the time distribution widget, Scale Center surfaces the top entry points running for over five seconds that contribute to these errors. An algorithm analyzes entry points based on their count and runtime, categorizing them as Critical, Major, and Minor to help prioritize optimization efforts. Each table details subsequent actions that users can undertake. Each row also highlights the dominant stack for each entry point in red. This indicates that for each entry point, the columns highlighted in red are the areas requiring fixes to reduce the overall runtime of those entry points.
Developer/Architect actions
Developers and architects should use these identified entry points to run debug logs and gain detailed insights for further code optimization. You can instrument Apex code around these entry points with logging using the Nebula logging framework and then identify the call stack to determine troublesome queries and DMLs, etc. For this blog post, we’ll use Apex Log Analyzer. Turn on the Debug Logs with Finest Mode filter, re-run just the specific use case that triggers these entry points, and download the log to visualize it using the Apex Log Analyzer. Hover over the Analysis tab and look for the section where most of the time is being spent.
This way we can determine the actions, exact SOQLs, and entry points of the code that are contributing to high database time and appropriately remediate.
Analyzing database time contributors
Database time is often a significant factor in long-running transactions. Scale Center now details the elements contributing to database time.
- Top SOQLs and DMLs: The top five SOQL and DML operations (Insert, Update, Delete) that are major contributors are listed. Optimizing these by applying best practices like using indexed fields and appropriate filters can reduce database time and, consequently, concurrency issues.
- Row locks: Database wait time can also result from row locks, which can significantly increase database time for synchronous requests and lead to concurrent Apex errors. The report now identifies transactions experiencing high row locks and the entities involved. Resolving these row lock issues will also help reduce platform concurrency.
Here are possible ways to fix these rowlocks:
- Data model changes: Determine the objects that are frequently involved in locking conflicts and identify relationships between contending objects and look for locks arising due to the data model itself.
- Usage of FOR UPDATE: Investigate conflicting transactions. Analyze the transactions that are attempting to acquire locks on the same objects simultaneously and look for any deliberate locks introduced from the application using FOR UPDATE.
- Reduce transaction size: Break large transactions into smaller batches to reduce the likelihood of row locks.
This improved concurrent Apex report presents findings and actionable key takeaways for users to address concurrent errors. Recommendations regarding database time, Apex execution time, and other metrics will provide specific guidance on optimizing application elements.
To ensure continuous improvement, an in-app feedback mechanism, consisting of thumbs up and down buttons, has been implemented. We encourage you to provide feedback to help us assess the report’s effectiveness and inform necessary enhancements.
About the author
Anand Vardhan is a Product Owner for the Scalability Products Team, helping customers develop scalable solutions on the Customer 360 platform. Anand designs features for Scale Center and Scale Test products. His background includes performance and scale engineering, server-side optimization, Lightning, application design, caching, and managing large data volumes. Follow Anand on LinkedIn.
