Apex test classes in Salesforce are a way to unit test our development. In Salesforce’s own words, they “facilitate the development of robust, error free code and configurations”. Yes, configuration too, because we want to test our validation rules, processes, workflows, approval process, etc – not just our Apex code.

Unit tests are defined by class methods which take no arguments, commit no data to the database, send no emails, and are flagged with the @isTest annotation in the method definition.

When and why we need to run all tests

For enterprise organizations and medium to large scale implementations, we recommend having a Continuous Integration (CI) process. You can find more information about this here:

• Documentation
• Blog post
• Continuous Integration Using Salesforce DX Trailhead module

Developers who work on Scratch Orgs or Developer Sandboxes check in their code into a source control repository branch. This process simply takes the changes from a source control repository, and merges the same into a build sandbox (usually on a nightly basis). This ensures that the metadata checked into the source control repository is the following:

Valid (deployable to a fresh sandbox). Validate the deployment in checkonly mode using sfdx mdapi command.

Does not cause any test class failures (to new or existing test classes). Validate the deployment with TestLevel as RunAllTestsInOrg mode using sfdx mdapi command. We can also run all Tests in Salesforce, details here.

Test covered to a test class standard defined by the organization (read test results and coverage information).

Conforms to code scan policies defined by the organization. Perform a code scan with a tool (e.x. – Clayton, PMD, Checkmarx, SonarQube). Possibly define custom rules specific to Organization standards.

Since we need to ensure that no existing test classes are failing on a new deployment, running all test classes in an org becomes important. Now, as we’re talking about large scale implementations, the time to run test classes can be huge (in order of 10-20 hours, or more). This could be a blocker in itself, considering there are multiple deployments and releases that could be lined up and will need to wait for this step to complete.

We need a way for the solutions to be moved up and released quickly to end users. In other words, a faster time to market for new features. However, we also need to ensure quality, regression, and that standards and best practices are followed.

Some terminologies first

There are two factors to be considered when talking about the execution time of unit test classes.

• Mode – Real Time / Asynchronous
• Type – Serial / Parallel

How we can speed up testing

Run tests in Parallel (from UI)

This is a good option, but it can present data contention issues.

• When test classes try to update the same records at the same time, locking issues occur.
• When test classes try to create/update child records, parent records get locked.

The use of seeAllData is discouraged, as it can make row locking issues substantially worse. It can also make setting up CI environments dramatically more complex.

A common example is creating users for an existing profile. If multiple test classes are creating users for Unit Tests, using the same profile, the Profile record will face lock issues and the test will fail in parallel.

Add isParallel = true to individual test classes

An added parameter to the isTest annotation can be added (For Salesforce API version 24.0 onwards). See more details here.

This is a workable option. However, based on the data getting created and updated, it may not always be possible to add this. As described above, data contention issues might prevent us from adding this to the test classes.

But, when it comes to running the test classes from an automated CI process, UI / Dev Console cannot be an option.

What else? APEX to THE rescue!

We need an automated asynchronous way to trigger the test classes, and to speed it up as well.

For large customers, we usually have many dev sandboxes kept for the purpose of Continuous Integration. The idea here is not just to utilize Apex to start the test classes asynchronously, but to also divide the test classes across multiple sandboxes for faster execution.

Steps – Apex

1. A webservice in Apex to take two parameters – Offset, Chunk Size
2. Query the Test Classes using Offset and Chunk Size
3. Add them to an ApexTestQueueItem list and Insert, and Return the TestRunResultID
4. Another Webservice to return results based on the passed TestRunResultIds

Steps – Automation Server (Jenkins) / Script

1. Decide on the Offset
2. Chunk Size = number of Sandboxes available for CI process
3. Call the same Webservice in different sandboxes
4. Poll back each to fetch the results

Code Snippets

As an example, let’s say an Org has 450 test classes and we dedicate 3 sandboxes for CI testing. Each sandbox will run 150 classes, with an offset.

Sample WebService Apex Code for getting invoked from the Automation Server. You can call the same web service for each sandbox with a different chunkNumber, to start the test classes in parallel.

Warning Note : OFFSET has a hard cap of 2k items. If the org is massive, and has thousands of test classes, consider another approach (Order by ID, and pass the last queried ID back with the JSON. Pass the Last Used ID again when calling the method for the next chunk).

The chunk size you choose depends on the number of sandboxes we can spare for CI.

Using another web service the Automation server can poll the sandbox for results, for which we’ll need the “TestRunResultID”.

In another Web Service, pass these Ids to fetch the results from each sandbox.

This will give us a massive improvement in time taken to run all Test classes, thereby speeding up the CI process and time to market. See how our previous CI diagram now changes to show this:

Summary

This article describes an approach for achieving parallelism in running all test classes in an org, typically useful for build validations in an automated CI – CD process. It involves dividing the test classes to run in multiple sandboxes using Apex based web services.

References

1. ApexTestQueueItem – https://developer.salesforce.com/docs/atlas.en-us.api.meta/api/sforce_api_objects_apextestqueueitem.htm
2. Testing Best Practices – https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_testing_best_practices.htm
3. Running Unit Test Methods – https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_testing_unit_tests_running.htm

About the authors

Ritika Dhandia, Senior Program Architect
Certified Technical Architect
CSG Services, Bangalore, India

Imran Rasheed, Senior Program Architect
CSG Services, Bangalore, India

Special thanks to Jeroen de Vries for providing great insights on this subject and Sambit Kumar Mund who assisted with development.