Apex Test Code Segregation

Starting with the Summer ’13 release of the platform, Apex test methods need to be created in test classes.  (A “test class” is one marked with the @isTest annotation.)  Test methods will no longer be able to compile in the same class as your regular executable code.  This is a departure from the pattern we’ve used for the past six years, and I am sure many people are scratching their heads and asking why we are enforcing this new coding pattern.

To start, let me reassure you that your existing code does not need to change.  If you’ve created tests in executable classes, these will continue to compile in the Apex version in which they were created.  The new pattern needs to be followed for new classes written in v28.0 and above, and for any classes for which you change the version from an older version to v28.0 or later.  The check is done at compile time, so if you don’t need to change anything or recompile anything, you will not notice any difference.

There is a direct benefit to this change in our caching strategy, and this is the first part of some new enhancements to our testing framework you’ll see over the coming year.  But first, some background…

We’ve Come A Long Way

So, why the new pattern? This change is a part of our ongoing effort to support the growth of Apex.

Thanks to everybody out there reading this, Apex is growing like crazy.  We are running almost 6 billion Apex transactions a month.  This is more than double what we saw a year ago.  This kind of volume was way beyond the horizon when the language was initially created, when it was conceived as a database triggering mechanism.  Now that this volume is here, it is becoming necessary to modify some of the original assumptions and make changes like this one.

The development team spends a large amount supporting the growth in Apex transaction volume.  With every release, we verify that you still see equal or better performance to what you saw in the past.  In addition, we must be prepared for another doubling of transaction volume every several months.  We are committed to staying ahead of what you create – and you have proven time and again to be very creative and prolific.

Cache is King

One key aspect to language performance is the caching of your code for execution.  If we had to load every class for every transaction, things would not go so quickly.  Instead, we store an executable form of your class in memory to use when you next need it.  The cache is smaller than all code combined, so some percentage of requests miss the cache and have to load classes from deeper storage.

The smaller everyone’s cached classes are, the more stuff we can fit in the cache.  More stuff in the cache means fewer misses; fewer misses means faster performance for everyone.  Faster = Better, if the elementary school kids in those AT&T commercials are to be believed.

Test methods are rarely executed, relative to how often executable methods are run.  However, they are still compiled as part of the cached class, which means they are taking up valuable cache space.  Moving all test methods to test classes means they’ll only be in the cache when you are testing.  This segregation of infrequently-used code (test methods & supporting code) from frequently-used code (executable code) will help us scale to meet the growth you are driving.

Making Testing Easier

In addition to helping achieve better performance, this change is the first of several testing-related changes we are planning to make testing your Apex code more efficient.  By more efficient I mean more efficient to create tests AND more efficient to run tests.  Here are some of the planned improvements you should see in the coming year or so.

Please Note!  The following list is forward-looking and might not happen as quickly as you or I would like.  Crises occur and resources get redirected from time to time; our number one priority is service reliability.  As cool as these new features will be, they are second-fiddle to any major issues that arise.  Imagine giant safe harbor statement projected on the sky.

Test-to-Parent class linking – You will be able to have a test class explicitly related to regular class or trigger.  Currently, many of you have implicit connections by adding a suffix or prefix to a class name, but this doesn’t make managing or interacting with your codebase all that much easier.  The explicit linkage will allow us to do better UI treatments for related classes.  A direct mapping will also make the upcoming Fast Deploy feature much easier to utilize.

Shared Data Setup Methods – Rather than creating reference data for every individual test method, you will be able to specify a data setup method to be shared among all the test methods within a class.  Since an average of 70% of testing time is spent setting on database operations, this should make your testing run much faster without needing to reduce the reference data being used for individual tests.

Improved Data Creation – The Test.loadData method has probably saved many of you lots of code space in creating test data.  We will be iterating on this, offering relative date field values and generic object population.

All of these items will depend on having tests be independent of executable code, so they can be run in a different manner.  At some point, you may need to migrate some of your old tests to the new pattern to take advantage of these features.

Help Me Help You

Migrating your test methods and supporting test code to test classes will be beneficial for the platform, but it will also be beneficial to you.  As you streamline your code, you will see better cache hit rates and thus better performance for your org.  As everyone streamlines their code, everyone’s org will get a lift from increased efficiency.  You don’t need to migrate test methods immediately, but I encourage you to consider this new pattern when you are making changes to existing code.

With your help, we can continue our mission of providing a scalable execution environment that can support all of what you and your colleagues create.

Bookmark the permalink. Trackbacks are closed, but you can post a comment.
  • Anonymous

    Yes, Faster salesforce is always better .. ” Faster = Better, if the elementary school kids in those AT&T commercials are to be believed.” 😀

  • This is great news! All of these features will definitely help make tests easier to write. I actually blogged about why you should write good tests for Salesforce code last week: http://www.laceysnr.com/2013/04/three-reasons-you-should-write-proper.html

    • Matt, I’ve crossed linked to your excellent blog in my Apex Testing section of my latest blog.

  • Anonymous

    This is excellent guys, and long overdue. Glad to see the progress and plans!

  • Fantastic news!! Really looking forward to the working with some of the new unit test functionality, and of course anything that can help make our code run faster IS better.

    Out of curiosity, if we have any existing classes with unit tests mixed with business logic, would there be a theoretical performance gain by moving those unit tests out – even if the API version is < 28? Or, is the caching and performance improvement more in aggregate across all of your customers?

    • Yes, you’ll get slightly better performance. The size of the cached class is part of the algorithm that kicks out classes, and smaller items are more likely to stay cached.

  • Gareth Kavanagh

    Hey Josh, those developments sound really positive. In our organisation we have a very tight integration with a back end system and we found that the loadData method wouldn’t work for us. We were told by the developer support team that this was due to the fact that we were doing DML (for setting up our test data) in the same test class as our callouts.

    This means that we are still using test data classes (which just return Strings which contain the xml responses that we expect) is this something that the changes above will address at all? that would be a huge productivity boost for us.

    • This DML issue for mock tests will be fixed in the Summer ’13 release.

      • Gareth Kavanagh

        that’s great! Thanks so much Josh

  • andy.larter

    Josh, will there be a change to the Apex Classes page to separate apex classes from test classes? We are getting reports that our customers are reluctant to install packages as doing so increases the difficulty for their admin staff to find their own classes. Our SalesMethods package already has about 100 classes. Doubling that could cause problems for our customers.

    • Chris Noe

      One thing that we have done is create two list views on the Apex Classes page. One for “All Non-Package Classes” (set filter criteria Namespace Prefix = “”). One for “All Package Classes” (set filter criteria Namespace Prefix != “”). This separates out our classes from the classes installed by managed packages. You can do the same on the Apex Triggers and Pages tabs. This doesn’t help with your question about separating out test classes but it does help distinguish between classe, triggers, and pages that we developed vs. those installed by a managed package.

      • Andy Larter

        Thanks Chris. Thats a really good idea and so darn simple. I’ll be sure to pass it onto my customers.

  • Great stuff Josh, especially like the TestVisible Annotation described in the release notes, some good improvements to help make Apex a better language for writing great unit tests! Next up Mockito style testing? 😉 https://code.google.com/p/mockito/

  • Ashish Sharma

    Looking forward to upcoming features specially shared data setup method……

  • A couple of ideas that would help with Apex test cases I’d like to see:
    1) Allow single test method execution from an Apex Class (https://success.salesforce.com/ideaView?id=087300000007YDCAA2) and/or
    2) Run anonymous apex as if it were a test case (https://success.salesforce.com/ideaView?id=08730000000inJZAAY)

  • Great post Josh! Looking forward to seeing shared data setup methods, and also very interested in test-to-parent class linking. At that point it might be nice to be able to limit TestVisible(LinkedClass=true) ?


    The shared data setup for test and improved data creation would save the day.

  • Anonymous

    are we going to allow test class to create standard pricebook in test code? with out currently @seealldata=false annotation is irrelevant.

  • Saam Abbas

    Sounds Good.

  • Tom Briggs

    I wonder if this is the place to ask…
    I’m in need of some help regarding the writing of test script that covers enough of the below trigger that I have managed to get working on my Sandbox account.
    The trigger is to create extra assets when certain types of Opportunities are closed. The Trigger seems to run fine but I really have no idea how to start writing test cases… In order for these Opportunities to close, the Account needs to have the following completed (I’ve included some example data – They are picklists so need to be specif amounts):
    a.TurnoverBand__c = ‘<£10 million';
    a.Joining_Fee__c = '£1,920';
    a.Annual_Subscription__c = '£1,320';

    Trigger as follows:

    trigger CreateInclusiveAssetonMembershipWon on Opportunity (after insert, after update)
    for(Opportunity o: trigger.new)
    if(o.isWon == true && o.HasOpportunityLineItem == true && ( o.Type == 'A Membership' || o.Type == 'AB Membership' || o.Type == 'A Membership Upgrade' || o.Type == 'AB Membership Upgrade' ) )
    String opptyId = o.Id;
    Asset[] ast = new Asset[]{};
    Asset a = new Asset();
    a = new Asset();
    a.AccountId = o.AccountId;
    a.Product2Id = '01tA0000003N1pW';
    a.Quantity = o.Inclusive_Training_Spaces_Allocated__c;
    a.Price = 300;
    a.PurchaseDate = o.CloseDate;
    a.Status = 'Purchased';
    a.Description = 'Allocated Spaces';
    a.Name = 'Membership Inclusive Training';
    insert ast;

    If anyone could help me out on this I would be grateful!