Code Coverage and the Force.com Developer Console
With Winter '14, there were two areas of change in the Developer Console affecting code coverage: How it is viewed in the developer console, and issues with code coverage itself. Josh explains.
Many of you have encountered issues with code coverage results following the Winter ’14 release. In fact, you’re probably here because you saw the title and thought, “Yes, I am having problems with code coverage in the developer console, what’s with that?” and clicked through. You are not alone, and we are not unaware.
First off, I would like to make sure that we separate out two different changes that have happened in Winter ’14: Moving code coverage viewing to the developer console, and the fact that there are issues with code coverage. The two originate in the same place, but the developer console has nothing to do with the incomplete coverage results and missing coverage functionality you may be experiencing.
The common origin of these is the addition of code coverage results to the Tooling API. We had some good reasons to do this feature! First and foremost, we are always looking to make testing and debugging perform better, and changing the way in which we store coverage is going to help us scale and perform better, today and moving forward. Beyond that, having code coverage results in the Tooling API will allow us to include them in the Eclipse plug-in, and will allow tools like Brain Engine and MavensMate to include code coverage as well. If you are not a fan of the developer console for one reason or another, having the coverage information available in the Tooling API will eventually allow you to avoid the console for viewing code coverage.
Note: You should Run All Tests to clear out the old-school code coverage results. If you run piecemeal, you will end up with old-school and new-school results at the same time. This is like going into the past and running into yourself (or Marty McFly). It’s bad. Run the tests to get your org to a consistent state.
Because of the structural changes to the underlying objects, code coverage no longer appeared in the old setup UI pages. In general, we are moving away from the setup UI pages as our browser-based development alternative and towards the developer console. It’s hard to support multiple tools that do the same thing. Doing so takes time away from the other initiatives you’ve asked us to work on! We’re not trying to delete the setup pages tomorrow, but we are not putting effort into enhancing them. Since the changes to code coverage were structural, the existing setup UI functionality was going to need refactoring to continue to work. With a finite team and infinite plans for improvement, we chose not to do the refactoring work, which would only provide duplicate functionality to what is available in the developer console.
As unluck would have it, the code coverage structural changes didn’t go as planned. For the past 5-6 weeks, we have had half of the team trying to get things aligned. This is far slower than usual, but we are being far more careful than usual. When you run tests, you do NOT want us to commit your test data. But you DO want us to commit the code coverage results. We are making sure that we never make the mistake of committing test data while committing the coverage results. The coverage results you are seeing today are representative of us erring on the side of saving too little, to avoid accidentally saving too much.
The good news is that we believe have the problem licked. You should start to see better code coverage results now. This means you should see all classes and triggers in the test tab in developer console, and you should be able to see red/blue in the class and trigger files. The developer console has always been able to show these things, but only if the data are available to be shown. The coverage results haven’t been available, and we believe this is fixed now.
I know that the developer console is not perfect, and I know that developers have experienced issues in using it. Believe me, I would love to make it perfect, and we are always working in that general direction (even if we accidentally take a temporary wrong turn like this one). Please keep visiting; with every release, we are improving the experience and improving the performance. If this was your first experience with it, I apologize that your first impression was bad; please understand that the issue was not in the developer console, but in the underlying data structure changes that the console relies on.
You should be returning to seeing your coverage, and I think you will agree that the developer console provides an upgrade to the setup UI pages for viewing those results.