New Regression Testing Process for ISVs Using Pre-Release
As salesforce.com prepares for the Summer ’14 release, we have some exciting updates to share. ISVs have frequently asked us to provide better pre-release environments so that they can do more thorough testing ahead of each major Salesforce release. We have good news! We’ve made significant improvements to the pre-release environment to address these issues.
As salesforce.com prepares for the Summer ’14 release, we have some exciting updates to share. ISVs have frequently asked us to provide better pre-release environments so that they can do more thorough testing ahead of each major Salesforce release. Prior to Summer ’14, we advised ISVs to use sandbox orgs to do their pre-release testing. Unfortunately, sandbox orgs are upgraded only a few weeks before the production release, leaving ISVs little time to test and report issues. Sandbox orgs also don’t allow ISVs to easily test the upload and installation of their managed packages. We have good news! We’ve made significant improvements to the pre-release environment to address these issues.
Available now, there is a new pre-release environment where ISVs can get an early preview of Summer ‘14 and have the ability to test your package before your customers are upgraded. You can sign up for new Salesforce organizations on this environment by going here. You can sign up for any Salesforce edition except Unlimited and Performance editions.
Please note that the pre-release environment is:
- Available for testing only
- Not intended for development purposes. Your pre-release Developer Edition orgs and other pre-release orgs may be periodically deleted in anticipation of the next release.
- Not intended for distribution. You cannot submit an app built on this environment for Security Review or AppExchange.
You are in the best position to determine what to test in your package, but here are some high level suggestions:
- End-to-end regression testing of your app.
- Install, upgrade and uninstall different package versions in editions that you support (for example: PE, EE). The install urls of your official packages should work and you can install those packages in this environment.
- Create a test packaging org (Developer Edition org) on pre-release, copy the metadata (associated with the package) to this org, create a test namespace and upload versions of this package. Then try install, uninstall, and upgrade operations with this test package. Please note that you have to select a unique namespace for your test package as you cannot reuse the current namespace of your actual package.
- Try out changes that you are planning to make with the test package and see if anything breaks.
- Test push upgrade and install apex script functionality.
- Spin up patch orgs from test packaging org (on pre-release) and test out upload, install, upgrade and uninstall operations.
During the release window when the pre-release environment is on Summer ’14 and the rest of the instances are still on Spring ’14, the following hold good:
- Packages built on other instances can be installed in pre-release environment using the install urls.
- Test packages built on pre-release environment can be installed on pre-release environment, but cannot be installed in other instances.
If you see any issues, log a case with partner support. Please ensure that the title of the case starts with “Pre-release Testing.”
And be sure to follow each release on our new Partner Community site at http://p.force.com/releases. This site is specifically for partners, and contains links to the webinars, resources, and information you need during a release – as they become available. We also have a “Releases for Partners Chatter Group” where we post when assets become available (like the Release Notes), and you can collaborate with our team during a release cycle. Remember, if you discover a specific issue during regression testing that impacts your app, you should log a case.