Editors Note: This post is part of a “Guest” series entitled Enterprise Architecture with Force.com. Our guest blogger, Greg Cook, is a managing partner of CloudPremise and currently holds all seven Salesforce certifications.
Establishing an effective Environment Management strategy is critical for utilizing the Salesforce1 Platform. Salesforce.com has already established itself as a leader in cloud technology and innovation, however some aspects of dealing with the platform still require good-old-fashioned IT management skills. Salesforce provides a LOT of content around governance, environment management, and change management. However I have noticed at multiple clients the information and options are sometimes too plentiful. Many customers want to be given a recipe and a framework as opposed to designing their own solutions from best practices. This installment describes a typical environment management strategy that can be emulated on many Enterprise projects.
What do I mean by Enterprise Projects? This article applies to your organization if you meet the following criteria:
If this sounds familiar than you are in luck and this article was written for you. If your projects do not sound like the above list then many of the recommendations in this article may not apply to your environment (but feel free to keep reading!)
The first thing to understand about environment management on the Salesforce1 Platform are some of the differences to typical environment management.
In traditional software development your pristine environment can be housed within source control and configuration management tools. In order to maintain a stable production deployment the emphasis is using configuration management techniques to maintain your code, configuration files, and deployment scripts. Production can be rebuilt, deployed at will, and rolling back to a previous version of the code is possible. However in Salesforce it is not possible to take production offline or to deploy the entire application in a big-bang deployment event. Instead you are trying to migrate “differences” between your environments into production. You can utilize source control and configuration management tools with Salesforce, however your production environment is (almost) always live regardless of the state of your configuration management procedures.
Salesforce supports a much higher volume of “production changes” due to its metadata based configuration design. Therefore many changes can be done safely in production without the need for a code deployment. While this has the effect of producing immediate value to the business (Admin Hero’s anyone?), it can have dreadful consequences on future deployments if the right controls are not in place.
As I already described, it is not possible to move the entire code base from one state to another (i.e. Dev –> QA –> Prod, etc). The nature of Salesforce’s migrations means that the emphasis should not be on your configuration management artifacts, but rather the emphasis should be on environment synchronization. You may have the best code and most thorough migration plan – but if your environments are out of sync with production changes you will have a VERY difficult time deploying changes of significant complexity.
This is NOT uncommon on traditional projects. However there are specific pain points around Salesforce sandboxes. The most noticeable difference will be the database size. Many customers do not even have a full copy sandbox, let alone multiple. Most sandboxes will not come loaded with data, and those that do may need changes to support testing and integration. There are other differences outside the scope of this article, however the main pain point remains: there will be overhead in each sub-prod environment to manage data and metadata. This overhead increases as the number of sandboxes increase.
Salesforce is releasing significant new features three times per year. While many of these new features can be managed and maintained using the APIs, some changes are not possible except through the web-browser interface. That means that maintaining a strict documentation set is VITAL for successful migration and environment maintenance.
With those prerequisites out of the way it is time to introduce my reference model for Enterprise Environment Management:
Let’s walk through the diagram and explain the concepts and purpose of each component, including the types of change to each environment.
As mentioned, this is your only pristine environment. Refreshes are only available from production into the sandboxes (reflected by the red lines). Therefore your code and configuration changes are “swimming upstream” – trying to migrate successfully to the production environment from the lower sub-prod environments.
Each developer on the project will need an environment. Typically these environments are Developer Sandboxes. The care and feeding of this environment is usually done by the developer themselves. These environments typically only exist for the length of time it takes to migrate a feature to production, and sometimes even less. I call these “unmanaged” environments because typically they will not be maintained by your environment manager. A developer should complete Unit testing and apex-based automated testing in this environment.
Each project needs to have an environment that can isolate changes from other projects. Many projects may be occurring at the same time and on different release schedules. These managed environments are the holding grounds for project features that are not ready for release. This is where I would have testers complete Functional testing. I call these “managed” environments because typically they are (or should) be maintained by an environment manager (a system administrator with responsibility for environment integrity).
Note: I have included Citizen Developer Environments in this region. Citizen Developers would be small projects that do not require multiple developers/sandboxes. However it is important to have the Citizen Developers migrate into production using the same path as your enterprise projects (i.e. the release train).
I use the concept of a release train to describe the activities necessary to migrate successfully to production. A release train can be a simple schedule detailing when changes can be made to each environment, or it can be a complex automated solution. You can read about release trains more here. For the purpose of this article I will use a very simple example of a release train:
Let’s look deeper at each stop on the release train:
This is the environment where parallel projects comes together. This is also a good spot for system integration testing. Automated apex tests should be continually run in this environment to ensure a project has not broken something. Consider the use of more sophisticated automation tools to ensure your business processes are protected. Selenium is a good open source tool for testing the user interface. You can also test APIs and integrations with tools like SoapUI or Postman.
This environment should be reflective of your desired production state. This is where I would recommend acceptance testing. You should be able to conduct end-to-end business processes in this environment. I typically would use this environment for my Full Copy sandbox as it will change less often than other environments. Load testing and performance testing can also be validated in this environment due to the larger database size.
This environment’s sole purpose is to ensure your deployments to production will succeed. I would definitely not use a full sandbox here. In fact I would refresh this environment prior to EACH production deployment. That way you know you have the most up to date configuration from production. I hope you followed my advice and used change sets or ANT throughout the migrations. That means you have a comprehensive deployment manifest. I also hope you followed my advice and kept detailed documentation for all manual changes. That means you have a comprehensive set of deployment instructions. After deploying to stage you should conduct smoke testing to ensure your deployment was successful.
If your deployment was successful then you have a very high assurance that your production deployment will also be successful. Repeat the steps used in the stage deployment during the production deployment.
Training is a difficult issue due to the nature of environment setup and tear down. It is also difficult to determine whether to have the code that is ABOUT to go live versus the code that HAS ALREADY gone live. So you have a few choices. You can decide on one, multiple, or all of the following:
No matter what you do, someday a Sev 1 bug will reveal itself. So where should you make the fix and how do you get it deployed to production as quickly as possible? My recommendation in these situations is to generate a new sandbox only to respond to the Sev 1 defect. The configuration or code can be fixed, immediately tested, and immediately deployed to Production. This should ONLY be done under dire circumstances as you are bypassing many of the controls I have outlined in this article. Just make sure to reapply the fix into your sub-prod environments as necessary.
As you can see there is quite a bit of activity that needs to take place in order to orchestrate a pristine environment plan. And this article only covers migration actives, not even production support issues like data backup, archive, etc. I hope it is clear by now that I HIGHLY recommend hiring a full-time Salesforce.com Environment Manager to plan, execute, and monitor all of the items outlined above. An Environment Manger would be similar to a System Administrator – however their focus is very different. A Salesforce system administrator is typically focused on users. An Environment Manager would only be focused on the technical infrastructure.
When consulting with enterprise customers I often encounter issues like “how to maintain sandboxes” and “how to ensure smooth production deployments”. And typically none of these customers have a dedicated Environment Manager who is following a robust strategy and executing these detailed tactics. An investment in a dedicated Environment Manager will allow your company to scale Salesforce1 must more effectively.
Environment management is one of the most difficult and under-realized aspects on enterprise projects. Companies often under estimate the amount of work necessary to orchestrate the movement of data and metadata throughout the environments. The right strategy, the correct resources (human and technical), and effective processes can radically accelerate and improve your consumption of the Salesforce1 Platform. You can harness the agility of Salesforce with the predictability of enterprise class deployments. If you are looking for help on getting started, consider obtaining the help of a Salesforce.com Certified Technical Architect to help your company define and execute upon an Environment Management strategy.