Salesforce.com has always been a highly innovative company that isn’t afraid to think outside the box. That was true when we were 4 employees, and it’s true now that we are over 15,000. Our innovation isn’t just based on features and technology. We also aren’t afraid to go against the status quo when it comes to testing. Salesforce.com was founded on a culture of quality and customer advocacy. We truly understand that we exist because of our customers and we need to provide the best possible experience for them.
What does providing the best possible customer experience mean? It means that testing and quality are continually evaluated and promoted as part of the process from the very beginning. This required that we approach quality in a new way.
The old way worked like this: A development team wrote the code and threw the code over the wall to the QA team. The QA team found bugs in the code, threw the tested code back over the wall to the dev team… and on and on it went. An infinite game of volleyball. An extremely long feedback loop. As a result, some companies started treating their quality assurance people like second-class citizens. They were not on the “same level” as a developer. The “wall,” in other words, became a barrier.
A common scenario in projects done “the old way” is as follows. You’re given a project with a six-week deadline. You set aside four weeks for development and two weeks for QA. But then development runs a little longer than you planned (humans really are bad estimators), and you can’t push the deadline. What happens? The testing cycle gets condensed. You make sacrifices in the testing process in order to meet the deadline. But the end result is a subpar outcome. Now, you have problems. Externally, the client isn’t happy. Internally, team members are pointing fingers. It’s a bad situation for everybody.
(Yes, agile addresses some of this, but typically when it comes to large projects, it means that the sprint is condensed instead of the testing cycle.)
So how did we break down that barrier to get to shorter and more effective feedback loops about quality? We chose to unify our feature teams. What did it mean to have a unified feature team? It meant that a feature team consisted of developers, QE, doc writers, product managers and several other roles who were 100% dedicated to that specific team. People didn’t throw anything over the wall anymore because there was no wall to speak of. Quality was there from the get-go. They were involved in the design process and the design reviews. Quality engineers helped developers grow and mature a quality mindset, and developers helped quality engineers refine their development skills. We essentially became partners and everybody began learning from each other. It worked. It still works.
However, we do more than just embed a quality engineer into a feature team. We encourage our quality engineers to write feature code. We also encourage our developers to automate functional tests, not just unit tests. The QE will typically extend those test suites based on the feature dependencies, complexity, etc.
Not all quality engineers want or are able to write feature code, but that option is there if they so desire. By not restricting them from writing feature code we are able to remove one of the factors that contributes to the second-class citizen perception. Of course, QEs don’t just write test automation and feature code. They also perform exploratory testing, organize test bashes (all hands on deck for a fixed period of time trying to find defects in a piece of software), and continually evaluate quality best practices.
How does this impact the condensed testing cycle example given earlier? Since there are dedicated quality engineers on the team, testing is done from the beginning. It is built into the process. There is no longer an “us vs. them” divide between software developers and quality engineers. We are one team working toward one goal – the best product possible for our customers.
Embedding quality engineers and developers on the same feature team is great for collaboration and quick feedback loops, but it does present a management challenge. While both quality engineers and software engineers are engineers, they need different mentoring and career advice. To address this, we provide different management chains. This allows the unified team to function as one, yet still lets the individual engineers advance along the career path tailored for their discipline.
I’ll finish with the most important thing we’ve learned about unified teams. For a unified team to really work, you have to have the right people in place. In the end, a team is only as good as its members. If you have software engineers who are unwilling to write unit tests — or if they believe quality engineers are second-class citizens — they aren’t going to function well on a unified team. They’re going to drag down the ship. You have to overcome those challenges, which can take some time. Even here at salesforce.com, it’s been an evolving process. But for us, it’s been well worth the effort.
