For several years, I've worked on projects that provide software to OEMs to help them build final products with our hardware. I write tests for the software, and often perform root-cause analysis when the tests fail. On one hand, my job is to catch quality problems before our software is released, standing up for our customers to make sure they really get our best work (at times having to use the phrase "How will our customers like this behavior?"). On the other hand, I'm a customer for internal tools that help us build the software, manage and execute tests. I have my own expectations for those tools.
What customers wantThough they want perfect software, my customers must have something in their hands to start their own development. I could tell them they can have a release as soon as the next feature is complete, but they will want bugs fixed before then. They need to start writing applications to use the new feature, so they really need partial implementations first. Further, by making them wait for completion, I'm increasing the chance that they'll run into more defects or design misunderstandings when they finally start working with a release. These will of course delay their development further. This approach also increases frustration when customers want design changes.
I could promise to give them whatever is ready, every Friday at 4 PM, but I cannot guarantee that any particular feature will be complete, or even that the software will be usable. Our customers will start ignoring the regular releases, and request one to be promoted to some higher status, with an assumed guarantee of quality.
I will fail them if I am absolutely focused on quality or timeline. But which one is most important?
What my customers need is something they can start with now, with regular releases that enable them to make continuous progress. I can accomplish this by setting a release cadence that allows for thorough testing, while being willing to delay a release by a day or two to fix bugs that would slow my customers down. They are happier to be following our development process closely, while we provide them with continuously improved software.
Even between development and testing organizations, this applies. Effective test development closely resembles our customers' application development. I have been involved with projects where I had no access to code under development until it was supposedly complete. At that point, I was expected to have test development complete and be executing tests. In reality, I had insufficient insight into the development process, testing suffered (depth and schedule), and I even encountered hostility toward fixing early design issues.
Quality is a goal; regular releases could be the meansWe release software in multiple versions because quality is a long-term goal. Each version should bring us closer to that goal, by being better than the previous version. Perfect software is unachievable without rigid specifications, which are unrealistic outside of space programs.
If we define quality as the suitability of software to accomplish its intended task, the quality actually decreases as defects are found. Undiscovered defects are still bad, but they become worse when they impede progress. In other words, quality decays as releases age. We can fight that by writing perfect code, or by making incrementally improved, frequent releases.
Our goal should be to enable our customers to make continous progress.
