Interesting feedback from my QA post below. (All of the quotes are paraphrases)
Dan Sugalski wrote essentially:QA People don't need to know how to cook, they only need to be discerning eaters!
MRH: I don't think so. If the QA Analyst is a "business person" who can only evaluate the food once it's been cooked, then you can't catch errors until the end of the process - even with an iterative model, that's too darn late.
MRH: The QA person needs to find errors immediately after - and preferably before - they are introduced. To do this, the QA person needs to know how to cook, and customer relations/front office skills.
Tessa Welsh: You sort of implied that the QA person should continue to write some code ...
MRH: Yes, I did. Two years after the QA person is promoted, the new hires won't know that he's a senior coder - and his skills won't be as current. So, to truely "Walk the lifecycle", when it gets time to code, the QA guy writes some code. The different is that he can no longer allow himself to be consumed by coding - it must move from "who he is" to "something he does." Of course, this is a natural transition for many coders anyway.
Tessa Welsh: So, would thier be many QA people, then?
MRH: Perhaps, in a large organization - any given project would have at least one person playing a QA role, and would be involved from project inception. But, if you push too far in that direction, you end up with "everyone is a QA person."
Shawn Stanford: Exactly! so why not just instill the "attiude of QA" into the entire group instead of having a separate "QA" oganization?
MRH: Actually, that's a pretty good idea. The original post was to address the question "If you insist on having a QA person, how can you make that actually work?"
MRH: The need for a QA person arises because of a conflict of interest - generally "Ship Date" Vs. "Quality". It's an insane conflict of interest - if you cut quality early, you generally add hours onto a project - but it's a conflict, non-the-less.
MRH: A QA person, then, is not motivated by Bonus, Review, or Ship Date, but is instead motivated by Quality and pride-in-work. (Notice this is a very formal divorce of project management and QA. Under my proposal, they would not be the same people - the PM would be on the management track, and the QA person is on the Technical Track. But more about that in another post ...)
FWIW, quality assurance is not just the process of finding bugs so they won't effect end users. Its also the process of keeping the system at high quality throughout development to reduce development time so the programmers aren't spending all their time chasing bugs. It works much better when you do it as you go. It also works much better when the person testing it knows something about how it works. You'll have to take my word on that because I'm a bit rushed, but this is my experience.
So with that in mind, yes, everyone on the team should be thinking about quality, writing their own tests, etc... have the process going on at all times.
This is not to say a dedicated QA person is not useful. While I'm a huge advocate of developers writing their own automated tests, sometimes there's stuff that's just too hard to automate. GUIs, web forms, etc... Its good to have a meat puppet to do these hard things that often get forgotten. Its also good to hae someone whom its their job to play User Advocate. Thinking like the user, using the system like a user and complaining like the user. This could even be considered something akin to XP's On Site Customer.
If your system has already been kept at an internal high quality by the developer's own QA process then the external QA won't find so many bugs and there won't be so much tension between developers and QA. Its also less likely they'll find a bug that causes you to go back and redo three months of design work. Finally, if your code is kept at continual high quality and the developers practice an incremental release strategy, ie. release something that is usable every N weeks, then the QA doesn't have to wait until the end but can be done as you go.
A dedicated QA person is a valuable addition to a development team already handing their own testing internally. Not a substitute. They think differently. They approach the problem from a different perspective. They have the time to work on it. They are detatched from the labors of building what they are critiquing (ie. its hard to admit that the system you just spend two months on is crap).
This being said, I've never had the luxury of a dedicated, non-programming QA person, though I have often played the role of a developer who's primary job is QA.
Re:A QA person is a valuable *addition*
lachoy on 2003-08-22T22:17:06
I think another great job of a QA person is creating whatever frameworks are necessary for testing and making them easy to use by the everyday developer. This serves a couple purposes:
a) In theory, the QA person is more familiar with all types of testing (unit, functional, acceptance, etc.) and knows more about common usage, utilities, tools, etc. (And if they don't siccing them on this task will make it so!) So you take advantage of specific knowledge.
b) It scratches the itch most good programmers have to create something. Creating frameworks must be fun to lots of people, otherwise we wouldn't have so many reinvented wheels. So letting QA people do "real" development -- centered on testing, but still real -- keeps them intellectually happy.
c) It ensures that the biz programmers don't complain about tests being too difficult to write. The fewer excuses for that the better!
d) It forces them to keep up-to-date on a sizable area of research and development and provides opportunities not only for learning (e.g., attending agile development conferences since they tend to have lots of talks/workshops on testing) but also for teaching. And teaching looks good on the resume!Just to be clear:
heusserm on 2003-08-25T11:28:37
FWIW, quality assurance is not just the process of finding bugs so they won't effect end users. Its also the process of keeping the system at high quality throughout development to reduce development time so the programmers aren't spending all their time chasing bugs.
YES! That's what I'm arguing for. That's why I said the QA person must follow through the "Entire Lifecycle." It doesn't matter if you are good at catching bugs if the codebase is a pile of trash and you don't start testing until coding is complete.
A dedicated QA person is a valuable addition to a development team already handing their own testing internally. Not a substitute
Yes! Ten years ago, in "Writing Solid Code", Steve MacGuire aruged that separating the responsibility for the quality to QA, without separating the authority, was very very bad. He was right.
If the QA person is a coder with credibility, that person can work with the coders to make sure they use modern techniques - an advocate and evangelist of quality, not just the scap-goat. Then he becomes and assist and backup, again, not just the scape goat.
It sounds like Schwern is arguing for the same points as I am. Was I not clear in the blog, or am I just dense?:-) Just TBC: 2nd Attempt
heusserm on 2003-08-25T11:31:16
Ugh. Formatting was messy in last reply. Take two:
FWIW, quality assurance is not just the process of finding bugs so they won't effect end users. Its also the process of keeping the system at high quality throughout development to reduce development time so the programmers aren't spending all their time chasing bugs.
YES! That's what I'm arguing for. That's why I said the QA person must follow through the "Entire Lifecycle." It doesn't matter if you are good at catching bugs if the codebase is a pile of trash and you don't start testing until coding is complete.
A dedicated QA person is a valuable addition to a development team already handing their own testing internally. Not a substitute.
Yes! Ten years ago, in "Writing Solid Code", Steve MacGuire aruged that separating the responsibility for the quality to QA, without separating the authority, was very very bad. He was right. If the QA person is a coder with credibility, that person can work with the coders to make sure they use modern techniques - an advocate and evangelist of quality, not just the scap-goat. Then he becomes and assist and backup, again, not just the scape goat. It sounds like Schwern is arguing for the same points as I am. Was I not clear in the blog, or am I just dense?:-) Re:Just TBC: 2nd Attempt
schwern on 2003-08-27T21:23:42
I think I was just thinking out loud on what you said.
