Precursor: I don't like artificially created generations or bogus models. In the post below, I use this first/second/third generation methodology paradigm - mostly because I want to get this post out quickly. It's a contrivance I'm using to describe generalities - please don't think i'm taking myself too seriously. :-)
Also note that methodology is just a fancy way of saying "the way we think or conceptualize our approach to solving problems around here." I use the word a lot in the post, I'm open to other options for word choice.
And now, on with the show ...
Sean McMillan and I were talking about the evolution of methodologies today at lunch. He had some really interesting observations.
Think about the first methodologies that came out. (Waterfall, various customized versions of the waterfall, and, eventually, the uber-model, the CMM) Try to read between the lines. These models were all about FEAR: Fear that management does not have control, fear that the customer will not accept the finished work, fear that the project will be late, over budget, etc, etc.
So these first methologies addressed that fear in a number of ways: They froze requirements, they forced requirements "sign off", they insist on artifacts that have review, are version controlled, audited, etc - they tried to make software development "defined, repeatable, stable, and measured", they try to institutionalize process so that management is not dependent on the heroics of any individual, they eliminate "single points of failure", etc.
A lot of those early methodologies are monday-morning quarterbacking: "We noticed one time we had this one problem, so we are going to insert this new step in EVERY project to eliminate the risk of that problem."
Sean made a methaphor of a pothole methodology - We notice that we ran over some potholes and it gave us some car trouble, so we place gates around every pothole we see. Requirements Review, Code Review, Test Case Review, Requirements Signoff, Configuration Management, have a separate team move code into production ... this list goes on.
As the gates constantly propigate, we find that slowing down to drive around the gates becomes more of a hassle than we had before!
This makes complete sense to me. In "Quality Software Management" Jerry Weinberg makes the assertion that the "level 2 repeating" organization was a fantasy created by non-technical management, wheren they manage to wrestle control back from the developers. After all, at level 2, any individual developer is expendible; the process is king.
Sidebar: I've read a LOT of the stuff that came out of the academic/government sector about methodologies. One of the things I consistently notice is the use of big words to sound important and the lack of philisophical depth. This is a real problem if you're trying to develop software. On the other hand, if the REAL goal is to remove fear from executives and give them a feeling of control ...
Eventually, people begin to realize that dealing with fear is not good; instead we need to be adults, taking calculated risks and living with the consequences. We need to drive out fear in the organization (think Deming) and instead live with uncertainty. This brings us to what we called (again, lack of a better term) "second generation" methodologies that are a backlash to the first. XP is a great example - the whole point is simplest process that could possible work, combined with just enough safety nets that you can live with a little risk. Stop living in fear that the developers won't hit some meaningless deadline for a huge piece of work, and instead have a simple working system up in weeks and add features as you go. Duh.
As I said before, these second-generation methodologies are, for the most part, just a backlash to the first. They are saying "well, heck, too much process didn't work. Let's take away and take away and take away process untill we have something workable."
So, second generation languages are still caught up in fear - that is, driving it out. That's half the sales pitch. If we just take those gates down, maybe pave the road, we will get home faster.
The next logical step is how to stick a turbo-booster on the car: To find ways to keep developers focused on work and improving the pace of delivery of software. (Hey, I'm giving a talk on that on Oct 7th in Indiana, but it's not a methodology yet.)
Test-Driven Development is pretty good example of this - it transforms testing from a boring, manual task into a coding task. It transforms API design (function signatures) from a "design" task into a coding task; exercising the API to run the tests. Developers like to code; it makes testing a coding a task they can then automate, so they can do more coding.
What other 'tasks' of a methodology actually force developers to do a better job, or keep them focused on the job at hand? Pair Programming could keep them from surfing the web, and I'm sure there are more. Every XP book says that local adaptors will 'customize' XP to fit. Sean's theory is that those customizations contain the secret sauce to improvement. If the theory above holds true, we just have to find the cutting edge shops that are allready doing some of these practices, interview them, and we'll be defining the leading edge of this 3rd generation methodology movement.
I'm willing to give it a shot; in fact, I'll be talking to folks about it tomorrow in the office, next week at the Better Software Conference in San Francisco, then next month at the Indiana QA Conference.
The whole idea needs more work, but I wanted to jot something down before I forget. I'm curious as to comments ...
