Are we Agile yet?
It doesn't matter what process you follow, the people involved will cause the success or failure of a project.
In the beginning, there was chaos. Developer were making software, but business couldn't really manage it.
Then came waterfall. This was nice because it was easy to manage. And luckily, we now had computers to manage the schedules, because they were always slipping.
When it was realized that developers are virtually incapable of estimating how long it takes to do anything longer than a few months (or more often days), clever managers came up with a solution: shorten the cycle! It was called the spiral method, and it is an iterative waterfall. Sure the schedules still slipped, but they didn't slip as much because the iterations were shorter. Thus was the Spiral method born.
While the managers were trying to fix waterfall (because they loved it's predictability) other smart people were trying other ways to solve these problems and learning from their experiments. Eventually, they got together to talk about their findings and the Agile Manifesto was born.
Agile in the early days was awesome. It was mostly XP-like. The iterations were short. The code was test driven. The customer was part of the team. The team members had Courage and they valued Simplicity, Communication, Feedback and Respect. They had practices that helped them achieve success.
Managers saw the awesome things that were happening with these Agile Teams and they took the parts that they thought were crucial and formed a methodology for software. It was mostly Scrum-ish. There were iterations and planning and schedules and all kinds of lovely charts for the managers. The XP practices (pairing, TDD, full team engagement, continuous integration, etc.) were minimized if they were used at all. And it was mostly OK, because the teams were still doing a lot better than they had before under waterfall.
So with Scrum, we have solved all of the problems, right? Maybe, but let's what happens to a project over time:
- For the first few sprints, the team goes really fast! Features are coming left and right. The customers are happy!
- Then the customer start to notice things that they want changed. Some are defects that need correction, but mostly, this is just the natural process of figuring out what you REALLY want after you have seen what you asked for.
- As the team changes the features, they start to slow down. After all, the architecture wasn't designed for these features because no one had thought of them when they were architecting.
- The team continues to make progress, but then the testers start to complain that the old features aren't quite right anymore. Also they don't have time to thoroughly test what the developers did before because there are new features are still coming pretty quickly. Couldn't the developers please stop breaking existing features?
- Development slows further as the amount of code increases and the number of hacks and work arounds increase. Quality starts to suffer.
I don't think that is surprising to anyone who has been involved in these projects, but that wasn't how it was in the beginning of Agile. What went wrong? The team followed Scrum, but since the developers were less experienced than the original Agilists, they didn't really follow the XP practices. Besides, everyone was Sprinting! You don't have time to pair program or learn TDD or refactor when you are Sprinting! Duh!
So, what brings success to a project isn't the process. It must be something else. It's the people. Skilled, smart, dedicated people will find a way to make the best product with the given constraints. If these people have better constraints, they will produce a better product of course. And this begs the question, how do we find these people?