In the first part of my review of Rally Software’s AgileZen, I focused wholly on what the software did and how, whether features were implemented well and if they were generally useful. That was in the middle of an 8-month development project during which it was difficult to be objective. I had to believe to succeed, and AgileZen was our core user story management system. Unless my team urinated upon it from great heights during iteration reviews, I wasn’t going to change midstream. Now that that project has reached its inevitable conclusion and is in the hands of the users, I want to focus this second part of the review solely on value. 

(A side note: I also thought AgileZen was going to have a new version out by the time I got part two of this review so I could say of the software, “See? I told them [among others] to do that, and there it is.” From what I’ve read, it’s still percolating.)

For the price - for the time invested in training, using, adjusting our practices, and nagging my team to use it – was it worth it, and would I use it again? Flat out, yes, I would use AgileZen again on a project this size. There are a few things that the system doesn’t do, for which I had to employ different software/methods. 

Beyond that, I feel that for smaller projects, it may be easier to just go with the corkboard/Excel file method. Additionally, the core user group, our developers, was fairly reticent about using AgileZen. A few took a real shine to it, but others had to be constantly reminded to shift their work through the various columns, or keep notes/acceptance criteria up to date. However, I was very happy with the level of adoption, as I feel it was internally embraced more than TFS and even Basecamp.

We had about 150 User Stories run through the system during the course of the project, which I think is a good size for this software. We were able to organize the stories around users, and in some instances, around themes. There’s no functionality around themes/epics (groups of stories that make up a functional whole), so we had to sort of wing that part using colors and tags. It worked out, but I would like to see something a little more codified around themes in AgileZen’s new release. Lack of codification is AgileZen’s M.O., so this may never happen. 

My primary personal use of the AgileZen system as the team’s scrum master was to manage the backlog, and that was a breeze. It’s just so user-friendly and easy to manage stories that it really didn’t feel like work. My only issue was reconciling the backlog with my main Excel spreadsheet. Why did I have to maintain two lists? Reporting.

The reporting in AgileZen is its weakest area. The reports provided just don’t make much sense or add any value to what I needed to get out of the system. The core issue is that AgileZen sees every story as having the same weight/size/effort, so all reports are based solely around the NUMBER of stories that are completed, not the size of those stories. Since we use the story size and our velocity to derive our budget for the client, it was a major issue. This forced me to have a second set of identical stories in Excel to track where we were in story points and iterations with a separate burn-down chart (also in Excel), to show progress to the client rather than just using AgileZen. This was not ideal, and could be ameliorated simply by creating a report that focuses on the story size rather than just the number of stories, or by simply building a good old burn-down chart.  I had a heck of a time during the busier iterations making sure the wording of the stories (and sizes) in my Excel doc and AgileZen were exactly the same. On a project with double or triple the number of stories (some teams like to break things out really granularly so this could happen on even small projects), I would have gone slightly more insane than I already am. 

Note that AgileZen is a Kanban board rather than a specific scrum board, and our current methodology is straight from Mike Cohn’s “Succeeding with Agile” and “User Stories Applied” with as little modification as possible. Kanban does not have the same focus on Velocity and Story Points that form the absolute core of Cohn’s methods. While this is frustrating, AgileZen is flexible enough to fit the method very well, except reporting. This needs improvement.

Project Timeline
I don’t care much for detailed project timelines. If they are more than a page, no one is going to read them. Worst case scenario is you’ve just wasted hours putting together a document that is instantly out of date the moment it’s printed, and ultimately forgettable if too lengthy. That said, everyone still needs one, and this is where, like using Excel to track velocity, I needed another software system other than AgileZen. Now I don’t expect AgileZen to have any type of Basecamp-esque project timeline tracking or Gantt capabilities (shudder), so I don’t begrudge it for not doing for a project what Basecamp does, but I do want to note that AgileZen is not going to help you manage timelines/work outside the user story system. Need a training manual to get to the printer? Need to track dates of client availability and meetings? Track dates that team members are OOTO? You’re going to have to use something other than AgileZen. 

Bug tracking
During our iteration cycles we would track bugs as tasks attached to stories, and then color them yellow while in the QA column. If they were assigned to a developer he knew he had bugs to fix. If assigned to QA, and still yellow, there were questions around some of the issues from the developers. While not an explicit way of handling bugs in AgileZen, this method was discussed and formulated in our scrum review meetings, and definitely shows the flexibility of the system. Many scrum teams use story cards for bugs, but some are so conditional and tied to the stories from which they derive that there is no sense mucking up the story list. This is another method that would work very well using AgileZen’s Kanban board. It was at this moment when the team agreed this was how we were handling bugs, and when I became a fan of the AgileZen software. The flexibility of the Kanban board implementation is just amazing. There’s just enough stuff attached to each story to be able to track something like bugs well, but not too much so the whole system is bogged down.

A Release Iteration and AgileZen
It’s pretty much against scrum training and reading, but we did have release iteration during this big project. We built up a lot of technical debt and needed an iteration (almost two actually) to shine up the systems and fix bugs without mucking everything up with new User Stories to complete. The core thing that happens in release iteration is the shift in focus to tasks rather than User Stories. 

User Stories at the end of the iterations that they are assigned are functionally complete and potentially shippable when you hit the release iteration (or should be), but some of them are ugly both in code and for the end-user. The set of issues surrounding the shine up of these stories aren’t exactly bugs (everything works as expected most of the time) and are, in most cases, not new User Stories. So that leaves a pile of tasks, from infrastructure refinements to usability tweaks, aesthetics, etc. I found that our team pretty much ignored AgileZen during the release iteration. While we could have used it to track tasks (either as story cards or as tasks attached to a story) we rarely did. What’s more, if you start to attach User Stories to tasks in order to track your shine up, you will need to drag stories out of the “done” or “completed” columns. That gets extremely messy when trying to determine which stories are actually potentially shippable and which are not started or incomplete. We probably should have used AgileZen more during the release iteration, but the bottom line is, we didn’t and I don’t think anyone missed it much. Should scrum teams have release iterations? That’s a dog for a different day.

Conclusion
AgileZen is an amazing feat of design, usability, and implementation, for the price, its value is unmatched as far as what I’ve been able to find in the field of software. Like Basecamp, it’s tough to believe you’re getting something so good for so cheap. It needs a lot of work in the reporting area and I would love to see it fall in line more with scrum methodologies. (But would that constrain other methods? Probably.) However, what it does in its core functionality it does the best because it strips away all the stuff you THINK you need, but simply don’t. I can’t wait for my next project to use this again, and am absolutely looking forward to an updated release.