One of the most important measurements in software development is related to the defect queue. I don’t mean simply the count of defects. A queue has a depth, latency, service time and throughput.
The depth is the count of issues assigned to a team or person. The latency tells you how long an issue sits in the queue before it is fixed. The service time is how long an issue takes to be resolved. And the throughput is how many are fixed per unit of time. The QA team is typically responsible for adding items to the defect queue.
A QA organization might focus on adding as many defects as possible; however, if they find more defects than the throughput, then there will be a backlog. Empirical evidence shows that deepening the backlog will increase the service time. Anyone who knew me when I was a QA manager knows that I loved to bury the dev team in defects. In retrospect that was not a great idea. If we found 5000 defects in the first week, it might help planning how many developers are needed but it doesn’t really help fix them faster.
As a development manager or release manager, you can address queue depth issues by either adding more workers or improving the service time. Adding more workers has a certain cost and overhead. The efficiency of an organization doesn’t scale linearly without careful division of labor. Dividing the labor at the defect level is not a practical solution since it is effect driven.
Improving the service time is possible
One of the best ways to improve defect turnaround time is finding the defect very soon after the change is introduced. Think about what you did in the last hour, compared to what you did at 10AM two weeks ago. It will take you minutes to fix a change in the past hour versus a day or more to debug a two week old item. Continuous integration gives the development team feedback via automated continuous test loops. Ideally the developers that made the change are notified immediately and efficiently.
In one agile and efficient organization, I have seen the team address 10 or more issues per person during sprints. They didn’t even have complete unit test coverage; what they did have was robust automated smoke tests. As a result of this automation collaboration with the test and development team, they were able to drive an amazing defect turnaround time. Using web service level test automation created a much more robust solution prior to handoff to the final user acceptance test.
The members of the team improved from an average defect age of 10 days to 1 day all the way down to an hour. That is geometric improvement via CI.
David Guimbellot, Area Vice President of Test Data Management & Continuous Delivery at Orasi Software
