Participation growing in first-ever industry study exploring disparity between local and global programming results
PITTSFIELD, Mass., August 28, 2012 – A first-of-its-kind study analyzing the Agile development practices of one programming community –in this case, Columbus, Ohio— reveals a level of achievement that far exceeded expectations of both the analysts and the participants. The study, being conducted by software productivity researchers QSM Associates in tandem with the Central Ohio Agile Association (COHAA), bodes well for adopters of Agile software development.
Despite some initial concerns by a few participants on how they might fare on industry comparisons, "the defect data were significantly better than industry averages," according to Michael Mah, managing partner of QSM Associates. Most installations expect an improvement in schedule when adopting new technologies like Agile, said Mah, "but the improvement in quality was striking."
Early results from the Columbus-area participants show that a typical business system comprising 50,000 lines of code is completed 31% faster than the industry average in the QSM industry database of completed projects (4.4 months vs. 6.4 industry average). Even more remarkable, said Mah, is the defect rate, which is 4x better than the industry norm.
Not all participants may achieve such extreme results, Mah cautioned, because not all participants have adopted all of the best practices that lead to success. The survey does show that concepts embraced by Agile deliver remarkable results in areas of compressing a schedule and reducing defects. Some of these approaches include acceptance-test-driven development (TDD), pair programming, and co-location, Mah said. But even participants that had not adopted all of these techniques achieved better-than-average results.
The QSM Associates expertise is providing the Columbus Agile community with valuable information on factual patterns on productivity and quality instead of anecdotal claims. Moreover, the data helps answer questions about addressing development projects schedules and budgets. "Aside from the value inherent in knowing the current status, the Columbus experience will be analyzed on current and future projects, and that experience will be reflected in a continuous improvement process," said Ben Blanquera, Curator, Columbus Executive Agile SIG. "Participants never stop learning about their own productivity gains when compared with industry practices. The benchmark, as important as it is, is just a starting point that will help establish the Columbus community as a hotbed for knowledge and experience in all manner of Agile development."
The study allows participants to objectively benchmark the organization's performance to create an initial productivity baseline. This enables companies to identify strategic directions and goals, and to focus their improvement efforts with optimal efficiency, Blanquera said.
"The early results of the Columbus Agile Benchmark Study are consistent with our experience at Nationwide," said Guru Vasudeva, Senior VP and Enterprise CTO. "The data from participating companies shows a strong pattern of productivity, fast time-to-market and low cost, along with an impressively low level of defects. The fact that our own results are consistent with a larger community, in both quality and productivity, adds credibility to the claims of Agile's benefits."
"Agile is maturing. And, as more companies increase their emphasis on Test Driven Development and perhaps Acceptance Test Driven Development, we can anticipate even greater improvements," said Vasudeva. "Within Nationwide, we are seeing significant improvement in productivity and quality thru Acceptance Test Driven Development."
Survey participants are able to see their own results contrasted with the industry at large; and, the Columbus community also compared the regional results in the aggregate with worldwide data. However, participants cannot identify the individual results of other companies, thus protecting the confidentiality of divisions or companies involved in the study. For this reason, Mah said, he has been able to confirm information that did not seem intuitive at first: for example, some successful Agile development shops are reversing the trend of outsourcing or "offshoring" some of their software development efforts.
"Outsourcing is proving to be an old-fashioned concept that might have worked well in old-line industries, such as manufacturing, but it is coming back to haunt new-age industries," said Bart Murphy, Treasurer of COHAA.
"Outsourcing or offshoring may make sense in an Industrial Economy based in cost-effective manufacturing," agreed Mah. "It is harder in an Information Economy when knowledge workers are trying to solve design challenges. And, now we have data to prove that. That being said, if companies do choose to outsource, benchmarking techniques serve a vital purpose in negotiation and relationship management."
The results from Columbus were aggregated into the SLIM software lifecycle management solution, which includes an industry-wide database of thousands of completed projects. SLIM allows "normal humans" to accomplish sophisticated analysis with ease, Mah said. Chief among the results is the fact that programming teams that are co-located tend to be more effective than those where expertise is geographically divided. This is one of the facts that have lead to the reassessment of outsourcing software development.
"Agile development has proven itself. In fact, we are now seeing some medium-to-large software shops actually repatriating programming resources that had been shipped to Asia or other countries, in what were considered strategic cost saving initiatives," observed COHAA's Murphy. "Although overseas development can sometimes bring dramatic schedule improvement, this often comes at a price: more defects. The cost to remediate and maintain the delivered work product outweigh the cost savings gained in the outsourced model, especially as offshore operating costs continue to rise." he added.
QSM Associates is accepting ongoing enrollments for companies that want to join the study, which will be continuous throughout 2012 and 2013. As an incentive to participate, companies receive a temporary use license of SLIM, a data capture template that captures key metrics for Agile projects, including stories, story points, time, effort, defects, velocity, and backlog.
Participants will receive a private, confidential analysis of their patterns along with the study results for the group.
QSM Associates has more than 30 years experience in helping Fortune 500 companies, and has access to the largest benchmarking database, comprising more than 10,000 projects, and believed to be the world's largest and most complete benchmarking database. The company, an affiliate of QSM, Inc., uses state-of-the-art measurement and estimation tools that are part of the QSM SLIM software lifecycle management suite.
About COHAA
The Central Ohio Agile Association (COHAA) is a non-profit group of IT and Business professionals dedicated to finding a better way to deliver software. COHAA promotes the use of Agile practices and principles in project management, software development, quality assurance, and business analysis with an emphasis on solution delivery. COHAA supports and promotes the adoption of Agile practices in software development and delivery, and acts as a resource for individuals and organizations that use, or are interested in using Agile frameworks and practices such as XP, SCRUM, TDD, FDD, Lean, & KANBAN.
About QSM Associates
QSM Associates, Inc. helps organizations measure, plan, estimate and control software projects. It offers the SLIM (Software Lifecycle Management) Suite of tools, so managers can benchmark and forecast Agile, waterfall, in-house, offshore/multi-shore or ERP/package implementation projects. SLIM contains statistics from a worldwide database of more than 10,000 completed projects, enabling productivity benchmarking on the desktop. Using SLIM to dynamically run "virtual project simulations,” companies can model and forecast waterfall and Agile releases to deliver on time, within budget with >90% estimation accuracy. SLIM can also derive ROI achieved by Agile methods and other process improvements. QSM Associates offers consulting, training, and coaching to help accelerate this capability. Information is available at www.QSMA.com or email info@qsma.com .
Michael Mah (Twitter: @MichaelCMah), Managing Partner is also the Benchmark Practice Director at the Cutter Consortium, a Boston-area IT think-tank. Along with Microsoft, Oracle, and Salesforce.com, QSMA is a Rally Software Strategic Partner.
# # #
QSM and SLIM are registered trademarks of Quantitative Software Management, which is headquartered in McLean, VA with partner offices and affiliates throughout the U.S., Europe and Asia.
Contact: Edward Bride, 413-442-7718 (Ed@edbride-pr.com)
