In 2008 James W. Flosdorf, Jr. published the study “A PROGRAM EVALUATION OF THE ITIL-BASED CHANGE MANAGEMENT PROGRAM AT GENERAL MOTORS CORPORATION“.
The focus of this study was to determine which commonly implemented ITIL best practices in the Change, Release and Configuration Management disciplines were, by statistical measure, the best predictors of IT performance excellence. The ITPI researchers condensed their findings and categorized them into seven sets of related practices, comprising 30 different individual practices (ITPI, 2007). Of these 30 individual best practices, five were found to match the KPIs used in this evaluation of GM’s Change Management program. These five ITPI best 59 practice performance indicators were change success rate, emergency change rate, unauthorized change rate, release impact rate, and release rollback rate. The ITPI study defined these KPIs as follows:
- Change Success Rate – changes that met functional objectives and were completed during planned time
- Emergency Change Rate – changes are tracked, but do not get standard review before they are implemented
- Unauthorized Change Rate – percentage of changes that are unauthorized; changes made without being tracked by the standard change/release process
- Release Impact Rate – percentage of production releases that cause a service outage or incident
- Release Rollback Rate – percentage of production changes in the last 12 months that were rolled back
Part of the study compares GM’s performance to the ITPI study ranking of top-, medium-, and low-performing IT organizations for these five best practices. Top performers are defined by ITPI as the IT organizations performing in the top 20th percentile of all survey respondents (ITPI, 2007). It was found that GM performed better than the average of the top-performers in all of the best practice areas except for Emergency Change Rate (Urgent Changes), where GM (at 10.08%) had more urgent changes on average then the top-performers (at 7.10%) but less than the medium-performers (at 12.70%). Or put another way, GM was higher than the topperformer’s mean emergency change rate by 41.97%.
Table 11: Comparison of GM ChM Program Performance to ITPI Study KPIs
———————————————————————————————————
ITPI Study Performance Ranking
IT Best Practice KPI General Motors Top Medium Low
Change Success Rate 98.03% 96.40% 92.50% 81.30%
Emergency Change Rate 10.08% 7.10% 12.70% 22.90%
Unauthorized Change Rate 0.05% 0.70% 3.20% 11.40%
Release Impact Rate 0.21% 2.90% 5.60% 11.10%
Release Rollback Rate 1.05% 3.30% 3.80% 8.50%
———————————————————————————————————
ITPI–IT Process Institute; KPI-Key Performance Indicator
———————————————————————————————–