9.7 Process control

9.7.1 Key performance indicators

A number of key performance indicators (KPIs) should be monitored to assess the effectiveness of the Release Management process. Consider choosing some measures that show a clear indication of at least some of the following:

• Releases built and implemented on schedule, and within budgeted resources (but care should be taken to isolate any problems that are outside the control or responsibility of Release Management, such as application development delays)
• very low (ideally no) incidence of Releases having to be backed out due to unacceptable errors (note however that software Releases need not be entirely error-free; a decision can be made to go ahead with a Release despite the presence of errors, provided that they are of a minor nature, and within the permitted fault tolerances - see Chapter 6, Problem Management)
• low incidence of build failures
• secure and accurate management of the DSL
• no evidence of software in the DSL that has not passed quality checks and no evidence of reworks on any software that was extracted from the DSL
• DSL sizing matching the demand for space, and timely and accurate housekeeping of the DSL
• compliance with all legal restrictions relating to bought-in software
• accurate distribution of Releases to all remote sites
• implementation of Releases at all sites, including remote ones, on time
• no evidence of unauthorised reversion to previous versions at any site
• no evidence of use of unauthorised software at any site
• no evidence of payment of licence fees or wasted maintenance effort, for software that is not actually being used at any particular location
• no evidence of wasteful duplication in Release building (e.g. multiple builds of remote sites, when copies of a single build would suffice)
• accurate and timely recording of all build, distribution and implementation activities within the CMDB
• a post-mortem carried out on all Release activities, and all necessary corrective or follow-up action taken, together with any process improvements
• the planned composition of Releases matching the actual composition (which demonstrates good Release planning)
• IT and human resources required by Release Management being subject to good ongoing forward planning.

9.7.2 Management reporting

Other metrics that may be monitored include:

• the number of major and minor Releases per reporting period
• the number of problems in the live environment that can be attributed to new Releases, which need only be measured during the first few months of a new Release's life, classified by root cause, (e.g. 'wrong version of file' or 'missing files')
• the number of new, changed and deleted objects introduced by the new Release - e.g. how many modules and programs
• the number of Releases completed in the agreed timescales; this requires the central Release Management function to publish predefined targets (service levels or SLAs) for software distributions and other common tasks.