ISO 15504
Encyclopedia
ISO/IEC 15504 Information technology — Process assessment, also known as SPICE (Software Process Improvement and Capability Determination), is a set of technical standards documents for the computer software development
process and related business management functions. It is another joint International Organization for Standardization
and International Electrotechnical Commission
standard.
ISO/IEC 15504 initially was derived from process lifecycle standard ISO 12207
and from maturity models like Bootstrap, Trillium and the CMM
.
. SPICE initially stood for "Software Process Improvement
and Capability Evaluation", but in consideration of French
concerns over the meaning of "evaluation", SPICE has now been redefined as "Software Process Improvement and Capability Determination". SPICE is still used for the user group of the standard, and the title for the annual conference. The first SPICE was held in Limerick, Ireland in 2000, "SPICE 2003" was hosted by ESA in the Netherlands
, "SPICE 2004" was hosted in Portugal
, "SPICE 2005" in Austria
, "SPICE 2006" in Luxembourg
, "SPICE 2007" in South Korea
, "SPICE 2008" in Nuremberg
, Germany
and SPICE 2009 in Helsinki
, Finland
.
The first versions of the standard focused exclusively on software development
processes. This was expanded to cover all related processes in a software business, for example, project management
, configuration management
, quality assurance
, and so on. The list of processes covered, grew to cover six business areas:
In a major revision to the draft standard in 2004, the process reference model was removed and is now related to the ISO 12207
(Software Lifecycle Processes). The issued standard now specifies the measurement framework and can use different process reference models. There are five general and industry models in use.
Part 5 specifies software process assessment and part 6 specifies system process assessment.
The latest work in the ISO standards working group includes creation of a maturity model, which is planned to become ISO/IEC 15504 part 7.
The International Standard (IS) version of ISO/IEC 15504 now comprises 6 parts. The 7th part is currently in an advanced Final Draft Standard form and work has started on part 8.
Part 1 of ISO/IEC TR 15504 explains the concepts and gives an overview of the framework.
. The reference model defines a process dimension and a capability dimension.
The process dimension in the reference model is not the subject of part 2 of ISO/IEC 15504, but part 2 refers to external process lifecycle standards including ISO/IEC 12207 and ISO/IEC 15288. The standard defines means to verify conformity of reference models.
With new parts being published, the process categories will expand, particularly for IT service process categories and enterprise process categories.
The capability of processes is measured using process attributes. The international standard defines nine process attributes:
Each process attribute consists of one or more generic practices, which are further elaborated into practice indicators to aid assessment performance.
Each process attribute is assessed on a four-point (N-P-L-F) rating scale:
The rating is based upon evidence collected against the practice indicators, which demonstrate fulfillment of the process attribute.
This includes:
One of the requirements is to use a conformant assessment method for the assessment process. The actual method is not specified in the standard although the standard places requirements on the method, method developers and assessors using the method. The standard provides general guidance to assessors and this must be supplemented by undergoing formal training and detailed guidance during initial assessments.
The assessment process can be generalized as the following steps:
An assessor can collect data on a process by various means, including interviews with persons performing the process, collecting documents and quality records, and collecting statistical process data. The assessor validates this data to ensure it is accurate and completely covers the assessment scope. The assessor assesses this data (using their expert judgment) against a process's base practices and the capability dimension's generic practices in the process rating step. Process rating requires some exercising of expert judgment on the part of the assessor and this is the reason that there are requirements on assessor qualifications and competency. The process rating is then presented as a preliminary finding to the sponsor (and preferably also to the persons assessed) to ensure that they agree that the assessment is accurate. In a few cases, there may be feedback requiring further assessment before a final process rating is made.
The process assessment model (PAM) in part 5 is based on the process reference model (PRM) for software: ISO/IEC 12207.
The process assessment model in part 6 is based on the process reference model for systems: ISO/IEC 15288.
The standard allows other models to be used instead, if they meet ISO/IEC 15504's criteria, which include a defined community of interest and meeting the requirements for content (i.e. process purpose, process outcomes and assessment indicators).
There are a limited number of computer based tools that present the indicators and allow users to enter the assessment judgment and notes in formatted screens, as well as automate the collated assessment result (i.e. the process attribute ratings) and creating reports.
These skills include:
The competency of assessors is the subject of part 3 of ISO/IEC 15504.
In summary, the ISO/IEC 15504 specific training and experience for assessors comprise:
within a technology organization. Process improvement is always difficult, and initiatives often fail, so it is important to understand the initial baseline level (process capability level), and to assess the situation after an improvement project. ISO 15504 provides a standard for assessing the organization's capacity to deliver at each of these stages.
In particular, the reference framework of ISO/IEC 15504 provides a structure for defining objectives, which facilitates specific programs to achieve these objectives.
Process improvement is the subject of part 4 of ISO/IEC 15504. It specifies requirements for improvement programmes and provides guidance on planning and executing improvements, including a description of an eight step improvement programme. Following this improvement programme is not mandatory and several alternative improvement programmes exist.
software development needs to have a good understanding of the capability of potential suppliers to deliver.
ISO/IEC 15504 (Part 4) can also be used to inform supplier selection decisions. The ISO/IEC 15504 framework provides a framework for assessing proposed suppliers, as assessed either by the organization itself, or by an independent assessor.
The organization can determine a target capability for suppliers, based on the organization's needs, and then assess suppliers against a set of target process profiles that specify this target capability. Part 4 of the ISO/IEC 15504 specifies the high level requirements and an initiative has been started to create an extended part of the standard covering target process profiles. Target process profiles are particularly important in contexts where the organization (for example, a government department) is required to accept the cheapest qualifying vendor. This also enables suppliers to identify gaps between their current capability and the level required by a potential customer, and to undertake improvement to achieve the contract requirements (i.e. become qualified). Work on extending the value of capability determination includes a method called Practical Process Profiles - which uses risk as the determining factor in setting target process profiles. Combining risk and processes promotes improvement with active risk reduction, hence reducing the likelihood of problems occurring.
On the other hand, ISO/IEC 15504 has not yet been as successful as the CMMI
. This has been for several reasons:
Like the CMM, ISO/IEC 15504 was created in a development context, making it difficult to apply in a service management context. But work has started to develop an ITIL
-based process reference model that can serve as a basis for a process assessment model. This is planned to become part 8 to the standard. In addition there are methods available that adapt its use to various contexts.
Software development
Software development is the development of a software product...
process and related business management functions. It is another joint International Organization for Standardization
International Organization for Standardization
The International Organization for Standardization , widely known as ISO, is an international standard-setting body composed of representatives from various national standards organizations. Founded on February 23, 1947, the organization promulgates worldwide proprietary, industrial and commercial...
and International Electrotechnical Commission
International Electrotechnical Commission
The International Electrotechnical Commission is a non-profit, non-governmental international standards organization that prepares and publishes International Standards for all electrical, electronic and related technologies – collectively known as "electrotechnology"...
standard.
ISO/IEC 15504 initially was derived from process lifecycle standard ISO 12207
ISO 12207
ISO/IEC 12207 Systems and software engineering — Software life cycle processes is an international standard for software lifecycle processes...
and from maturity models like Bootstrap, Trillium and the CMM
Capability Maturity Model
The Capability Maturity Model is a development model that was created after study of data collected from organizations that contracted with the U.S. Department of Defense, who funded the research. This model became the foundation from which CMU created the Software Engineering Institute...
.
Overview
ISO/IEC 15504 is the reference model for the maturity models (consisting of capability levels which in turn consist of the process attributes and further consist of generic practices) against which the assessors can place the evidence that they collect during their assessment, so that the assessors can give an overall determination of the organization's capabilities for delivering products (software, systems, and IT services).History
A working group was formed in 1993 to draft the international standard and used the acronym, SPICESPICE
SPICE is a general-purpose, open source analog electronic circuit simulator.It is a powerful program that is used in integrated circuit and board-level design to check the integrity of circuit designs and to predict circuit behavior.- Introduction :Unlike board-level designs composed of discrete...
. SPICE initially stood for "Software Process Improvement
Process improvement
In organizational development , process improvement is a series of actions taken by a process owner to identify, analyze and improve existing business processes within an organization to meet new goals and objectives. These actions often follow a specific methodology or strategy to create...
and Capability Evaluation", but in consideration of French
France
The French Republic , The French Republic , The French Republic , (commonly known as France , is a unitary semi-presidential republic in Western Europe with several overseas territories and islands located on other continents and in the Indian, Pacific, and Atlantic oceans. Metropolitan France...
concerns over the meaning of "evaluation", SPICE has now been redefined as "Software Process Improvement and Capability Determination". SPICE is still used for the user group of the standard, and the title for the annual conference. The first SPICE was held in Limerick, Ireland in 2000, "SPICE 2003" was hosted by ESA in the Netherlands
Netherlands
The Netherlands is a constituent country of the Kingdom of the Netherlands, located mainly in North-West Europe and with several islands in the Caribbean. Mainland Netherlands borders the North Sea to the north and west, Belgium to the south, and Germany to the east, and shares maritime borders...
, "SPICE 2004" was hosted in Portugal
Portugal
Portugal , officially the Portuguese Republic is a country situated in southwestern Europe on the Iberian Peninsula. Portugal is the westernmost country of Europe, and is bordered by the Atlantic Ocean to the West and South and by Spain to the North and East. The Atlantic archipelagos of the...
, "SPICE 2005" in Austria
Austria
Austria , officially the Republic of Austria , is a landlocked country of roughly 8.4 million people in Central Europe. It is bordered by the Czech Republic and Germany to the north, Slovakia and Hungary to the east, Slovenia and Italy to the south, and Switzerland and Liechtenstein to the...
, "SPICE 2006" in Luxembourg
Luxembourg
Luxembourg , officially the Grand Duchy of Luxembourg , is a landlocked country in western Europe, bordered by Belgium, France, and Germany. It has two principal regions: the Oesling in the North as part of the Ardennes massif, and the Gutland in the south...
, "SPICE 2007" in South Korea
South Korea
The Republic of Korea , , is a sovereign state in East Asia, located on the southern portion of the Korean Peninsula. It is neighbored by the People's Republic of China to the west, Japan to the east, North Korea to the north, and the East China Sea and Republic of China to the south...
, "SPICE 2008" in Nuremberg
Nuremberg
Nuremberg[p] is a city in the German state of Bavaria, in the administrative region of Middle Franconia. Situated on the Pegnitz river and the Rhine–Main–Danube Canal, it is located about north of Munich and is Franconia's largest city. The population is 505,664...
, Germany
Germany
Germany , officially the Federal Republic of Germany , is a federal parliamentary republic in Europe. The country consists of 16 states while the capital and largest city is Berlin. Germany covers an area of 357,021 km2 and has a largely temperate seasonal climate...
and SPICE 2009 in Helsinki
Helsinki
Helsinki is the capital and largest city in Finland. It is in the region of Uusimaa, located in southern Finland, on the shore of the Gulf of Finland, an arm of the Baltic Sea. The population of the city of Helsinki is , making it by far the most populous municipality in Finland. Helsinki is...
, Finland
Finland
Finland , officially the Republic of Finland, is a Nordic country situated in the Fennoscandian region of Northern Europe. It is bordered by Sweden in the west, Norway in the north and Russia in the east, while Estonia lies to its south across the Gulf of Finland.Around 5.4 million people reside...
.
The first versions of the standard focused exclusively on software development
Software development
Software development is the development of a software product...
processes. This was expanded to cover all related processes in a software business, for example, project management
Project management
Project management is the discipline of planning, organizing, securing, and managing resources to achieve specific goals. A project is a temporary endeavor with a defined beginning and end , undertaken to meet unique goals and objectives, typically to bring about beneficial change or added value...
, configuration management
Configuration management
Configuration management is a field of management that focuses on establishing and maintaining consistency of a system or product's performance and its functional and physical attributes with its requirements, design, and operational information throughout its life.For information assurance, CM...
, quality assurance
Quality Assurance
Quality assurance, or QA for short, is the systematic monitoring and evaluation of the various aspects of a project, service or facility to maximize the probability that minimum standards of quality are being attained by the production process...
, and so on. The list of processes covered, grew to cover six business areas:
- organizational
- management
- engineering
- acquisition supply
- support
- operations.
In a major revision to the draft standard in 2004, the process reference model was removed and is now related to the ISO 12207
ISO 12207
ISO/IEC 12207 Systems and software engineering — Software life cycle processes is an international standard for software lifecycle processes...
(Software Lifecycle Processes). The issued standard now specifies the measurement framework and can use different process reference models. There are five general and industry models in use.
Part 5 specifies software process assessment and part 6 specifies system process assessment.
The latest work in the ISO standards working group includes creation of a maturity model, which is planned to become ISO/IEC 15504 part 7.
The ISO/IEC 15504 standard
The Technical Report (TR) document for ISO/IEC TR 15504 was divided into 9 parts. The initial International Standard was recreated in 5 parts. This was proposed from Japan when the TRs were published at 1997.The International Standard (IS) version of ISO/IEC 15504 now comprises 6 parts. The 7th part is currently in an advanced Final Draft Standard form and work has started on part 8.
Part 1 of ISO/IEC TR 15504 explains the concepts and gives an overview of the framework.
Reference model
ISO/IEC 15504 contains a reference modelReference Model
A reference model in systems, enterprise, and software engineering is a model of something that embodies the basic goal or idea of something and can then be looked at as a reference for various purposes.- Overview :...
. The reference model defines a process dimension and a capability dimension.
The process dimension in the reference model is not the subject of part 2 of ISO/IEC 15504, but part 2 refers to external process lifecycle standards including ISO/IEC 12207 and ISO/IEC 15288. The standard defines means to verify conformity of reference models.
Processes
The process dimension defines processes divided into the five process categories of:- customer-supplier
- engineering
- supporting
- management
- organization
With new parts being published, the process categories will expand, particularly for IT service process categories and enterprise process categories.
Capability levels and process attributes
For each process, ISO/IEC 15504 defines a capability level on the following scale:| Level | Name |
|---|---|
| 5 | Optimizing process |
| 4 | Predictable process |
| 3 | Established process |
| 2 | Managed process |
| 1 | Performed process |
| 0 | Incomplete process |
The capability of processes is measured using process attributes. The international standard defines nine process attributes:
- 1.1 Process Performance
- 2.1 Performance Management
- 2.2 Work Product Management
- 3.1 Process Definition
- 3.2 Process Deployment
- 4.1 Process Measurement
- 4.2 Process Control
- 5.1 Process Innovation
- 5.2 Process OptimizationOperations researchOperations research is an interdisciplinary mathematical science that focuses on the effective use of technology by organizations...
.
Each process attribute consists of one or more generic practices, which are further elaborated into practice indicators to aid assessment performance.
Each process attribute is assessed on a four-point (N-P-L-F) rating scale:
- Not achieved (0 - 15%)
- Partially achieved (>15% - 50%)
- Largely achieved (>50%- 85%)
- Fully achieved (>85% - 100%).
The rating is based upon evidence collected against the practice indicators, which demonstrate fulfillment of the process attribute.
Assessments
ISO/IEC 15504 provides a guide for performing an assessment.This includes:
- the assessment process
- the model for the assessment
- any tools used in the assessment
Assessment process
Performing assessments is the subject of parts 2 and 3 of ISO/IEC 15504. Part 2 is the normative part and part 3 gives a guidance to fulfill the requirements in part 2.One of the requirements is to use a conformant assessment method for the assessment process. The actual method is not specified in the standard although the standard places requirements on the method, method developers and assessors using the method. The standard provides general guidance to assessors and this must be supplemented by undergoing formal training and detailed guidance during initial assessments.
The assessment process can be generalized as the following steps:
- initiate an assessment (assessment sponsor)
- select assessor and assessment team
- plan the assessment, including processes and organizational unit to be assessed (lead assessor and assessment team)
- pre-assessment briefing
- data collection
- data validation
- process rating
- reporting the assessment result
An assessor can collect data on a process by various means, including interviews with persons performing the process, collecting documents and quality records, and collecting statistical process data. The assessor validates this data to ensure it is accurate and completely covers the assessment scope. The assessor assesses this data (using their expert judgment) against a process's base practices and the capability dimension's generic practices in the process rating step. Process rating requires some exercising of expert judgment on the part of the assessor and this is the reason that there are requirements on assessor qualifications and competency. The process rating is then presented as a preliminary finding to the sponsor (and preferably also to the persons assessed) to ensure that they agree that the assessment is accurate. In a few cases, there may be feedback requiring further assessment before a final process rating is made.
Assessment model
The process assessment model (PAM) is the detailed model that is used for an actual assessment. This is an elaboration of the process reference model (PRM) provided by the process lifecycle standards.The process assessment model (PAM) in part 5 is based on the process reference model (PRM) for software: ISO/IEC 12207.
The process assessment model in part 6 is based on the process reference model for systems: ISO/IEC 15288.
The standard allows other models to be used instead, if they meet ISO/IEC 15504's criteria, which include a defined community of interest and meeting the requirements for content (i.e. process purpose, process outcomes and assessment indicators).
Tools used in the assessment
There exist several assessment tools. The simplest comprise paper-based tools that are manually used. In general, they are laid out to incorporate the assessment model indicators, including the base practice indicators and generic practice indicators. Assessors write down the assessment results and notes supporting the assessment judgment.There are a limited number of computer based tools that present the indicators and allow users to enter the assessment judgment and notes in formatted screens, as well as automate the collated assessment result (i.e. the process attribute ratings) and creating reports.
Assessor qualifications and competency
For a successful assessment, the assessor must have a suitable level of the relevant skills and experience.These skills include:
- personal qualities such as communication skillCommunication skillCommunication skill or communication skills may refer to*Rhetoric,*Communication, or*English studies....
s. - relevant education and training and experience.
- specific skills for particular categories, e.g. management skills for the management category.
- ISO/IEC 15504 related training and experience in process capability assessments.
The competency of assessors is the subject of part 3 of ISO/IEC 15504.
In summary, the ISO/IEC 15504 specific training and experience for assessors comprise:
- completion of a 5 day lead assessor training course
- performing at least one assessment successfully under supervision of a competent lead assessor
- performing at least one assessment successfully as a lead assessor under the supervision of a competent lead assessor. The competent lead assessor defines when the assessment is successfully performed. There exist schemes for certifying assessors and guiding lead assessors in making this judgment.
Uses of ISO/IEC 15504
ISO/IEC 15504 can be used in two contexts:- Process improvement, and
- Capability determination (= evaluation of supplier's process capability).
Process improvement
ISO/IEC 15504 can be used to perform process improvementProcess improvement
In organizational development , process improvement is a series of actions taken by a process owner to identify, analyze and improve existing business processes within an organization to meet new goals and objectives. These actions often follow a specific methodology or strategy to create...
within a technology organization. Process improvement is always difficult, and initiatives often fail, so it is important to understand the initial baseline level (process capability level), and to assess the situation after an improvement project. ISO 15504 provides a standard for assessing the organization's capacity to deliver at each of these stages.
In particular, the reference framework of ISO/IEC 15504 provides a structure for defining objectives, which facilitates specific programs to achieve these objectives.
Process improvement is the subject of part 4 of ISO/IEC 15504. It specifies requirements for improvement programmes and provides guidance on planning and executing improvements, including a description of an eight step improvement programme. Following this improvement programme is not mandatory and several alternative improvement programmes exist.
Capability determination
An organization considering outsourcingOutsourcing
Outsourcing is the process of contracting a business function to someone else.-Overview:The term outsourcing is used inconsistently but usually involves the contracting out of a business function - commonly one previously performed in-house - to an external provider...
software development needs to have a good understanding of the capability of potential suppliers to deliver.
ISO/IEC 15504 (Part 4) can also be used to inform supplier selection decisions. The ISO/IEC 15504 framework provides a framework for assessing proposed suppliers, as assessed either by the organization itself, or by an independent assessor.
The organization can determine a target capability for suppliers, based on the organization's needs, and then assess suppliers against a set of target process profiles that specify this target capability. Part 4 of the ISO/IEC 15504 specifies the high level requirements and an initiative has been started to create an extended part of the standard covering target process profiles. Target process profiles are particularly important in contexts where the organization (for example, a government department) is required to accept the cheapest qualifying vendor. This also enables suppliers to identify gaps between their current capability and the level required by a potential customer, and to undertake improvement to achieve the contract requirements (i.e. become qualified). Work on extending the value of capability determination includes a method called Practical Process Profiles - which uses risk as the determining factor in setting target process profiles. Combining risk and processes promotes improvement with active risk reduction, hence reducing the likelihood of problems occurring.
Acceptance of ISO/IEC 15504
ISO/IEC 15504 has been successful as:- ISO/IEC 15504 is publicly available through National Standards Bodies.
- It has the support of the international community.
- Over 4000 assessments have been performed to date.
- Major sectors are leading the pace such as automotive, space and medical systems with industry relevant variants.
- Domain-specific models like Automotive SPICE and SPICE 4 SPACE can be derived from it.
- There have been many international initiatives to support take-up such as SPICE for small companies.
On the other hand, ISO/IEC 15504 has not yet been as successful as the CMMI
Capability Maturity Model Integration
Capability Maturity Model Integration is a process improvement approach whose goal is to help organizations improve their performance. CMMI can be used to guide process improvement across a project, a division, or an entire organization...
. This has been for several reasons:
- ISO/IEC 15504 is not available as free download but must be purchased from the ISO (Automotive SPICE on the other hand can be freely downloaded from the link supplied below.) CMM and CMMI are available as free downloads from the SEI website.
- The CMMI is actively sponsored (by the US Department of Defense).
- The CMM was created first, and reached critical 'market' share before ISO 15504 became available.
- The CMM has subsequently been replaced by the CMMI, which incorporates many of the ideas of ISO/IEC 15504, but also retains the benefits of the CMM.
Like the CMM, ISO/IEC 15504 was created in a development context, making it difficult to apply in a service management context. But work has started to develop an ITIL
Itil
Itil may mean:*Atil or Itil, the ancient capital of Khazaria*Itil , also Idel, Atil, Atal, the ancient and modern Turkic name of the river Volga.ITIL can stand for:*Information Technology Infrastructure Library...
-based process reference model that can serve as a basis for a process assessment model. This is planned to become part 8 to the standard. In addition there are methods available that adapt its use to various contexts.
Further reading
- ISO/IEC 15504-1:2004 Information technology — Process assessment — Part 1: Concepts and vocabulary
- ISO/IEC 15504-2:2003 Information technology — Process assessment — Part 2: Performing an assessment
- ISO/IEC 15504-3:2004 Information technology — Process assessment — Part 3: Guidance on performing an assessment
- ISO/IEC 15504-4:2004 Information technology — Process assessment — Part 4: Guidance on use for process improvement and process capability determination
- ISO/IEC 15504-5:2006 Information technology — Process Assessment — Part 5: An exemplar Process Assessment Model
- ISO/IEC TR 15504-6:2008 Information technology — Process assessment — Part 6: An exemplar system life cycle process assessment model
- ISO/IEC TR 15504-7:2008 Information technology — Process assessment — Part 7: Assessment of organizational maturity
- ISO/IEC PDTR 15504-8 Information technology — Process assessment — Part 8: An exemplar process assessment model for IT service management
- ISO/IEC TS 15504-9:2011 Information technology — Process assessment — Part 9: Target process profiles
- van Loon, H. (2007a) Process Assessment and ISO 15504 Springer ISBN 9780387300481
- van Loon, H. (2007b) Process Assessment and Improvement Springer ISBN 9780387300443