ISO/IEC JTC 1/SC 7/WG 4 - Tools and environment

This document is a specialization of the more general reference model for software and systems product line engineering and management described in ISO/IEC 26550. The specialization defined herein addresses a class of methods and tools referred to as feature-based software and systems product line engineering, or feature-based PLE, which has emerged as a proven and repeatable product line engineering and management (PLE) practice supported by commercial tool providers. This document: —   provides the terms and definitions specific to feature-based PLE; —   defines how feature-based PLE is a specialization within the general ISO/IEC 26550 reference model for product line engineering and management; —   defines a reference model for the overall structure and processes of feature-based PLE and describes how the elements of the reference model fit together; —   defines interrelationships and methods for applying the elements and tools of the product line reference model; —   defines required and supporting tool capabilities. In this document, products of feature-based PLE include digital work products that support the engineering of a system. Some of the artefacts are actually part of the delivered products, while other artefacts can be non-deliverable, such as physical or digital design models. The intended audience for this document comprises: —   technology providers who wish to provide automated tool support for the reference model and processes described in this document; —   champions within an organization who wish to introduce feature-based PLE throughout that organization; —   IT staff within a PLE organization who will introduce and maintain the necessary technology to support feature-based PLE; —   practitioner stakeholders who will use the provided technology to practice feature-based PLE; —   technical and business managers who will sponsor and direct the methods necessary to practice feature-based PLE; —   university professors, researchers, corporate trainers, and other educators who will create and share pedagogical materials about feature-based PLE and its benefits.

This document defines the capabilities of issue management tools and is used to select the most appropriate one from many issue management tools. The evaluation and selection of the issue management tools is performed in accordance with ISO/IEC 20741 which defines the general evaluation selection process and evaluation characteristics. Issue management is based on the tasks described in several activities in their processes (e.g. project assessment and control, decision management, and system/software requirements definition) of ISO/IEC/IEEE 12207. This document is independent of development methodology or approaches (e.g. Waterfall or Agile) or lifecycle processes (e.g. implementation or operation).

This document specifies the capabilities of a tool to support review work. The evaluation and selection of the review tools are performed in accordance with ISO/IEC 20741 which defines the general evaluation selection process and evaluation characteristics. This document defines capabilities specific to review tools in the process. By using these two standards together, it is possible to derive objective and reasonable results of the evaluation and selection of review tools. The review work is based on the process, activities, and tasks defined in ISO/IEC 20246. It is also assumed that the review targets are defined in ISO/IEC 20246. The review work in this document is assumed not to be performed by a 3rd party, but within a project. The review tool capabilities specified in this document harmonize with the review process defined in ISO/IEC 20246. This document does not include automated process, activities, or tasks for conducting reviews such as automated source code checkers defined in ISO/IEC 30130. Issues which are identified in the review are recorded and managed by the tool; but defects found in tests and issues found in general except for reviews are out of the scope of this document.

This document specifies requirements for the vendors and gives guidelines for both the users and the developers of software safety and security verification tools. The users of such tools include, but are not limited to, bodies performing verification and software developers who need to be aware and pay attention to safety and/or security of software. This document guides the verification tool vendors to provide as high-quality products as possible and helps the users to understand the capabilities and characteristics of verification tools. This document introduces use cases for software safety and security verification tools and entity relationship model related to them. This document also introduces tool categories for software safety and security verification tools and gives category specific guidance and requirements for the tool vendors and developers.

This document, within the context of methods and tools for supporting the transitioning the organization's current development approach to software and systems product line engineering: — defines processes for product line transition management. Those processes are described in terms of purpose, inputs, tasks and outcomes; — defines method capabilities to support the defined tasks of each process; — defines tool capabilities that automate or semi-automate tasks and methods. This document does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

This document, within the context of methods and tools for supporting the diagnosis of the organization's capability to adopt or improve software and systems product line engineering: — defines processes for product line technical probe; those processes are described in terms of purpose, inputs, tasks and outcomes; — defines method capabilities to support the defined tasks of each process; and — defines tool capabilities that automate or semi-automate tasks and methods. This document does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

This document, within the context of methods and tools for product line product management: — defines product line-specific processes and their subprocesses for product management of software and systems product lines. Those processes are described in terms of purpose, inputs, tasks and outcomes; — defines method capabilities to support the defined tasks of each process; — defines tool capabilities to automate/semi-automate tasks or defined method capabilities. This document does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

This document, within the context of methods and tools for architecture design for software and systems product lines: — defines processes and their subprocesses performed during domain and application architecture design. Those processes are described in terms of purpose, inputs, tasks and outcomes; — defines method capabilities to support the defined tasks of each process; — defines tool capabilities to automate/semi-automate tasks or defined method capabilities. This document does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

This document, within the methods and tools of testing for software and systems product lines: — provides the terms and definitions specific to testing for software and systems product lines; — defines processes performed during product line testing (those processes are described in terms of purpose, inputs, tasks and outcomes); — defines method capabilities to support the defined tasks of each process; and — defines tool capabilities to automate/semi-automate tasks or defined method capabilities. This document concerns processes and capabilities of testing methods and tools for a family of products, not for a single system.

This document, within the methods and tools of organizational management for software and systems product lines: — enables the users of this document to holistically understand, adopt and enact the processes, tools and methods for product line organizational management; — helps the users evaluate and select relevant tools and methods based on business and user-related criteria; — helps make product line engineers, developers and tool vendors informed about capabilities of tools and methods that are required for supporting product line implementation from organizational aspects; and — provides product line-specific processes and capabilities of tools and methods in organizational management. This document concerns processes and capabilities of methods and tools for organizational management for a family of products, not for a single system. NOTE System Architecture is a set of logical and physical principles used to achieve a mission within a given environment. Components that can be subsystems derived from System Architecture are: software products, human-based products such as crew or operators, or hardware products like mechanical structures, electronic boards and chemicals. The scope of this document spans from the system to subsystems and components. Both hardware-intensive and software-intensive systems are included, if they are part of a product family.

ISO/IEC 26559:2017, within the context of the tools and methods of variability traceability for software and system product lines: - provides the terms and definitions specific to variability traceability for software and systems product lines; - defines process groups and their processes for establishing and managing variability traceability at product line lifecycle processes. Those processes are described in terms of purpose, inputs, tasks, and outcomes; - defines method capabilities to support the defined tasks of each process; - defines tool capabilities to automate/semi-automate tasks or defined method capabilities. ISO/IEC 26559:2017 does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

ISO/IEC 26558:2017, within the context of methods and tools for supporting explicit and/or separate variability modelling, variability model management and variability model support in software and systems product lines: - provides the terms and definitions specific to variability modelling for software and systems product line; - defines processes for variability modelling, variability model management and variability model support throughout the product line lifecycle. Those processes are described in terms of purpose, inputs, tasks and outcomes; - defines method capabilities to support the defined tasks of each process; - defines tool capabilities that automate or semi-automate tasks and methods. ISO/IEC 26558:2017 does not concern processes and capabilities of tools and methods for a single system but rather deals with those for a family of products.

ISO/IEC 20741:2017 gives guidelines for the evaluation and selection of software engineering tools, covering a partial or full portion of the software engineering life cycle. It establishes processes and activities to be applied for the evaluation of software engineering tools and selecting the most appropriate software engineering tools from several candidates. It establishes, for selected processes, the tasks and activities that can be applied for the evaluation of software engineering tools and selecting the most appropriate software engineering tools from several candidates. It establishes processes that can be applied for the evaluation of software engineering tools and selecting the most appropriate software engineering tools from several candidates. As these processes are generic, organizations can adapt these generic processes to meet organizational needs. The software engineering tool evaluation and selection processes can be viewed in the larger context of the organization's technology adoption process. ISO/IEC 20741:2017 provides the following: a) guidance on identifying organizational requirements for software engineering tools; b) guidance on mapping those requirements to software engineering tool characteristics to be evaluated; c) a process for selecting the most appropriate software engineering tool from several tools, based on measurements of the defined characteristics. NOTE 1 Guidance on mapping those requirements to software engineering tool capabilities to be evaluated is not covered by this document, but is covered by a series of standards for each tool area. Primary users of this document are organizations that intend to adopt software engineering tools to support their software life cycle processes. Software tool suppliers can also use this document to describe characteristics of their software engineering tools. ISO/IEC 20741:2017 is not intended to apply to: a) software engineering frameworks whose purpose is to provide mechanisms for data, control and presentation integration; b) general purpose tools (e.g. word processors, spreadsheets) which can be used in software engineering activities, nor software engineering tools of very narrow scope or specific purpose (e.g. a compiler); c) planning for the implementation of software engineering tools within an organization. NOTE 2 A user of this document can make the best possible selection of a software engineering tool and yet have no guarantee of a successful implementation. The methods described in this document are useful not only for the selection of software engineering tools, but for any project where COTS/FOSS software can be selected instead of engaging in new software development. To follow the guidance provided in this document consists in applying the activities and tasks that are attached to the defined processes to evaluate and select software. Organizations using this document for trade purposes can specify the minimum set of processes and their related activities and tasks, suitable to their given application.

ISO/IEC 26557:2016, within the context of tools and methods of variability mechanisms for software and system product lines: - provides the terms and definitions related to variability mechanisms for software and systems product lines; - defines processes and their subprocesses for operating variability mechanisms at each product line life cycle stages and those for providing managerial supports. Those processes are described in terms of purpose, inputs, tasks and outcomes; - defines method capabilities to support the defined tasks of each process; - defines tool capabilities to automate/semi-automate tasks or defined method capabilities. ISO/IEC 26557:2016 does not concern processes and capabilities of tools and methods for a single system, but rather deals with those for a family of products.

ISO/IEC 26551:2016, within the context of tools and methods of requirements engineering for software and systems product lines: - provides the terms and definitions specific to requirements engineering for software and systems product lines and associated member products; - defines process groups and their processes performed during product line requirements engineering (those processes are described in terms of purpose, inputs, tasks, and outcomes); - defines method capabilities to support the defined tasks of each process; - defines tool capabilities to automate/semi-automate tasks or defined method capabilities. ISO/IEC 26551:2016 concerns processes and capabilities of requirements tools and methods for a family of products, not for a single system. This International Standard is not applicable to physical artefacts. Instead, system-level artefacts and software lifecycle artefacts such as requirements documents, architectural data, validation plans, behavioural models, etc. are produced using methods and tools in this International Standard. In the case of the software components of a system, this International Standard can apply twice: once to handle the system elements of the product line and a second time to handle the software elements of the product line, if any. The product line processes are recursive within the different levels of products. NOTE The requirements in this International Standard apply to the family of systems, software or services.

ISO/IEC 30130:2016 defines the framework to which capabilities of software testing tools are allocated in order to identify the capabilities of products being used by any project for software testing. Software testing processes are identified in ISO/IEC/IEEE 29119‑2 and software verification processes are identified in ISO/IEC 12207. This International Standard is fully harmonized with these existing standards in terms of software testing processes. ISO/IEC 30130:2016 focuses on the following areas that the existing ISO/IEC standards do not deal with the following: - categorization of software test entities and software testing tools (Clauses 4 and 5); - characterization of each software testing tool category (Clauses 5 and 6); - mapping of software testing tool capabilities and characteristics (Clauses 6 and 7).

ISO/IEC 26550:2015 is the entry point of the whole suite of International Standards for software and systems product line engineering and management. The scope of this International Standard is to ? provide the terms and definitions specific to software and systems product line engineering and management, ? define a reference model for the overall structure and processes of software and systems product line engineering and management and describe how the components of the product line reference model fit together, and ? define interrelationships between the components of the product line reference model. ISO/IEC 26550:2015 does not describe any methods and tools associated with software and systems product line engineering and management. Descriptions of such methods and tools will appear in the consecutive International Standards (ISO/IEC 26551[1] to ISO/IEC 26556[2]). This International Standard does not deal with terms and definitions addressed by ISO/IEC/IEEE 24765:2010 that provides a common vocabulary applicable to all systems and software engineering work. Whenever this International Standard refers to "products", it means "system-level products" consisting of software systems or both hardware and software systems. It may be useful for the engineering and management of product lines that consist of only hardware systems but it has not been explicitly created to support such hardware product lines. This International Standard is not intended to help the engineering, production, warehousing, logistics, and management of physical items that, possibly combined with software, comprise the products. These processes belong to other disciplines (e.g. mechanics, electronics). NOTE Annex A provides further information on products. ISO/IEC 26550:2015, including the product line reference model and the terms and definitions, has been produced starting from References [6], [7], and [8] which finally resulted in a broad consensus from National Member Bodies at the time of publication. In addition to this background process, structures from ISO/IEC 12207:2008, ISO/IEC/IEEE 15288:2015, ISO/IEC 15940:2006 and ISO/IEC 14102:2008 have been used as a baseline. [1] Second edition to be published. [2] Under development.

ISO/IEC 26555:2015 deals with the tools and methods of technical management for software products, software services, software-intensive systems (including System Architecture and excluding hardware) within a product line. The scope of this International Standard is as follows: ? Enable the users of this standard to holistically understand, adopt, and enact the processes, tools, and methods for product line technical management. In addition, this International Standard helps the users evaluate and select relevant tools and methods based on business and user-related criteria. ? Help product line engineers, developers, and tool vendors become informed about capabilities of tools and methods that are required for supporting product line implementation from technical aspects. ? Provide product line-specific processes and capabilities of tools and methods in technical management. ISO/IEC 26555:2015 does not concern processes and capabilities of tools and methods for technical management for a one-of-a-kind system but rather deals with those belonging to a family of systems. NOTE: System Architecture is a set of logical and physical principles used to achieve a mission within a given environment. From System Architecture are derived components that can be subsystems, software products, human-based products like crew or operators or hardware product like mechanical structures, electronic boards, chemicals, etc. The scope of the International Standard spans from the system, to sub-systems, and software products. Other types of components and especially those related to human beings and to hardware parts are not within the scope of this International Standard.

ISO/IEC 15940:2013 aims to update ISO/IEC 15940:2006 to take into account the changes of ISO/IEC 12207:2008 and to add/complete the set of Software Engineering Environment services for software and system processes. Software engineering environments services, or SEE services, refers to a collection of services, partially or fully automated by software tools, that are used to support the execution of human activities in systems and software engineering. ISO/IEC 12207:2008 describes a comprehensive set of processes, activities and tasks to be performed when acquiring or developing a system/software. It does not address their implementation or their automation. These activities are usually carried out within a software or system development/maintenance project, and cover such areas as the specification, development, re-engineering or maintenance of systems. ISO/IEC 15940:2013 describes SEE services and relates them to ISO/IEC 12207:2008 in a manner applicable to a range of organizations. In defining a life cycle process for an organization, the user needs to find the appropriate level of automation provided by a software engineering environment. This may result in establishing a new SEE service or improving an existing one. The suite of SEE services described supports the process definitions in ISO/IEC 12207:2008. The purpose is to define a set of SEE services that are compatible with ISO/IEC 12207:2008, and that can be used either as a general reference, or to define an automated software and system process.

Configuration management (CM) is a process central to the software engineering life cycle. CM has been established as an ISO/IEC standard life cycle process in ISO/IEC 12207:2008, Information technology — Software life cycle processes and ISO/IEC 15288: 2008, Information technology — System life cycle processes. ISO/IEC 12207:2008 and ISO/IEC 15288:2008 describe a comprehensive set of processes, activities and tasks to be performed when acquiring or developing software. However, these documents do not address the capabilities that a CM tool user can expect from a tool in order to support the CM process and other software engineering life cycle activities. There is a gap between CM process descriptions and corresponding CM process automation services which affects both tool users and tool suppliers. ISO/IEC TR 18018:2010 provides guidance in the evaluation and selection for CM tools during acquisition. CM tool evaluation by prospective users can be complex, time consuming, and expensive. ISO/IEC TR 18018:2010 helps to characterize what a CM tool can and cannot do in the CM process. ISO/IEC TR 18018:2010 provides guidance for tool manufacturers in implementing a minimum set of capabilities. The capabilities defined in ISO/IEC TR 18018:2010 are linked to ISO/IEC 12207:2008 and ISO/IEC 15288:2008, and will provide tool manufacturers with guidance on the characteristics their tools should support to meet these International Standards.

Requirements engineering (RE) is an essential process of the systems and software engineering life cycles. RE has been established as an ISO/IEC standard life cycle process in both ISO/IEC 15288:2008, Systems and software engineering — System life cycle processes and ISO/IEC 12207:2008, Systems and software engineering — Software life cycle processes. This Technical Report provides guidance on desirable capabilities of RE tools. It supplements ISO/IEC 14102:2008, Information technology — Guideline for the evaluation and selection of CASE tools, which details a set of evaluation criteria for CASE tools without referencing a specific activity or service area.

Within systems and software engineering, Computer-Aided Software Engineering (CASE) tools represent a major part of the supporting technologies used to develop and maintain information technology systems. Their selection must be carried out with careful consideration of both the technical and management requirements. ISO/IEC 14102:2008 defines both a set of processes and a structured set of CASE tool characteristics for use in the technical evaluation and the ultimate selection of a CASE tool. It follows the software product evaluation model defined in ISO/IEC 14598-5:1998. ISO/IEC 14102:2008 adopts the general model of software product quality characteristics and subcharacteristics defined in ISO/IEC 9126-1:2001 and extends these when the software product is a CASE tool; it provides product characteristics unique to CASE tools. This larger set of characteristics is then organized into four groups: characteristics related to life cycle process functionality; characteristics related to CASE tool usage functionality; general quality characteristics; general characteristics not related to quality. This grouping provides a more manageable approach to the overall evaluation and selection process. The technical evaluation can indicate how well a CASE tool meets its user's stated requirements. It can also indicate how well the tool meets its claimed functionality. The objective of the technical evaluation process is to provide quantitative results on which the final selection can be based. Measurement assigns numbers (or other ratings) to attributes of entities; a major activity of evaluation is to obtain these measurements for use in selection. The final selection results should aim to achieve objectivity, repeatability and impartiality. These objectives and the confidence in the outcomes will in part depend on the resources allocated to the overall evaluation and selection process. The user of ISO/IEC 14102:2008 is asked to deal with these issues at an early stage.To be widely acceptable, these CASE tool evaluation and selection processes must be of value to the users of CASE tools, and to the suppliers of CASE to the community at large. The information outlined in ISO/IEC 14102:2008 should lead to more cost effective selections of CASE tools and to a greater uniformity in how CASE tool functions and features are described.

Since CASE (computer aided software engineering) adoption is a subject of the broader technology transition problem, ISO/IEC TR 14471:2007 addresses the adoption practices appropriate for a wide range of computing organizations. ISO/IEC TR 14471:2007 neither dictates nor advocates particular development standards, software processes, design methods, methodologies, techniques, programming languages, or life-cycle paradigms. ISO/IEC TR 14471:2007 will: identify critical success factors (CSF); propose a set of adoption processes; guide successful adoption in consideration of organizational and cultural environment. The following groups are targeted as potential audiences: CASE users; information systems managers; chief information officers (CIO); CASE suppliers; software engineering consultants; those involved in the acquisition of CASE tools and technology. Therefore ISO/IEC TR 14471:2007 addresses aspects of CASE tools adoption. It is best used in conjunction with ISO/IEC 14102 for CASE tool evaluation and selection. It is complementary to related ISO/IEC documents which deal with the general aspects of these topics. The purpose of ISO/IEC TR 14471:2007 is to provide a recommended practice for CASE adoption. It provides guidance in establishing processes and activities that are to be applied for the successful adoption of CASE technology. The use of ISO/IEC TR 14471:2007 will help to maximize the return and minimize the risk of investing in CASE technology. However, ISO/IEC TR 14471:2007 does not establish compliance criteria.

ISO/IEC 26550:2013 provides a reference model consisting of an abstract representation of the key processes of software and systems product line engineering and management, and the relationships between the processes. In the product line there are two development lifecycles (domain engineering and application engineering), the involvement of variabilities, and two process groups (organizational management and technical management). Domain Engineering lifecycle processes define and implement domain assets commonly used by member products within a product line, while Application Engineering lifecycle processes develop applications by exploiting the domain assets including common and variable assets. During domain engineering, product line variability, which reflects the specific needs of different markets and market segments, are explicitly defined. Variabilities are embedded in domain assets and during application engineering they are selected based on the defined variability models. Organizational Management process group helps organizations to establish and improve capabilities for nurturing their product lines from conception to retirement and for establishing and managing relationships with customers, providers, and other key stakeholders. Meanwhile, Technical Management process group provides actual means used to support, monitor, and control the activities of both domain engineering and application engineering of a product line. The reference model for software and systems product line engineering and management of ISO/IEC 26550:2013 can be used in subsequent standardization efforts to create appropriate standards for requirements engineering and management, design, realization, verification and validation, and organizational and technical management processes, tools, and methods of software and systems product line engineering.

ISO/IEC 26555:2013 deals with the capabilities of tools and methods of Software and Systems Product Line (SSPL) Technical Management. Since product lines deal with multiple products that have similarities, product lines have an unprecedented level of technical management complexities. In the product line there are inherent differences in technical considerations due to the parallel development processes, domain and application engineering. The two processes are tightly related with each other centered around assets, and the close relationships among domain engineering, application engineering, and assets require highly matured managerial capabilities. Technical management provides management support for a timely and proper deployment of product line in balance with pre-defined product line objectives such as reusability, reducing cost, and improving quality, as well as its planned cost, schedule, and resources. Technical management addresses actual means used to support, monitor, and control the activities of both domain engineering and application engineering of a product line. There are needs for defining product line specific technical management processes that integrate the involved product line disciplines with those for a single product. Furthermore, support of tools and methods are required so that a product line organization can perform technical management under the systematic control of complexities. ISO/IEC 26555:2013 addresses the product line specific processes in technical management by dividing those into Process Management, Variability Management, Asset Management, and Support Management areas with the guidance of a set of tools and methods capabilities for supporting tasks for product line technical management. ISO/IEC 26555:2013 is intended to benefit people who acquire, supply, develop, operate, and maintain tools and methods for product line technical management. ISO/IEC 26555:2013 can be used in one or more of the following modes: 1) By an organization intended to implement product lines, to understand, adopt, and enact the processes, tools, and methods for product line technical management. This also helps the organization to evaluate and select relevant tools and methods based on business and user-related criteria. 2) By a tool vendor who facilitates or leverages product line engineering practices, to provide a set of tool capabilities that should be embodied in a tool for supporting product line technical management.

ISO/IEC 26551:2012 provides the capabilities of tools and methods that support Software and Systems Product Line (SSPL) requirements engineering. In SSPL requirements engineering, there are three core processes: Product Line scoping, Domain Requirements Engineering, and Application Requirements Engineering. The major purpose of Product Line Scoping estimates the costs and benefits of a product line, and thereby lets an organization make a go/no-go decision.The costs and benefits estimation results play a pivotal role as an indicator for assessing the effectiveness and efficiency of a product line. The major purpose of the Domain Requirements Engineering is to analyze commonality and variability for the product line based on the initial features defined in Product Line Scoping, where the major purpose of the Application Requirements Engineering is to define application requirements based on domain requirements assets by reusing, selecting, or newly adding application specific requirements. For producing multiple products in a product line, the above processes need to be adequately integrated centered around core assets, which is causing the management of domain/application requirements complexities. Thus, the proper supports of methods and tools are essential, and the product line specific capabilities of methods and tools have to be defined. ISO/IEC 26551:2012 can be used in the following modes: - By the users: to benefit people who develop, operate, and manage requirements engineering for software and systems product lines. - By a product line organization: to provide guidance in the evaluation and selection for tools and methods for product line requirements engineering. - By providers of tools and methods: to provide guidance in implementing or developing tools and methods by providing a comprehensive set of the capabilities of tools and methods for product line requirements engineering.

ISO/IEC 15940:2006 defines the software engineering environment (SEE) services conceptually in a reference model that can be adapted to any SEEs to automate one or more software engineering activities. It describes services that support the process definitions as in ISO/IEC 12207 so that the set of SEE Services are compatible with ISO/IEC 12207. ISO/IEC 15940:2006 can be used either as a general reference, or to define an automated software process.

Provides guidance on identifying organizational requirements for CASE tools, on mapping those requirements to CASE tool characteristics to be evaluated and describes a process for selecting the most appropriate CASE tool from several tools, based on measurements of the defined characteristics.