Measurement & Business Case Working Group

Mission and Objectives

Develop an overarching approach that

leverages and harmonizes existing information assurance and software and system development measurement methods
provides a means for quantifying, and interpreting large amounts of data from disparate sources
increases efficiencies in software assurance (SwA) measurement and process improvement
provides quantifiable evidence to support assurance case/argument
helps decision makers quantify security risk exposures

Facilitate the compatibility of network, system, and software testing, assessment, and monitoring tools output to integrate data sources for measurement

Recent Releases and Updates

The following presentations were made at the 24th Annual Systems & Software Technology Conference (SSTC 2012):

Paul Croll from CSC presented Standards and Guideance for Engineering Secure Systems and Software Sustainability – Challenges for Acquisition, Engineering, and Capability Delivery in the Face of the Growing Cyber Threat.
Carol Woody from the SEI presented Software Assurance v. Security Compliance: Why is Compliance Not Enough?

The Software Security Measurement and Analysis (SSMA) Project advances the state-of-the-practice in software security measurement and analysis. The CERT Program at Carnegie Mellon University's Software Engineering Institute (SEI) chartered the SSMA Project and is exploring how to use risk analysis to direct an organization's software security measurement and analysis efforts. The overarching goal is to develop a risk-based approach for measuring and monitoring the security characteristics of interactively complex software-reliant systems across the life cycle and supply chain. To accomplish this goal, the project team has developed the SEI Integrated Measurement and Analysis Framework (IMAF) and refined the SEI Mission Risk Diagnostic (MRD).

The report Risk-Based Measurement and Analysis: Application to Software Security presents the foundational concepts of a risk-based approach for software security measurement and analysis and provides an overview of the IMAF and the MRD.
The SEI has developed the Mission Risk Diagnostic (MRD) Method Description to assess risk in interactively complex, socio-technical systems across the life cycle and supply chain.

Practical Measurement Framework for Software Assurance and Information Security was published in October 2008 through the Practical Software and Systems Measurement Support Center. It provides an approach for measuring the effectiveness of achieving Software Assurance (SwA) goals and objectives at an organizational, program or project level. It addresses how to assess the degree of assurance provided by software, using quantitative and qualitative methodologies and techniques. This framework incorporates existing measurement methodologies and is intended to help organizations and projects integrate SwA measurement into their existing programs.

Acquisition Measurement version 1.0, published through the PSM Support Center, provides a foundation for the discussion and advancement of acquisition measurement. Its primary focus is on improving the way in which acquisition projects and organizations manage and conduct their own activities. A secondary focus is management and oversight of the supplier. The paper provides guidance, a WBS, and Information Need - Category - Measure (ICM) table.

A new article in the Measurement content area, Predictive Models for Identifying Software Components Prone to Failure During Security Attacks, has been posted on the Build Security In website.

Related Documents

The CIS Security Metrics, Center for Internet Security, May 11 2009
Organizations struggle to make cost-effective security investment decisions; information security professionals lack widely accepted and unambiguous metrics for decision support. CIS established a consensus team of one hundred (100) industry experts to address this need. The result is a set of standard metrics and data definitions that can be used across organizations to collect and analyze data on security process performance and outcomes. This document contains twenty (20) metric definitions for six (6) important business functions: Incident Management, Vulnerability Management, Patch Management, Application Security, Configuration Management and Financial Metrics. Additional consensus metrics are currently being defined for these and additional business functions.

Measuring Cyber Security and Information Assurance: State-of-the-Art Report (SOAR), Information Assurance Technology Analysis Center (IATAC), May 8, 2009
Despite significantly increased funding for research, development, and deployment of information assurance (IA) defenses, reports of attacks on, and damage to the IT infrastructure are growing at an accelerated rate. While a number of cyber security/IA (CS/IA) strategies, methods, and tools exist for protecting IT assets, there are no universally recognized, reliable, and scalable methods to measure the “security” of those assets. CS/IA practitioners’ success in protecting and defending an uninterrupted flow of information on IT systems and networks is critically dependent upon their ability to accurately measure in real time the security status of the local system as well as on their understanding of the security status of regional, national, and international networks. This report assesses the current “state of the art” in CS/IA measurement to facilitate further research into this subject. Progress has been made, but much remains to be done to achieve the goal of real-time, accurate CS/IA measurement. Enabling such measurement would make it is possible to understand, improve, and predict the state of CS/IA.

Contact Information

To comment or request further information, contact the working group chair at software.assurance [at] dhs.gov.

To join the Software Assurance Measurement Working Group, see the instructions for joining a working group.