Detailed Overview

Project Description

Experience with security evaluations of products in recent years has shown that such evaluations are a very expensive and time consuming process from the point of view of vendors of IT products. Although Security functional testing is an important component of security evaluation, time and cost considerations have made it to occupy a backseat in the overall security evaluation schemes except in the case of high assurance products. This situation is due to several factors. Some of these factors are:

Developing test specifications and test codes requires a fairly detailed knowledge of the behavior of security functions as well as the product interfaces needed to exercise those functions and measure the responses.
There are very few automated tools available to support the above process.

Project Motivation

The primary objectives behind the project are:

Improve the economics of security functional testing by automating some of the processes involved like test vector generation, test code generation, results analysis and reporting.
Improve the confidence in the tests generated by developing a formal model for specification of security function behaviors and using that as the basis for test vector generation.

Project Approach

The underlying framework we have used to improve the economics of security functional testing is the Test Automation Framework (TAF). The TAF is a architectural framework that automates the process of system or software testing by providing end-to-end tool support for the various process steps. The process steps include functional model development, model analysis, test code development, test execution and results analysis. We have developed a toolkit called the TAF-SFT toolkit that applies TAF to security functional testing of a product.

The quality of the tests generated depends upon the integrity of the underlying behavioral or the functional model that is used as the basis. We have addressed this requirement by using a formal modeling language to develop the functional model. The language we have used is Software Cost Reduction (SCR) that was developed by Naval Research Laboratory (NRL). The advantage of using a formal modeling language is that there are tools available to check a given model (developed for a specific domain or product) based on that language for satisfaction of both domain-specific properties (using assertions and a class of tools called theorem-provers) as well as domain-independent properties (e.g. absence of circularity, contradictions, redundancy etc).

Reference Implementation

Our proof of concept implementation involves the use of TAF-SFT toolkit for testing the security functions of a commercial DBMS product. The specifications of security function behaviors expressed by the DBMS product vendor through an ISO/IEC 15408 Security Target document were used to develop a SCR model of security functions. The SCR model of the DBMS security functions is transformed into a set of input-out relations using the Model Translator module of the TAF-SFT toolkit. The transformed model is then fed into the Test Vector Generator module of the TAF-SFT toolkit to generate the Test Vector file containing test vectors for testing the behavior of each of the security functions. The generated test vector file and the transformed model file are then input into the Coverage Analyzer module of the TAF-SFT toolkit to verify whether the generated test vectors do provide the coverage for all the path conditions in all the input output relations of the transformed model. The description of the interfaces (in our case is JDBC/SQL) used for exercising the security functions were captured in a document called Object Mapping file. The algorithmic pattern required for initializing values, loading test vectors and invoking the tests were encoded in a file called Test Driver Schema file. The Test Vector file, Object Mapping file and the Test Driver Schema file were fed into the Test Driver Generator module of the TAF-SFT toolkit to generate the Executable test code in Java (*.Java files) as well as expected outputs for the given set of test vectors (captured in a file called Expected Outputs file). The generated java code was compiled using a java compiler to generate Java classes (*.class files). The Java classes were executed against the DBMS product under test. The results of the tests were captured in the Actual Outputs file. The Expected Outputs file and the Actual Output file were fed into the Cross Comparator module of the TAF-SFT toolkit to generate the Test Results file. (The data flow involved in the use of TAF-SFT toolkit for testing the security functions of a commercial DBMS product is shown in the figure "Application of TAF-SFT toolkit for DBMS Security Functional Testing."

Target Community

The approach used in developing the TAF-SFT toolkit and its reference implementation could be used by commercial security evaluation laboratories for improving the integrity and economics of independent security functional testing that form a component of their overall security evaluation schemes.