Benchmarking Report on
Business Process Analysis and Systems Design
for Electronic Recordkeeping
Conducted by the National Archives and Records Administration
September 30, 2005
Executive Summary
Background
The Business Process Analysis (BPA) Benchmarking Team was formed in the fall of 2004 as part of the National Records Management Program Fiscal Year 2005 work plan. The team was charged with "conducting at least four benchmarking visits with government agencies, university research groups, and private service providers on business process analysis and systems development to support electronic recordkeeping." The team's charge alluded to two possible ways of identifying recordkeeping requirements and ensuring that they are met in new systems design: 1) business process analysis and 2) integration of recordkeeping requirements into the systems development life cycle. The team investigated both approaches by conducting benchmarking interviews focused on six specific methodologies:
Business Process Analysis- Australian Standard: Work Process Analysis for Recordkeeping, AS 5090-2003
- Center for Technology in Government. Models for Action tool from Practical Tools for Electronic Records Management and Preservation
- The Minnesota State Archives. Trustworthy Information Systems Handbook
- US Patent and Trademark Office. USPTO Electronic Records Management Technical Standard and Guideline, July 2002
- The Federal Bureau of Investigation. FBI Electronic Recordkeeping Certification Manual
- The Central Intelligence Agency. Electronic Recordkeeping System (ERKS) Requirements
Findings
During its work, the Benchmarking Team discovered unique strengths in all six methodologies that make them valuable for identifying electronic recordkeeping requirements or otherwise improving electronic records management. The exemplary practices in these specific methodologies represent two different yet complementary ways of ensuring that recordkeeping requirements are identified and met in new information systems design. One approach, business process analysis, identifies process-specific recordkeeping requirements that cannot be identified except through examination of a particular function's needs. The other approach, certification of new information systems against a predefined list of requirements for recordkeeping system functionality, is the best way of ensuring that all important systems are able to handle records appropriately. An agency that uses both approaches can be confident that it is capturing the right records in all formats required to meet its business needs and that it is creating, maintaining, protecting, and providing appropriate access to authentic, reliable, and trustworthy records throughout the records' life cycle. The Benchmarking Team believes that the methodologies described in this report provide a wide range of practical tools and models that could enable all Federal agencies, regardless of their current electronic records management and system development sophistication, to develop comprehensive policies and procedures for integrating records management requirements into new information technology (IT) systems.
Common Themes
- Records managers need to focus on the business process.
- Business process analysis and system development are resource intensive, but including recordkeeping in preexisting processes minimizes additional cost.
- Risk management can help decide which processes justify intensive analysis and which systems must meet all requirements.
- Success in identifying and meeting recordkeeping requirements in new systems design depends on the interaction of people, processes, and technology.
- Records managers need new skills to participate in new processes.
1. Report
1.1 Purpose and Scope
The purpose of the Business Process Analysis Benchmarking Project is to identify workable, reproducible methodologies for integrating a records management perspective into business process analysis and into the systems development lifecycle so that recordkeeping requirements are identified and met in new systems design. NARA is publicizing them as best practices so that other organizations can learn from the early adopters.
The BPA Benchmarking Team's charge to investigate "business process analysis and systems development to support electronic recordkeeping" alludes to two basic approaches for identifying recordkeeping requirements and ensuring that they are met in new systems design: 1) business process analysis and 2) integration of recordkeeping requirements into the systems development life cycle. The team investigated both. (For a diagram of the systems or solution development life cycle, see Appendix C.)
Business Process Analysis: Business process analysis generally happens before or during the concept exploration phase of the systems development life cycle, regardless of whether or not records managers are involved. In the analysis that occurs during redesign, the business process is usually mapped and then examined for improvement opportunities. In a records-aware business process analysis, the analysts break a work process down into constituent tasks and subtasks and then ask a series of questions about how and why each task is documented. Questions include whether a record is created or changed by a subtask, who needs access to the record, and whether there are laws, regulations, or professional practices that guide how the subtask is performed or recorded. Because the Office of Management and Budget's (OMB) capital planning process requires all agencies to redesign their business processes before submitting a capital request for a new information technology (IT) system, agencies need to perform business analysis as part of their redesign efforts prior to the formal start of the systems development life cycle in any case. The BPA methodologies we investigated provide a records-aware set of activities and questions that can supplement the process that business analysts should be using already. The result of these methodologies is a set of detailed, process-specific recordkeeping requirements which can then be passed on to the requirements-gathering and design stages of the systems development life cycle. According to standard IT practice, business analysis also occurs as part of the requirements-gathering stage of the systems development life cycle. The purpose of this kind of analysis is to learn about the business process being automated and to identify system requirements relating to functionality, data, performance, security, user interfaces, and many other factors, but generally not explicitly to records. Identification of recordkeeping requirements requires a records-aware business process analysis, which can take place either during business process redesign, or during the later analysis phase of the systems development life cycle, or during both phases.
Integrating Records Management into the Systems Development Life Cycle: In modifying the systems development life cycle to integrate recordkeeping concerns, records managers gain recognition as full stakeholders with approval authority at the various control gates through which a new system passes. Records managers use a predefined list of requirements that a system must meet in order to manage records appropriately, in most cases derived from the Department of Defense's STD-5015.2: Design Criteria Standard for Electronic Records Management Software Applications (DOD 5015.2-STD). They work with system owners or project managers to ensure that these requirements are included in the system requirements documentation and in the system design, and then to ensure that the requirements are met in the system as built. If the requirements are not met, the records managers can withhold approval of the system until they are.
Both of these valuable approaches lead to improved management of electronic records through influence on the design of new IT systems, although in somewhat different ways.
The scope of the Team's search for organizations that were successfully using either of these practices was very broad. As specified in the Team's charge, it considered "government agencies, university research groups, and private service providers" in its search.
It should be noted that there are some important differences between the BPA Team's benchmarking practice and traditional benchmarking in a business context. Not only were we benchmarking publicized products or internal processes that our partners expressed an interest in sharing, but our goal was to identify best practices so that other agencies could learn from tested methodologies. In this context, the usual benchmarking emphasis on confidentiality of sources was contrary to our purpose. Consequently, we are naming our benchmarking partners because we (and they) want others to know about their useful innovations in electronic records management.
1.2 Results and Analysis
1.2.1 Findings
All of the benchmarked sites have developed excellent methodologies to improve the management of records in electronic systems from which other organizations can learn. Because the sites differed in their objectives and organizational contexts, however, they developed two distinct kinds of processes: 1) bringing a records perspective to business process analysis, or 2) ensuring that recordkeeping requirements are met in the systems development life cycle through a program of electronic recordkeeping system certification.
Three of the benchmarked organizations have no authority to enforce the application of their process. Instead they have sought to influence and provide guidance on the identification of recordkeeping requirements, often without even knowing where their process has been applied. The Center for Technology in Government (CTG) and the Minnesota State Archives have both produced tools that are freely available on the Internet for anyone to use. Australian Standard: Work Process Analysis for Recordkeeping was developed to provide additional guidance on the topic of analyzing business activities, which is important to the ISO Records Management Standard as Step B of the methodology for Designing and Implementing Records Systems but is not explained in detail in the standard. The Work Process Analysis Standard is also designed to be flexible enough for any organization to use. Because all of these methodologies were designed to be used widely, they focus on providing a flexible process that organizations can follow to identify their recordkeeping requirements rather than a specific or tailored checklist of predefined requirements.
The other three organizations, the FBI, CIA, and USPTO, developed processes to ensure that recordkeeping requirements were identified and met in their own organizations. They all started off with good structural relationships between records management and IT, all had well-developed systems development life cycle methodologies in place, and all shared a Federal recordkeeping environment. In this environment the most effective way for records managers to exert influence on the ability of IT systems to appropriately manage records is to piggyback on the existing systems development life cycle by integrating a predefined checklist of recordkeeping requirements (derived from NARA guidance and from DOD 5015.2-STD) into the overall requirements gathering activity. In the FBI and CIA, records managers have the authority to certify that a system meets all relevant recordkeeping requirements and can appropriately manage records. USPTO achieves a similar result with the Electronic Records Management Technical Standard and Guideline, an integral part of the systems development life cycle that is enforced by the Technical Review Board. Building records management into the robust systems development infrastructure at these agencies ensures that all systems above a defined threshold will be held to the standard recordkeeping requirements. Also, because these methodologies were designed specifically for each agency, the requirements and processes are customized for their environments and include the agency's relatively stable legal and regulatory requirements.
Most of the requirements identified by the certification processes are system functional requirements derived from DOD 5015.2-STD. These requirements focus on the capabilities a system must possess in order to manage records, but they do not provide guidance on: what specific records are required by the business process being automated, what constitutes a record, who should have access to it, what its retention period should be, and so on. Instead, the system functional requirements guarantee that the system will be capable of performing necessary actions such as capturing records, maintaining access control, and applying retention rules.
1.2.2 Strengths of Specific Methodologies
Although the methodologies we benchmarked fall into two broad categories, each has unique strengths that make it particularly valuable for identifying electronic recordkeeping requirements or otherwise improving electronic records management.
The Australian Standard: Work Process Analysis for Recordkeeping is unique for its practical guidance on mapping a work process and conducting both functional and sequential analysis. Particularly valuable are the clear and usable sets of questions that relate to each stage of the analysis and the focus on documenting variations in work processes. The Australian Work Process Analysis Standard moves the focus of records management to the work process and argues that recordkeeping should be a natural, integral part of the work process rather than a separate activity.
The Center for Technology in Government's Models for Action tool assumes that a business process analysis is already underway and supplies sets of requirement elicitation questions to ensure that recordkeeping requirements are identified at the business process level, at the record level, and at the system level. The tool asks the analyst to decide whether technology can satisfy each requirement identified and what management processes or procedures are necessary to ensure that all requirements are met.
The Australian Work Process Analysis Standard and the Models for Action tool can be used for more than automation projects. Either might be used, for instance, to help identify the problems in a broken process or to streamline an inefficient one. Models for Action, for example, provides tips on identifying process steps that may not add value and might be eliminated. Both methodologies are flexible and scalable, making them useful for a wide range of records management projects.
The Minnesota State Archives' Trustworthy Information Systems Handbook incorporates some characteristics of both the business process analysis methodologies and the certification model. It contains a checklist of criteria to consider when designing a system, but also includes a list of sidebar questions which elicit the kinds of process-specific information the business process analysis tools gather. Together, these provide a complete list of recordkeeping issues to consider when developing a system. The TIS Handbook's most noteworthy feature, however, appears to be its usefulness as an educational tool. Freely available on the Internet and useable without mediation from any records staff, the handbook's language targets IT and/or program staff who may never have thought about recordkeeping issues before. In addition to starting the handbook with sections explaining why trustworthy electronic records are important ("What's in it for you?" "What is a trustworthy information system?" "Why are metadata and documentation important?"), the Archives staff hired technical writers to enhance the document's clarity and layout. The result is a very clear, usable guide for IT or project management staff which can be used even in weak records management environments. The guide's success at explaining the importance of building recordkeeping requirements into new systems makes it a great tool for laying the educational groundwork in agencies where records management and IT do not yet have a close working relationship.
The US Patent and Trademark Office uses an Electronic Records Management Technical Standard and Guideline to integrate recordkeeping requirements into all new IT systems by requiring that system developers fill out a comprehensive recordkeeping checklist as part of the standard systems development life cycle. USPTO has taken advantage of the fact that many process-derived recordkeeping requirements for the agency are stable across many business processes and only need to be identified once. Therefore, its certification model includes more than the recordkeeping system functional requirements specified in guidance such as DOD 5015.2-STD, and it provides some of the kinds of detail that would be identified in a records-aware business process analysis (such as suggested case file metadata.) Additional strengths of USPTO's guideline are its clear outline of the roles and responsibilities of the various players as well as its placement of the electronic records management activities in the systems development life cycle.
The FBI and CIA also have well-elaborated electronic recordkeeping certification processes in place which ensure that major systems will either meet specified recordkeeping functional requirements as specified by law or explain why those requirements do not apply. The records management staff have authority to approve the system design at each of several major systems development life cycle control gates.
A notable feature of the FBI's process is its extremely comprehensive and logical FBI Electronic Recordkeeping Certification Manual, released in April 2004. This manual includes a clear statement of the roles and responsibilities of the Records Officer and system owner. It also includes an excellent graphic showing the relationships among the processes for electronic recordkeeping certification, capital planning and investment control, security certification and accreditation, and the systems development life cycle. The FBI's manual outlines four possible paths to certification, asking system owners to choose one path during the first phase of the project. It then outlines the process for certification for both new and legacy systems. The manual provides details on doing a project recordkeeping risk assessment and includes sample tests and expected results for each of the ERK assessment criteria. All of these features are unique to the FBI's process and make its documentation so thorough that an inexperienced system owner could figure out exactly how the process works.
The CIA, on the other hand, recently revised their 130-page Electronic Recordkeeping System Requirements certification document developed in 2000; they distilled it down to a very brief ERKS Certification Guide, including an overview of the process's purpose and steps with a short list of requirements. The CIA found that the longer manual intimidated project managers and made the ERKS process seem burdensome. Because of the support of CIA's excellent staffing structure (Information Management Officers are deployed in offices throughout the agency), the documentation does not need to bear the full burden of explaining the process. The CIA requires an Information Management Plan for each system to document all of the processes and procedures that will be used to ensure that records are managed appropriately through creation, maintenance, use, and disposition. The Plan also includes a description of the system, system records, and a traceability matrix for the ERKS requirements.
All of the methodologies that NARA benchmarked are among the leaders in identifying recordkeeping requirements and integrating them into new information systems design. Every Federal agency, from the most advanced to those just beginning to think about managing electronic records, can use these resources to find ideas for taking the next step toward electronic records management maturity. Those records managers who are still struggling to bridge the communication gap with their IT units might use the clarity and user-friendliness of Minnesota's Trustworthy Information Systems Handbook to help explain why such collaboration is worthwhile. In agencies where there is little infrastructure to support the systems development life cycle, or where records management cannot yet integrate an electronic recordkeeping certification process within that life cycle, records managers may be able to educate and influence system owners to think about recordkeeping questions and requirements, such as those suggested by these methodologies. If an agency has a good working and structural relationship between records management and IT, the agency might use a fully-documented certification process such as the FBI's or USPTO's Electronic Records Management Technical Standard and Guideline as a framework for setting up something similar. Even agencies that are leaders in certification could take the additional step of developing a standard recordkeeping business process analysis methodology for systematically identifying process-specific recordkeeping requirements. CTG's Models for Action or the Australian Standard: Work Process Analysis for Recordkeeping could serve as a model.
1.2.3 The Best of Both Worlds
In an ideal world, organizations would institute processes belonging to both approaches to ensure that they identify and meet all kinds of recordkeeping requirements: they would include a records management perspective in their business process analysis and would certify all new information systems against a checklist of electronic recordkeeping system requirements. The systems development life cycle process would include records managers as stakeholders, and records managers would have the authority to approve any system that contains records as it passes each life cycle control gate.
Process specific recordkeeping requirements would be identified using a records-aware business process analysis during business process reengineering or the analysis phase of the systems development life cycle. According to the Information Technology Management Reform Act (ITMRA, AKA Clinger-Cohen), agency CIOs, when requesting money for a large IT project, must certify to OMB that the process being automated has been redesigned. As some form of analysis is mandatory, records managers need only to influence the manner in which it is done to ensure that recordkeeping issues are addressed. Both the Australian Standard: Work Process Analysis for Recordkeeping and CTG's Models for Action tool argue that an awareness of what records are created and modified during a process provides a good framework for business process analysis generally. As a good first step toward integrating records concerns into this IT analysis, records managers could issue and publicize lists of recordkeeping questions to consider when doing analysis. This would increase the probability that business process analysis results will include process-specific requirements such as what records should be created, what they should contain, how long each type should be kept, who needs access to them, and what restrictions apply. The analyst should capture all identified requirements and their implementation strategies in a tracking system. (CTG provides a good example.) Where implementation strategies involve technology, requirements should be passed along to the system project manager for tracking with all other system requirements. Any requirements involving policy or management process changes, even if they do not involve technology, should also be tracked.
In addition to capturing recordkeeping requirements during business process analysis, an agency would also integrate requirements for records management functionality into the systems development life cycle. To ensure that consistent sets of recordkeeping system requirements are built into every system to which they pertain and to facilitate consistent systems approval from records managers at life-cycle control gates, agencies would develop a list of recordkeeping requirements that comply with all Federal recordkeeping laws and regulations. The requirements contained in DOD 5015.2-STD are the best starting place, although agencies might want to add other requirements for their own environments. NARA's guidance, Electronic Records Management Guidance on Methodology for Determining Agency-unique Requirements and Examples of System Functions for Electronic Recordkeeping (ERK) or Electronic Records Management (ERM) provide help in identifying additional requirements for the list. Agencies might also embed their list of requirements into a package that includes guidance on when particular requirements apply, test cases for checking that requirements have actually been met, and instructions on how to navigate the process. Records Officers should certify systems that meet all relevant recordkeeping requirements and withhold certification from systems that do not, possibly with a provision for exceptions in emergency or very low risk situations.
Information systems would also automate as many of the process-specific recordkeeping requirements as possible, using the results of the business process analysis. These requirements - both functional requirements for capabilities of the system and process-specific requirements that define (among others) what constitutes a record and how long it should be kept - would be integrated with the overall requirements document generated by the systems development life cycle process and tracked to the end of the system's life. Agencies would document policy and management process strategies for meeting any requirements identified during business process analysis that could not be met through technology alone; the documentation would include a migration plan for ensuring the preservation of records with value beyond the life of the system. Tracking all requirements - not just those met in a system - would provide confidence that the right records have been captured and managed appropriately.
Any system that is being upgraded or significantly modified would need to pass through the same certification process as a new system. High risk legacy systems would also be analyzed and their shortcomings addressed, even if they are not scheduled for an upgrade.
The process of certifying recordkeeping systems exerts backward pressure on system owners, encouraging them to consult records management staff earlier in the life cycle. Since they want their systems to pass through control gates without any problems, system owners have an incentive to seek the advice of records managers well before the first review. This early consultation might provide records managers with an additional opening to introduce question sets for use during business process analysis.
1.2.4 Common Themes
In addition to an enhanced understanding of business process analysis and integrating records management into the systems development life cycle, our benchmarking partners' experiences offer a number of other lessons.
Focus on the business process. The business process provides a common language and framework for records managers to talk to process owners and system developers about records concerns. Program staff members often enjoy talking about their process, and they care what information is available to it and produced by it. To increase consideration of recordkeeping issues, records managers need to speak a language that communicates well to other stakeholders, and a focus on the business process allows records staff to do just that. Our CTG Models for Action interview particularly stressed this, but the Australian Standard: Work Process Analysis for Recordkeeping also emphasizes that recordkeeping should be a natural part of the business process and not an additional set of steps outside of it.
Business process analysis and system development are resource intensive, but including recordkeeping in pre-existing processes minimizes additional cost. Agencies should already be undertaking business process analysis or business process reengineering projects in connection with all major systems, as mandated by ITMRA. Although these processes take time, money, and energy, integrating records concerns into the processes does not add significantly to the effort. Similarly, integration of recordkeeping certification into the system development life cycle does not noticeably affect overall timelines. These processes are both worth the investment in time; careful planning and analysis of requirements at the beginning of the process can prevent the waste of enormous expenditures on systems that do not function according to agency requirements or manage needed information.
Records managers need new skills to participate in new processes. Nearly all of our benchmarking partners mentioned the importance of records management staff possessing the same set of skills: process mapping, analytical thinking, speaking IT's language, leading meetings and facilitating discussions, managing projects, and familiarity with the systems development life cycle. Most important of all are written and verbal communication skills, especially the ability to explain records management concepts without jargon in a way that ties records to the business processes they document.
Use risk management. Risk management can determine which processes justify intensive analysis and which systems must meet every requirement. Almost every methodology and user integrated risk management and/or cost-benefit analysis into decision-making. To date, the National Archives of Australia has applied the procedures of the Australian Standard: Work Process Analysis for Recordkeeping to only two of its highest risk processes because the analysis is so time-consuming. The FBI manual includes a section on assessing the risk of failing to meet each criterion. The CIA uses a matrix analysis table that includes risk as a factor to determine which systems must pass through the full Project Management Process, of which the ERKS certification is a part.
Success in identifying and meeting recordkeeping requirements in new systems design depends on the interaction of people, processes, and technology. The major goal of records-aware business process analyses and electronic recordkeeping certification programs is to ensure that IT systems are designed to capture and manage records appropriately. Although it is a major part of the solution, technology alone cannot provide assurance that recordkeeping requirements will be identified and met. Documented processes such as the methodologies discussed in this report fulfill that role, along with trained people to implement those processes. Several of our benchmarking partners noted the importance of the human element in electronic recordkeeping. Certification processes work best once the records manager and system developer have established familiarity and trust. Business process analysis may only be used while there is a champion encouraging its application. Some sites reported that progress in electronic records management turned out to be very dependent on the interests of individuals; when those people left the organization, the focus on electronic records management receded. While the importance of establishing trust and individuals as champions will not disappear, organizations can reduce the negative impact of losing critical people by documenting and enforcing a standard process that many people understand and use. Organizations can train all records staff to be comfortable with new electronic records processes, the language of IT, and the life cycle of information systems.
1.2.5 Recommendations
For NARA:
- Use the online Electronic Records Policy Working Group (ERPWG) Toolkit to publicize the best practices identified in this report
- Advocate agency certification of electronic recordkeeping requirements as part of the systems development life cycle
- Advocate integration of recordkeeping questions into business process reengineering as required by ITMRA
- Provide business analysis training to NARA records management staff so that they can assist agencies in the implementation of these processes
- Continue to use benchmarking of these and other records management processes as a tool to identify best practices and encourage their wide adoption
- Train records management staff in business analysis to encourage wide understanding of the systems development process
- Advocate more complete identification and implementation of recordkeeping requirements in new systems design
- Include records management in the business analysis and requirements-gathering processes for new systems Embed records management in the systems development life cycle and include the Records Officer as a stakeholder in system approvals
- Develop an electronic recordkeeping system certification methodology to meet agency needs
1.2.6 Conclusion
This benchmarking project identified several exemplary practices for two different yet complementary ways of ensuring that recordkeeping requirements are identified and met in new information systems design. Business process analysis can identify process-specific recordkeeping requirements that cannot be identified in any other way. Certification of new information systems against a predefined list of recordkeeping functional requirements and other requirements that apply across the agency is the best way of ensuring that all important systems can handle their records appropriately. An agency that uses both types of process can feel confident that it is capturing the right electronic records for its business processes and that it is maintaining, protecting, and providing appropriate access to them in a trustworthy way. The Benchmarking Team believes that the methodologies described in this report provide a wide range of practical tools and models that could enable all Federal agencies, regardless of their current electronic records management and system development sophistication, to develop comprehensive policies and procedures for integrating records management requirements into new IT systems
PDF files require the free Adobe Reader.
More information on Adobe Acrobat PDF files is available on our Accessibility page.
Blogs
Facebook
Flickr
RSS Feeds
Twitter
YouTube
