It takes less time to do things right than to explain why you did it wrong.

—Henry Wadsworth Longfellow

Building in Quality and Compliance

This is article eight in the SAFe for Government (S4G) series. Click here to view the S4G home page.

Local and national governments procure technology solutions for the welfare and security of their citizens. Indeed, governments define and procure some of the largest, most sophisticated, costliest, and most complex systems and solutions known to man.  Many of these systems are high- assurance systems where the human and economic costs of failure are simply unacceptable (nuclear technologies, space stations and spacecraft, medical devices, intelligence gathering systems, etc.). And now, with the rise of modern cyber threats, security failures of many types of systems (citizen data, social and welfare systems) can be equally catastrophic.

To assure the safety and efficacy of these programs, governments historically mandated that stage-gated and sequential ‘waterfall development’ processes be used to build these systems. Characteristics of this model include:

• Government fully specifies the system up front
• Suppliers build the solution to specifications
• The as-built system is passed to separate organizations for quality and compliance testing

Unfortunately, defects found late in the process often initiate massive redesign and rework (or worse, defects are not found, creating vulnerabilities to the system). Because of the rework, it might be months or years after a capability is initially developed before the solution can be fielded and the value received. In the interim, the mission needs often have changed so dramatically that by the time the solution is deployed, this expensive new technology is already out-of-date and no longer fit for its intended purpose. This often results in a massive waste of taxpayers’ money, subjects them to unnecessary risk, and denies them the benefits these systems were intended to provide.

The standard waterfall practice of deferring testing and quality activities to the end of a project is a significant contributing factor to these delays. Obviously, however, taking shortcuts to quality and compliance is not the solution. What is needed is a way to execute high assurance systems development in a way that verification, authorization, and accreditation activities are not pushed to the end of the development lifecycle. Any such alternative approach must enable faster delivery of such systems without decreasing the rigor or effectiveness of assurance outcomes.

This article describes the approach embodied in SAFe. While many of these concepts are already well documented in the Compliance article on the SAFe website, and further detailed in our white paper ‘Achieving Regulatory and Industry Standards Compliance with the Scaled Agile Framework® (SAFe®),’ (downloadable here), this article describes quality and compliance guidance that is particularly applicable to the government context.

Details

No matter the development approach, government programs must integrate quality and compliance concerns into staffing, budgeting, and planning discussions. Technology initiatives have to satisfy numerous compliance standards for both the technical attributes of the solution and the processes used to build it. They might be based on international standards such as ANSI, ISO, CMMI, etc., or country-specific laws and the policies and regulations of particular government agencies.

Traditionally, most quality and compliance activities have taken place at the end of a project primarily because, as the top of Figure 1 illustrates,  a working solution is not available for testing until the end. But the bottom of the Figure illustrates how an agile approach to building a system incrementally presents an opportunity to change how compliance work is conducted.

With SAFe, compliance activities can focus on smaller parts of the system as they are built. Two critical benefits explain how this improves the quality and compliance of the system:

First, compliance and quality officials have greater visibility and more opportunities to provide feedback when they are inspecting smaller parts of the system. (It’s easier to find defects in 500 lines of code compared to 500,000 lines of code.)

Second, any quality and compliance issues discovered can be fixed more effectively early, when the cost of change is small. If not corrected early, other parts of the system will have been built on defective components, thereby creating new issues of their own, and exponentially compounding the effort required to fix the problem.

Clearly, agile development is a better approach. The remainder of this article describes three essential practices that government organizations can use to build quality and compliance into the development process.

Integrate IV&V and Compliance Activities within Development

Many government agencies use independent verification and validation (IV&V) practices as part of the quality and compliance process. IV&V commonly occurs in a testing phase once the solution is completely developed. Instead of this work being performed separately following development, the Lean-Agile best practice is to integrate this work into the effort required to bring each Feature to completion, as illustrated in Figure 2.

During backlog refinement and PI Planning, the Feature’s size and scope include all of the work, including quality and compliance activities. Knowledge from those activities is then applied immediately as inputs to the next features to be pulled from the backlog. Although frequent feature validation and review will not provide all the testing needed before releasing the solution, it can help make the final validation checks smoother and uneventful.

But making this shift is not a trivial effort. Government quality and compliance organizations and suppliers are often staffed and managed based on the phase-gated model. Conducting these activities in smaller batches, on smaller pieces of the solution every PI (or even Iteration), requires changes to resource management and planning. If the organization doing this work is a third-party contractor, the change can also alter the terms and conditions of their contract with the government. Two strategies help mitigate these barriers to change.

First, many of these checks can be automated. Incorporating compliance tests into the automated build process ensures that the build does not pass unless the feature passes its compliance requirements. (And yes, even hardware-based systems can incorporate automation in the testing process.) This reduces manual verification and frees quality and compliance personnel to focus on higher-level activities.

Second, shifting to continuous IV&V can be incrementally tested and refined in pilot projects—before the approach is implemented across all agency programs.

Finally, SAFe provides routine opportunities to incorporate quality and compliance concerns directly into the various aspects and events of the development process. Figure 3 illustrates just a few examples, each of which is described further below.

Kanban and backlog – For proper estimation, prioritization, and dependency management, quality and compliance activities must be visible. Features should include stories that reflect the work needed to build-in quality and compliance as part of the definition of done (DoD).

Roadmap – Even after shifting many of the compliance activities into developing the feature, quality and compliance milestones will continue to be critical points in the delivery timeline (for example, the final check flights for a new aircraft). These events should be reflected on the solution roadmap.

Solution Intent – Quality and compliance artifacts, including test scripts and results, should be recorded in the Solution Intent repository. They should be used as inputs to the relentless improvement process.

PI Planning – Whether they are incorporated directly into the composition of the ART, or interact as Shared Services, quality and compliance experts should be included in the PI Planning process. Compliance work should be reflected in a team’s plans and PI Objectives.

PI Demo – Teams should consider how quality and compliance can be demonstrated during System Demos as objective evidence that these requirements are being built into the solutions while they are being developed.

Inspect and Adapt – In addition to the demonstration portion of Inspect and Adapt (I&A), quantitative and qualitative metrics of quality and compliance should be presented during I&A. Any issues discovered, and the quality and compliance processes themselves, can become the focus of improvement efforts.

Moving Forward

The next key concept for adopting SAFe in Government is Adapting governance practices to support agility and lean flow of value.