Design for Testability: A Vital Aspect of the System Architect Role in SAFe
By Alex Yakyma
Note: This article is part of the Community Contributions series, which provides additional points of view and guidance based on the experiences and opinions of the extended SAFe community of experts.
The bigger the system, the harder it is to develop and maintain, and the harder it is to test. Systems that can’t readily be tested can’t readily be changed. Systems that can’t be changed can’t be developed and delivered in an Agile manner. Moreover, when dealing with legacy systems, it can be extremely difficult to add unit- or functional tests, because the system was not designed with testability in mind. In this article, we suggest that SAFe System Architects can take a key role in helping the Agile Teams Design For Testability and we will consider a number of aspects of how the system design impacts testability, and how to start improving it.
Background
Design for Testability (DFT) is not a new concept. It has been used with electronic hardware design for over 50 years. The reason there is simple: if you want to be able to test an integrated circuit both during the design stage and later in production, you have to design it so that it can be tested. You have to put the hooks” in when you design it. You can’t simply add testability later., as the circuit is already in silicon; you can’t change it now. DFT is a critical non-functional requirement that affects most every aspect of electronic hardware design. In a like fashion, complex agile software systems require testing both during design and production, and the same principles apply. You have to design your software for testability, else you won’t be able to test it when it’s done.
The Economic Value of DFT
Agile testing covers two specific business perspectives: on the one hand, it offers the ability to critique the product, thereby minimizing the impact of defects’ being delivered to the user. On the other, it supports iterative development by providing quick feedback within a continuous integration process. None of these factors can come into a play if the system does not allow for simple system/component/unit-level testing. This means that agile programs, which sustain testability through every design decision, will enable the enterprise to achieve shorter runway for business and architectural epics.
DFT helps reduce the impact of large system scope, and affords agile teams the luxury of working with something that is more manageable. That is why the role of a System Architect is so important in agile at scale, but it also reflects a different motivation: instead of defining a Big Design Up-front (BDUF), agile architect helps to establish and sustain an effective product development flow by assuring the assets that are developed are of high quality and needn’t be revisited. This reduces the cost of delay in development because in a system that is designed for testability, all jobs require less time.
Impact on System Architecture
Experience teaches us that we ultimately achieve a better design when we design for testability. Indeed it is typically the case that DFT drives a clear separation of concerns, layered architecture or service orientation, high cohesiveness of entities in the code, etc. Our tests behave very much like system “clients”—unit tests imitates the behavior of a corresponding client-class or classes invoking target class methods; component tests imitate the behavior of client-components; functional tests imitate the end user—the “client” for the entire system. If we design for testability at all these levels, we are also providing for clear and understandable interfaces between classes, components, services and, ultimately, the user interface and rest of the system.
Cultural Impact
Designing for Testability cannot be considered solely the responsibility of the system architect. All agile teams must do DFT, a continuous journey that supports product development effort between developers, testers and system architects. It creates a culture of shared responsibility for product development flow, where all actively contribute to the system design and testability.
System Architect Role and DFT
With respect to designing for testability, the system architect can play a primarily role in DFT:
| Choice of technologies | In addition to their main purpose, software libraries, frameworks, repositories and services should also support testability. For example, technologies that support the inversion of control may be useful, not only in terms of designing a flexible system, but also in relation to testability. |
| Implementation Decisions | For instance, having too much logic at DB side (a common problem of many-many products) makes testing virtually impossible. So does the unreasonable usage of asynchronous message queues within the system. |
| Design conventions | Design patterns like façade, gateway, or observer foster testability. And yet, such a common thing like using a web service proxy class may not. A good convention is to use some “abstract” interface, which would interact with the proxy, thereby allowing the substitution of one with a mock object when necessary. |
| Approach to creating fake objects and mocks | What tools and approaches should be applied to create stubs, mocks and “spies” in the code, which would support unit- and component testing? |
| Logging and dumps | In large systems, often some system-level tests fail but all unit tests pass. Thus, it may difficult to diagnose the root cause of the problem without a good logging approach and the ability to retrieve thorough memory (or protocol) dumps. |
| Flexible configuration | …This allows the simple deployment to the test environment and easy linking of test data sources and external mock objects through simply updating the configuration files. |
Table 1. Aspects of the system architect’s role in fostering system testability.
Where to Start?
The table above is intended to provide an initial set of ideas that may spawn some new architectural epics for DFT. Of course, the incremental approach is always the best, as the teams can remain focused on the value-delivery of new features and stories, while adding testability. In addition, the cultural aspect of DFT is best achieved by implementing incremental steps that show the group—after every little design initiative—that it is easier to write new tests at different levels of abstraction and thereby how the team velocity increases.
Let the teams take a few current user stories and determine what would be the minimum redesign effort to let them write a few unit tests in the areas where they previously could not. Then they can share the findings and actual approaches with others (e.g. at the nearest Design CoP meet-up). In Table 1, you can see which of the aspects described above may be affected by these findings – are there new design guidelines for all teams or changes to the system configuration, etc.?
We suggest using a similar approach at the system level: let the system team or Feature Teams try to automate system-level tests, where they would have previously failed, but let them do so in the context of their current one or two features. DFT is a big job. But like everything else we can approach it incrementally.
When we talk about agile testing and how important it is to immediately start automating testing, we must take into consideration fact that current software systems are often test-adverse. On the one hand, this is a sad fact about the industry; on the other, with Design for Testability, the agile enterprise has a renewed chance of outperforming the competition as the company enhances its ability able to deliver much more value to their users, and to do much more quickly.
Last Update: 1 November, 2015