The Principles of Agile Architecture

Alex Yakyma and Dean Leffingwell, with contributions from Ryan Martens and Mauricio Zamora

You’ve got to think about big things while you’re doing small things, so that all the small things go in the right direction.

Alvin Toffler

Abstract

Large enterprises undertake large software initiatives to be able to compete in the market. Indeed, they become increasingly dependent on software for their very business success. However the increasing complexity of the solutions they build, and their constant aging in the face of rapid technologic and business change, make the systems, and the enterprise that are beholding to them, less responsive over time. The systems become brittle, hard to maintain, and hard to evolve to meet the changing needs of the enterprise and the external marketplace. This jeopardizes future success, as the software legacy systems can no longer be quickly evolved to innovate—nor even minimally meet—the needs of the next generation of business opportunities.

The Scaled Agile Framework is designed to help enterprises address the current and future software solution challenges by providing a set of proven patterns enterprises can use to build new software assets and more rapidly evolve current systems. Core to SAFe is an understanding and emphasis on the critical importance that software architecture plays in helping to assure business success. In this article we reprise and revise a set of seven principles of agile architecture designed to help the enterprise reason about software design and software architecture, and balance the role that these two different elements play in building and revising enterprise class software systems.

Details

In the text Agile Software Requirements: Lean Requirements Practices for Teams, Programs and the Enterprise [1], we introduced Eight Principles of Agile Architecture to provide guidance to the development of enterprise class systems in a Lean and Agile manner. Now, a few years later, we have the experience in applying these principles, to largely good effect, in a significant number of large enterprises adopting SAFe. Through use, we’ve found it helpful to revise and reorder them to make things clearer, slightly more prescriptive, and to make the story telling a little easier in our courseware and consulting practice. In this next section, we’ll describe the Seven Principles of Agile Architecture that we now apply to address the challenges of building enterprise class software architectures in a Lean and Agile fashion:

  1. Design emerges. Architecture is a collaboration.
  2. The bigger the system, the longer the runway
  3. Build the simplest architecture that can possibly work
  4. When in doubt, code, or model it out
  5. They build it, they test it
  6. There is no monopoly on innovation
  7. Implement architectural flow

Principle 1 – Design Emerges. Architecture is a collaboration

Traditional, waterfall development methodologies used what we have come to call Big Up-Front Design (BUFD) to create a roadmap and architectural infrastructure for our future system. Our hope was that BUFD would capture requirements and architectural plans sufficiently adequate to support the system for years to come. And perhaps they did. However, we also discovered considerable challenge when it inevitably came to adapting “a massive set of speculative, forward looking constructs” to the actuality of the real world. Soon enough, the designs were brittle and hard to change, and eventually a big-branch-and-merge to a new set of speculative assumptions was the routine course of action.

Emergent Design

Agile development eschews waterfall thinking and BUFD and replaces it with a simple belief “the best architectures, requirements, and designs emerge from self-organizing teams” (Agile Manifesto [2]). Out of this comes the practice of emergent design—the evolutionary process of discovering and extending the design only as necessary to implement and validate the next increment of functionality. Teams used their new coding languages, testing tools, and refactoring skills to rapidly evolve the design in accordance with the then-current requirements. This new practice works extremely well, up to a point.

Intentional Architecture

As Agile practices matured and became adopted by larger teams and teams of teams, however, there comes a point at which emergent design is an insufficient response to the complexity of large-scale system development. Simply, it is not possible for teams to anticipate changes that may well occur outside their environment, nor for individual teams to fully understand the entire system and thereby avoid producing redundant, and/or conflicting code and designs. Simply put, no one team in a larger software enterprise can see the bigger picture, nor reasonably anticipate some of the changes that are headed their way, many of which arise outside their local control. For this we need some Intentional Architecture—a set of purposeful, planned architectural initiatives to enhance solution design, performance and usability—and which provides guidance for inter-team design and implementation synchronization.

Collaboration

Clearly, in the context of Agile enterprise software systems, we need both: fast, local control of emergent design so that teams react appropriately to changing requirements without excessive attempts to future proof the system, and global control of Intentional Architecture, the guidance needed to assure that the system as a whole has conceptual integrity and efficacy.  Achieving the right balance of emergent design and intentional architecture drives effective evolution of the system, as Figure 1 illustrates.

Figure 1. Intentional architecture, emergent design and collaboration support system evolution

Moreover, they are not independent. Intentional architecture constraints the emergent design, but at a high enough level of abstraction to allow the teams to effectively adapt the “intentional” part to their specific context. At the same time, emergent design influences and corrects intentional architecture, and also feeds new ideas for future, centralized, intentional effort.

Such a deep reciprocity between emergent design and intentional architecture can occur only as a result of collaboration between Agile Teams, System and Enterprise Architects, Product Management and even Program Portfolio Management in leveraging architecture to optimize business performance. In SAFe, one primary construct, the self-organizing team-of-Agile-teams we call the Agile Release Train, plays a key role in fostering this collaboration and building effective solutions.

Principle 2 – The bigger the system the longer the runway

In SAFe, Architectural Runway exists when the enterprise’s platforms have sufficient technological infrastructure to support the implementation of the highest priority epics and features in the backlog without excessive, delay-inducing, redesign. In order to achieve some degree of runway, the enterprise must continually invest in refactoring and extending existing platforms as well as building and deploying those new platforms needed for evolving business requirements.

In the Agile enterprise, implementation of architectural epics is complicated by the fact that the “big-bang-up-front branch-and-merge” waterfall approach is abandoned. Instead, the enterprise commits to implementing architectural epics like all other epics (and features, and so on) incrementally in the main codeline. Doing so means that architectural epics must be split into Architectural Features, which builds the runway needed to host the new Business Features, as Figure 2 illustrates.

Figure 2. Architectural runway continuously evolves to support future functionality.

Each architectural feature must be completed within a PSI such that the system always runs, at least at the PSI boundaries. In some cases, this means that new architectural initiatives are implemented piece-meal and may not even be exposed to the users in the PSI in which they are implemented. In this way, architectural runway can be implemented and tested behind the scenes, allowing shipment throughout, and then exposed to users when a sufficient feature capability exists in the next PSI or so.

The concept of runway illustrates how intentional architecture and emergent design effectively complement each other at scale: intentional, high-level ideas in support of future functionality are adapted and instantiated by agile teams; they are empowered to figure out the optimal emergent design.

Principle 3 – Build the simplest architecture that can possibly work

We welcome changing requirements even late in development (Agile Manifesto [2]). Yes, we do, but surely enabling change is facilitated by systems that have understandable designs. As Kent Beck notes: If simplicity is good, we’ll always leave the system with the simplest design that supports its current functionality [3]. Indeed, at scale design simplicity is not a luxury, but a survival mechanism. There are many considerations that help us accomplish this, to name just a few:

  • Using a simple, common language when describing the system
  • Keeping the solution model as close to the problem domain as possible
  • Continuously refactoring
  • Assuring that object / component interfaces clearly express their intent
  • Following good old design principles [4,5].

Certain approaches to designing the system, such as Domain-Driven Design [6], usage of design patterns [4], and applying metaphor [3], simplify both the design and the communication between the teams. This “social” aspect of design simplicity is critical as it enables collective code ownership, which in turn fosters feature (rather than component) orientation of agile teams [7]. Most of the approaches mentioned above consider the system as a set of collaborating, meaningful entities. This is the dominant theme in evolving maintainable and extensible solutions which that prevents programs from typical design flaws such as concentrating too much logic at the database layer (often without a real reason) or creating a thick UI, or ending up with humongous, unmanageable classes. That is, simplicity requires design skill and knowledge and agile organizations may effectively leverage Communities of Practice to continuously develop and spread these best practices.

The right balance of intentional architecture and emergent design also allows for generalization of solution elements to address common problems at the component, system or enterprise level.

Principle 4 – When in doubt, code or model it out

Making good design decisions is an art. Often (and thankfully) there are disagreements of opinion amongst team members about which course of action is best. And while Agile teams and programs don’t mind refactoring, they surely want to avoid unnecessary refactoring. In order to decide, Agile teams can typically just “code it out”, using Technical or Functional Spikes, and even rapid prototyping. Iterations are short and Spikes are small; the result is fast feedback with objective evidence, which can then be subject to A/B testing by the teams, designers and architects, or even the users.

In cases where design changes are of such scope that spikes and prototyping are insufficient, it is useful to model the problem to gain an understanding of the potential impact prior to implementation. Domain Modeling and Use-case Modeling are lightweight Agile modeling constructs that are particularly valuable in these circumstances.

Figure 3. A domain model, system context and use-case diagrams are useful in understanding the system, and in assessing the impact of architectural change.

What’s more, both are helpful; for big decisions, apply spikes and prototyping in conjunction with modeling so that everyone can see the bigger picture and the evidence it produces.

Principle 5 – They build it they test it

Testing system architecture involves testing the system’s ability to meet its larger scale functional, operational, performance, and reliability requirements. To do this, teams must typically build an automated testing infrastructure that enables ongoing system-level testing. After all, if it can’t be tested, it is assumed not to work. If the architecture is evolving, testing approaches, testing frameworks and test suites must evolve with it. The responsibility for knowing how to test a system lies with those who collaborate on the design of the system. Therefore System Architects, agile teams and the System Team actively collaborate in order to continuously Design for Testability.

Principle 6 – There is no monopoly on innovation

In our balanced model, architecture is a collaborative effort of agile teams, architects and stakeholders. This can help foster a culture of innovation whereby innovation can come from anyone and anywhere. Many ideas come from individuals, agile teams, system architects, and technical management, but they may also require centralized scaling effort to the program or enterprise level. One of the responsibilities of the Enterprise Architect is to foster an environment where innovative ideas and technology improvements that emerge at the team level do not pass unnoticed, but can be synthesized into the building blocks of the architectural runway. And as Agile can foster the “tyranny of the urgent iteration”, programmatic time for innovation can be built into occasional HIP (Hardening | Innovation | Planning) sprints.

Principle 7 – Implement architectural flow

Organizations must be able to continuously improve their process of implementing large architectural initiatives that impact the runways of multiple systems. Unless properly managed, these cross-cutting Architectural Epics will induce excessive work in process and will create long queues in affected Agile Release Trains and can thereby inhibit delivery of business value.

Figure 4. Architectural Epic Kanban facilitates the flow of enterprise level initiatives

To provide for visibility and to help optimize the flow of such initiatives, SAFe applies an Architectural Epic Kanban system, which consists of four states:

  • Funnel, where all ideas and possible initiatives are collected
  • Backlog, where the understanding of the epics gets refined
  • Analysis, where a lightweight business case is developed, and final go / no go decision is made
  • Implementation, where epics, after being split into architectural features, proceed to implementation by Agile Release Trains

Effective implementation of such a system can be a key factor in achieving enterprise success.

Summary

In this article we introduced the notion of the balance between intentional architecture and emergent design as a Lean | Agile agile approach to addressing the complexity of building enterprise class software solutions. We noted the power of collaboration as a way to harness the creative power of all participants. We highlighted the need for Architectural Runway— the foundation for future development of business value – which must be continuously extended and validated, implemented incrementally using simple design metaphors to increase maintainability and extensibility. Innovation is an important facet for success, and that comes in large part from both individuals and the collaboration between the roles. Finally, we introduced a Kanban system to enable visibility and flow of architectural initiatives that cut across multiple value streams, thereby helpings the organization continuously build an effective technical foundation for arising business opportunities.

_____________________

Learn More

[1] Leffingwell, Dean. Agile Software Requirements: Lean Requirements Practices for Teams, Programs, and the Enterprise, Boston, MA: Addison Wesley, 2011.

[2] Manifesto for Agile Software Development, http://agilemanifesto.org/.

[3] Beck, Kent. Extreme Programming Explained: Embrace Change, Boston, MA: Addison-Wesley, 2000.

[4] Bain, Scott. Emergent Design: The Evolutionary Nature of Professional Software Development, Boston, MA: Addison Wesley, 2008.

[5] Shalloway, Alan, et al. Essential Skills for the Agile Developer: A Guide to Better Programming and Design, Boston, MA: Addison Wesley, 2011.

[6] Evans, Eric. Domain-Driven Design: Tackling Complexity in the Heart of Software. Boston, MA: Addison Wesley, 2003.

[7] Larman, Craig. Practices for Scaling Lean & Agile Development: Large, Multisite, and Offshore Product Development with Large-Scale Scrum, Boston, MA: Addison Wesley, 2010.

 

Last Modified 01/02/2013

© 2011-2013 Leffingwell, LLC. All rights reserved.