This is easy in small groups where there is an innate understanding of each other’s abilities and expectations are clear. However, this mechanism starts to break when you try to scale up to bigger groups. While there might be discussion around the validity of Dunbar’s number, we all experience that it is harder to stay aligned in larger groups [1]. 

This is where governance comes to the rescue. Governance, in essence, describes the company structure required for decision-making, accountability, and control. Governance is straightforward in linear and static systems; once the dependencies are mapped, they require minimal maintenance. For a long time, this worked relatively well even for development teams, but it all started to change with the mass adoption of the agile way of working.  

A quick look at the Agile Manifesto might lead to the conclusion that the basis of the classical governance model is seen as less valuable [2]. Agile encourages team autonomy, experimentation, and continuously changing things for the better (i.e. the statements on the left), while governance relies on certainty and clearly defined processes (i.e the statements on the right).

figure 1 agile manifesto 1 1
Figure 1: The Agile Manifesto

From a governance perspective, this leads to the following challenges: 

  • Accountability can become unclear, requiring a longer time to identify whom to contact when needed (e.g., during an incident). 
  • Less strict documentation and processes result in less visibility and reliance on trust (which is counterintuitive in the case of governance). It breaks the ‘trust but verify’ dogma. 

A first reflex might be to apply more vigorous enforcement of the right-hand parts of the manifesto to keep the right level of accountability. However, research around safety shows us that, statistically, impositions of these kinds are ineffective and may even worsen the situation. The reason is not freewill, autonomy, or unwillingness, but simply that most people are unable to comply with an intent that didn’t originate from themselves [3]. 

Though at first sight this seems like an unsolvable either/or problem, a simple polarity map shows us how to navigate towards viable solutions [4].

figure 2 polarity map 1 1
Figure 2: polarity map of agile governance challenges

A successful approach to governance in agile environments should stimulate the outcomes in the upper half of the map, while preventing the outcomes in the lower half of the map.   

At first this seems to present direct opposites like e.g.:  

  • Resilience vs Robustness 
  • “Creativity and innovation” vs “Clear direction and goals” 
  • “Collaboration and Adaptability” vs “Consistency and Predictability” and “Standardized Processes” 

Luckily, this is a well-researched problem, and, over the years, solutions have been identified on both the agile and governance side of the spectrum. (Fun fact: in April 2018 a group of DevOps specialists even wrote an apology letter directed at auditors, acknowledging the problems caused, and promising to do better in the future [5]). 

For example, both ‘Disciplined Agile’ and ‘Complex System Governance’ are frameworks that address the problem and solutions for governance from their specific perspectives but essentially describe similar solutions [6,7]. 

Disciplined Agile 

Disciplined Agile (DA) is an agile framework that provides a flexible and pragmatic approach to software development and project management. It builds on the principles of Agile and Lean practices while offering guidance for tailoring agile methods to fit the unique needs and complexities of an organization. 

Complex System Governance 

Complex system governance refers to the management and control of complex systems, which are dynamic networks of interconnected elements that exhibit emergent behaviors and interactions. In the context of governance, complex systems pose unique challenges due to their non-linear, unpredictable nature and the interconnectedness of various components. Traditional top-down, hierarchical approaches to governance may not be sufficient to address the complexities and uncertainties inherent in such systems. 

Disciplined Agile Complex System Governance 
Flexibility: Disciplined Agile recognizes that different projects and teams have varying needs and constraints. It offers a wide range of practices, processes, and lifecycles to choose from, allowing teams to tailor their approach.  Decentralization: Recognizing that complex systems are often composed of multiple interdependent parts, governance may involve decentralizing decision-making authority to allow for more responsive and context-specific actions.  
Integration: Disciplined Agile integrates various Agile and Lean approaches, such as Scrum, Kanban, and Extreme Programming (XP), to provide a coherent and comprehensive framework.  Experimentation: Experimenting with different approaches and policies within the system can help identify effective governance strategies and avoid unintended consequences.  
Scaling: The framework provides guidance for scaling Agile practices to larger teams and organizations, allowing for coordination and alignment across multiple teams.  Participation and Collaboration: Involving stakeholders from diverse backgrounds and perspectives fosters a more comprehensive understanding of the system, enhancing the effectiveness of governance interventions.  
Inclusive: Disciplined Agile is not limited to software development projects; it is inclusive of various business areas, including data management, IT operations, and governance.  Resilience: Fostering the resilience of complex systems is crucial, as it allows them to absorb shocks and recover from disturbances more effectively. 
Pragmatism: The framework encourages a pragmatic approach to Agile implementation, recognizing that the “right” approach is the one that works best for a given context.  Feedback Loops: Regularly collecting data and feedback from the system helps to monitor its behavior and performance, enabling governance to be continuously refined.  
Decision Making: Disciplined Agile emphasizes making decisions based on the needs of the situation, including factors like team size, domain complexity, and business goals.  Adaptive Management: Governance strategies need to be flexible and adaptive, capable of adjusting to changing conditions and new information that emerges from the system.  

Table 1: comparison between Disciplined Agile and Complex System Governance 

What does work 

It all starts with getting clarity on what autonomy means in your organization. In his book ‘Leading Teams’ J. Richard Hackman distinguishes four distinct types of teams: Manager-lead, self-managing, self-designing, and self-governing [9,10]. When there is unclarity about who is in the driver seat for a specific topic, this will lead to confusion or conflict. Therefore, the governance model should be aligned with the team structure. The easiest way to get this clarity is simply to ask product owners and management about their viewpoints and identify the overlaps and gaps.

Next, design a governance model together with the teams. Collaboration with teams is more effective than forcing them to conform to an external imposition and will lead to a higher sense of accountability. A good approach is using promises instead of impositions [3]. There is more to this than just rephrasing or wording. An imposition forces a particular behavior without explaining the reason behind it, while a promise states a clear outcome and allows for freedom on how to achieve that outcome. A governance model will be more successful when it is supported by people who understand its strategy and are based on positive goals [11]. 

The Authority Matrix by J. Richard Hackman
Figure 3: The Authority Matrix by J. Richard Hackman

Once the goals are clear and promises are documented, make processes as simple as possible and fit for automation. Many processes are still based on collecting screenshots from various systems copied into documents that must be placed in another tool, or mandatory filling in massive checklists for every tiny change.  These slow processes consume loads of time and fail both employees and regulators as they cannot keep up with how fast things change [12].  

Assuming the natural tendency for teams is to do the most straightforward thing possible, it is crucial to make ‘doing things the right way’ also the easiest way. This will encourage adoption by teams earlier in the process and keep teams and projects moving forward, adapting to change when necessary. Balancing automation and human intervention should be the basis of your governance strategy. 

Automation also provides many options for low-effort feedback loops on the checks in the process and can be easily implemented when it is designed to integrate in an already established workflow [7]. The learning opportunity should not be underestimated, as it provides teams with insights into their own process. This insight gives teams the information needed to secure and fix what is not working [13]. By implementing these mechanisms, you achieve the ‘easy way principle’, since breaking a documented promise becomes a conscious action.  

An advantage of automation is continuous monitoring. By leveraging existing tools, automation will provide rapid feedback for process adaptation, promoting innovation and competitiveness, while reducing overhead, enhancing visibility, usage, and effectiveness of controls [13]. This is crucial in the modern complex landscape of interconnected risks [12]. Continuous monitoring enables oversight of the tech supply chain, data aggregation for reporting, and timely deviation alerts [14]. 

Lastly, stress test your model. Stress testing is usually a Quality Assurance activity that subjects systems to extreme conditions beyond normal operating limits. This deliberate pushing of boundaries helps uncover vulnerabilities and weaknesses that might remain hidden during regular operations. The same approach should be used to test your governance model, e.g., by using TRIZ [15]. By simulating excessive load or incidents, stress testing reveals how your governance model holds under pressure, revealing crucial insights into potential bottlenecks, failure points, or design issues. Proactively testing a model helps identify areas for improvement, efficient ways of using data and ensures the model is effective – which are all fundamental to the agile principles [2]. This continuous improvement approach enables organizations to address and rectify issues before they escalate, enhancing overall reliability and resilience in an agile manner. 

Conclusion  

The concept of governance in agile environments remains crucial for fostering effective collaboration and ensuring accountability. While smaller groups may naturally understand each other’s expectations, challenges arise when scaling up due to potential misalignment. Governance should provide a structure for decision-making, accountability, and control. However, the principles of classic governance models can clash with agile values, which emphasize autonomy and experimentation. 

To address these challenges, the governance framework must be a flexible approach that encourages creativity, collaboration, clear goals, and adaptability while preventing inefficiency and loss of visibility. Effective governance involves aligning team autonomy with the organizational structure, designing governance models collaboratively, and simplifying processes for automation. Continuous monitoring, enabled by automation, ensures proactive oversight and timely alerts, and promotes innovation. Continuous improvement of the framework can be achieved by frequently stress testing the governance model. Agile governance allows organizations to navigate complexity, foster innovation, and maintain resilience. 

For those inspired by this topic and want to know more about it, we have added an exciting video, article, and book to the list of sources [12,16,17].

Sources 

  1. https://en.wikipedia.org/wiki/Dunbar%27s_number 
  1. https://agilemanifesto.org/ 
  1. Promising Digital Risk Management; Mark Burgess, Patrick Debois, chapter 3.1 
  1. https://www.sloww.co/polarity-thinking-101/ 
  1. http://dearauditor.org/ 
  1. https://www.pmi.org/disciplined-agile 
  1. https://www.jlab.org/sites/default/files/accel/docs/System%20Engineering_604/Article_Keating%20and%20Katina%20Complex%20System%20Governance.pdf 
  1. https://en.wikipedia.org/wiki/Cognitive_load 
  1. The design of work teams, Hackman, 1987, p. 334 
  1. https://www.linkedin.com/pulse/self-organization-versus-self-management-marty-de-jonge/ 
  1. https://blog.antwerpmanagementschool.be/en/why-fud-fails-and-bad-prevails 
  1. https://www.youtube.com/watch?v=2AiwvQcFLsU 
  1. The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis, chapter 22 
  1. The DevOps Handbook; Gene Kim, Jez Humble, Patrick Debois, John Willis, chapter 23 
  1. https://www.liberatingstructures.com/6-making-space-with-triz/ 
  1. https://humanisticsystems.com/2014/03/08/safety-ii-as-disruptive-innovation/ 
  1. https://itrevolution.com/product/investments-unlimited/