Stochastic Influences On Sprint Outcomes: Illusion Of Control In Retrospectives
The illusion of control in agile software development
The agile methodology emphasizes iterative development, continual testing, and adaptive planning. This promotes a sense of control and predictability in outcomes. However, software development inevitably involves many stochastic elements that reduce control.
Uncertainties arise in user needs, technical dependencies, testing results, and task estimation. Developers still exhibit cognitive biases that overestimate their control over outcomes. The sprint retrospective risks reinforcing these biases if teams do not properly account for stochastic influences.
User needs
The agile methodology relies on constant user feedback to drive development. But user needs evolve unpredictably, no matter how much upfront analysis occurs. Sudden changes in leadership, priorities, budgets, and markets can rapidly alter needs after a sprint begins.
Technical dependencies
Development tasks inevitably rely on layers of technical infrastructure and dependencies. Upstream API changes, resource constraints, network outages, and a myriad of unknown unknowns can suddenly impede progress at any time.
Testing surprises
Agile testing reduces surprises through continual integration and automated checks. Yet intricate systems still result in emergent behaviors that evade test coverage. Progress can halt for days as teams investigate mystifying results.
Estimation challenges
Task estimation remains more art than science. Unforeseen complexities consistently undo initial estimates. Poor estimates then require difficult scope tradeoffs mid-sprint, resulting in discomfort and frustration.
Cognitive biases that promote overconfidence
Decades of research shows that the human mind consistently exhibits biases that promote illusion of control even in stochastic environments. Retrospectives risk reinforcing overconfidence if teams focus purely on internal processes without considering external uncertainties.
Overprecision
People instinctively communicate more precision than truly exists. For example, task estimates presented quantitatively as 4 days imply unjustified precision versus a range of 2-6 days. Such fake precision wrongly implies predictability.
Confirmation bias
People overweight information confirming initial beliefs and underweight contradictions. For instance, positive testing results receive more attention than ambiguities pointing to hidden issues. This distorts retrospectives.
Hindsight bias
Outcome knowledge changes how people view past events as more predictable than they truly were. This afflicts retrospectives when teams judge decisions based on sprint outcomes rather than considerations at the time.
Fundamental attribution error
People overly attribute their own failures to circumstances outside their control while attributing other’s failures excessively to personality factors. But the opposite bias holds for success. This damages objective retrospectives.
The sprint retrospective as a biased review process
Given the above realities, sprint retrospectives exhibit substantial biases if teams simply review results without considering environmental uncertainties. Such biases negatively impact sprint planning and hide risks.
Anecdotal sampling
Recent, memorable events carry excessive weight during reviews. Yet sprints contain hundreds of small stochastic influences lost to memory. Missing crucial variables due to anecdotal sampling precludes holistic improvement.
Shared hindsight
Collaborative teams develop shared mindsets prone to the same judgment errors. Unquestioned assumptions seem obviously true in hindsight despite inspiration from random interim results. Groupthink then solidifies flawed sprint strategies.
Misattributed outcomes
Retrospective counterfactual analysis crucially hinges on properly attributing sprint outcomes between process versus environmental factors. Misattribution precludes correctly separating signal from noise, thereby corrupting learnings.
Over-extrapolation
Beyond misattribution errors, people instinctively perceive trends in randomness and place excessive weight on temporary patterns. In complex initiatives like software development, such over-extrapolation blows stochastic noise into seemingly deterministic processes.
Quantifying unpredictability in sprint outcomes
While variability exists everywhere, quantification provides grounding. Measuring fluctuations demarcates plausible expectations versus unrealistic consistency. This mitigates biases by highlighting randomness amid supposed predictability.
Typical sprint volatility
Typical sprints completes only 60-80% of targeted work despite full effort, with user story point totals fluctuating substantially week to week even on stable teams. Such outcomes remain incorrectly perceived as surprises warranting reactive changes.
Story point shortfalls
Teams estimate story points early during sprint planning when uncertainties run high. Later emergence of known unknowns and unknown unknown technical hurdles causes typical ~35% average shortfalls relative to initial sociable story point forecasts.
Defect escape rates
Rigorous test automation across integration, performance, security, and reliability testing dimensions still yields 5-15 defects escaping per average sprint into production. No amount of retrospected process improvements reduces this into insignificance.
Production incidents
Similarly, mature high-reliability software systems at leading firms still suffer multiple production incidents per month on average. Random outages remain unavoidable laws of nature rather than process deficiencies that retrospective insights can eliminate.
Adjusting retrospectives to account for stochasticity
Skilled teams thus run adjusted retrospectives fully factoring in sprint volatility, defects rates, and incidents as inherent Environmental constraints rather than preventable Process issues. This mindset shift reduces unrealistic expectations from endless process refinement.
Introducing measurement
Adopting basic measures for estimation accuracy, throughput volatility, defect escape rates, and incident counts seeds data-driven conversations. Such metrics highlight base variability versus excess outliers warranting closer scrutiny for true process issues.
Embracing uncertainty
Transparent measurement begets honest dialog. Teams mature to understand uncontrollable uncertainties, separating signal from noise. Such collective mindfulness of inevitable unknown unknowns and Black Swan events leads to psychological safety in sprint planning and execution.
Focusing on adaptability
Accepting irreducible uncertainty redirects retrospectives toward adaptive capacity. Teams implement decoupling architectures, fault isolation techniques, load shedding, rollback mechanisms, and modularization strategies to improve resilience. This outperforms unrealistic attempts at prediction and control.
Building slack time into sprints
Process rigor alone cannot overcome environmental randomness. Attempting to extract maximum productivity from constrained sprints inevitably breeds delays from unavoidable variability. Savvy teams thus embrace slack time as key sprint buffer against volatility.
Reducing utilization
Traditional process efficiency metrics like utilization/realization wrongly imply predictability amid variability. Leading teams now run only 70-80% utilization targets, with slack time for digestion, unknowns, recovery, and learning.
Expanding uncertainty ranges
Likewise, task estimates now range from 50% on the low end as protection against Black Swans to 200% on the high end as padding against volatility. Such expanded ranges anchor initial planning to foreseeable outcomes.
Raising maturity level
Increasing slack directly raises timeboxed sprint maturity by definition, counterintuitively resulting in more consistent throughput despite lower expectations. This further shields teams against unrealistic assumptions of consistency.
Embracing uncertainty as a core agile principle
Ultimately, sprint execution and retrospective analysis must fully accept overwhelming evidence that software development fundamentally constitutes an uncertain undertaking, no matter how precisely defined processes attempt to manufacture unrealistic predictability and controllability.
Destigmatizing unknowns
Leaders must relentlessly teach that unknown unknowns represent integral reality rather than something to be ashamed about and then retrospected away. Normalization prevents stifling psychological unsafety when near-constant surprises arise.
Celebrating slack
Similarly, realizing partial plans and finishing early should receive enthusiastic acclaim during retrospectives as demonstrations of wisdom and maturity rather than wasted potential. This further promotes slack time as tool for managing variability.
Quantifying the unquantifiable
Paradoxically, measurement of uncertainty itself – via variability ranges, volatility indices, surprise ratios, defect escape counts – makes unknown unknowns more visible and discussable. Retrospectives can then track such metrics to confirm behaviors stay within expectations.
