In episode 150, I reintroduced this series with a new pitch. It was my way of taking what I've learnt over the last three years, the last 150 episodes, and almost 33 hours of content and updating the why of the podcast. Over the coming episodes, I'll take a deeper dive into the themes of the pitch and why they made the cut. In this episode, I look at why Software Development cannot be deterministic - why do have so many unknowns - why can't we make it as deterministic as moving pig-iron or flipped hamburgers at McDonalds.
Or listen at:
Published: Wed, 07 Dec 2022 16:46:40 GMT
Links
Transcript
Hi, in episode 150 I reintroduced this series with a new pitch. It was my way of taking what I've learnt over the last three years, the last 150 episodes and almost 33 hours of content and updating the "why" of the podcast.
Over a number of episodes, I'm taking a deeper dive into the themes of the pitch and why they made the cut. If you've not listened to the pitch, it's worth pausing and going back to episode 150 for context.
In episode 152 to 154, I talked about the business side of the pitch. Almost half of the pitch was devoted to the business context that we are doing our Software Development in - and why our organisations are under so much pressure to change - and how each of our organisations needs to embrace changes something they do rather than something that happens to them.
I've now moved on to exploring the second half of the pitch, the actual Software Development part.
In episode 155, I talked about how software development was a young, evolving field - a long way from achieving maturity and, as such, need support from organisational leadership to learn and improve.
And in the last episode, I talked about those management practices of yesterday and how much of our management practice stem from the work by Fredrick Winslow Taylor on improving the efficiency of manual labour at the start of the 20th century. I described how Scientific Management, or Taylorism, worked well for deterministic activities - activities in which everything about the activity was known ahead of time, including the outcomes, steps to follow and resources required - practices that are still recognisable today within places like McDonald's Restaurant, famous for low prices and fast food, for using this style as a focus on efficiency and productivity.
I ended the episode explaining why Scientific Management wouldn't work with Software Development, an inherently non-deterministic activity.
So in this episode, I want to continue this exploration by looking at why Software Development cannot be deterministic, why do we have so many unknowns, why can't we make it as deterministic as moving pig iron or flipping hamburgers at McDonalds.
To help my explanation, I want to introduce the Cynfin framework - I find this is a great framework for understanding the different complexities and thus best approaches for any given activity.
Wikipedia describes the Cynfin framework as:.
"The Cynefin framework is a conceptual framework used to aid decision-making. Created in 1999 by Dave Snowden when he worked for IBM Global Services, it has been described as a "sense-making device".[ Cynefin is a Welsh word for habitat.
Cynefin offers five decision-making contexts or "domains" - clear (known as simple until 2014, then obvious until being recently renamed), complicated, complex, chaotic, and confusion - that help managers to identify how they perceive situations and make sense of their own and other people's behaviour. The framework draws on research into systems theory, complexity theory, network theory and learning theories
In 2007 Snowden and Mary E. Boone described the Cynefin framework in the Harvard Business Review.[2] Their paper, "A Leader's Framework for Decision Making", won them an "Outstanding Practitioner-Oriented Publication in OB" award from the Academy of Management's Organizational Behavior division."
The framework is often illustrated with four quadrants known as domains - Clear, Complicated, Complex and Chaotic - with a fifth domain of Confusion at the intersection of the other four.
An activity could be described as being in that Confusion domain until we've understood which of the other four domains the activity is actually in. Under the framework, no activity should be within the Confusion domain for very long.
Over time, an activity could move through the domains. Again, from Wikipedia:
"As knowledge increases, there is a "clockwise drift" from chaotic through complex and complicated to clear. Similarly, a "buildup of biases", complacency or lack of maintenance can cause a "catastrophic failure": a clockwise movement from simple to chaotic. There can be counter-clockwise movement as people die and knowledge is forgotten, or as new generations question the rules; and a counter-clockwise push from chaotic to simple can occur when a lack of order causes rules to be imposed suddenly."
Let's take a look at the four domains: Clear, Complicated, Complex and Chaotic. During this, I'll continue to quote heavily from Wikipedia, as I believe the description of it is exceptionally good quality. The quotes will often reference quotes from Snowden and Booth - along with Thomas A Stewart and Patrick Lambe, who have both written on the subject.
Clear:
"The clear domain represents the "known knowns". This means that there are rules in place (or best practice), the situation is stable, and the relationship between cause and effect is clear: if you do X, expect Y. The advice in such a situation is to "sense-categorize-respond": establish the facts ("sense"), categorize, then respond by following the rule or applying best practice."
In the previous version of the framework, this was described as "Simple". This is the domain within which all of those deterministic tasks take place - whether it be the manual moving of pig iron or flipping hamburgers - it's fully known and we just need to follow the predefined rules.
Complicated:
"The complicated domain consists of the "known unknowns". The relationship between cause and effect requires analysis or expertise; there are a range of right answers. The framework recommends "sense-analyze-respond": assess the facts, analyze, and apply the appropriate good operating practice. According to Stewart: "Here it is possible to work rationally toward a decision, but doing so requires refined judgment and expertise. ... This is the province of engineers, surgeons, intelligence analysts, lawyers, and other experts."
Complex:
"The complex domain represents the "unknown unknowns". Cause and effect can only be deduced in retrospect, and there are no right answers. "Instructive patterns ... can emerge," write Snowden and Boone, "if the leader conducts experiments that are safe to fail." Cynefin calls this process "probe-sense-respond". Hard insurance cases are one example. "Hard cases ... need human underwriters," Stewart writes, "and the best all do the same thing: Dump the file and spread out the contents." Stewart identifies battlefields, markets, ecosystems and corporate cultures as complex systems that are "impervious to a reductionist, take-it-apart-and-see-how-it-works approach, because your very actions change the situation in unpredictable ways."
Chaotic:
"In the chaotic domain, cause and effect are unclear.
Events in this domain are "too confusing to wait for a knowledge-based response", writes Patrick Lambe. "Action - any action - is the first and only way to respond appropriately."
In this context, managers "act-sense-respond": act to establish order; sense where stability lies; respond to turn the chaotic into the complex.
Snowden and Boone write: In the chaotic domain, a leader's immediate job is not to discover patterns but to staunch the bleeding. A leader must first act to establish order, then sense where stability is present and from where it is absent, and then respond by working to transform the situation from chaos to complexity, where the identification of emerging patterns can both help prevent future crises and discern new opportunities. Communication of the most direct top-down or broadcast kind is imperative; there's simply no time to ask for input.
The September 11 attacks were an example of the chaotic category.
Stewart offers others: "the firefighter whose gut makes him turn left or the trader who instinctively sells when the news about the stock seems too good to be true." One crisis executive said of the collapse of Enron: "People were afraid. ... Decision-making was paralyzed. ... You've got to be quick and decisive - make little steps you know will succeed, so you can begin to tell a story that makes sense."
Now Chaotic would be a bad place for your Software Development to be. And while it can sometimes feel that it's Chaotic, it's much more likely to be in that Complicated or Complex domain.
That being said, one example of where you would be in a Chaotic domain is if you're under active cyber attack. You may know, or believe, an attack is in progress, but you don't know enough to be able to respond with anything other than action - maybe turning off production servers in the hope that it will mitigate any longer term damage.
As I've said, Software Development activities will largely be in the Complicated or Complex domain. Interestingly, in his book "Sooner, Safer, Happier", Jonathan Smart describes Complicated as the sweet spot for using Lean and Complex as a sweet spot for using Agile - and I would concur with that.
The Complicated domain is repetitive and good practice works well here, which marries up well with the Lean Software Development techniques, which focus so heavily on waste reduction techniques.
The Complex domain is unique and emergent practice works well here, which suits the outcome focussed approach of Agile Software Development well.
Being practical, though, different software work can fall into either camp.
So maybe your team is working predominantly in the Complicated - that isn't to say a specific piece of work won't fall into the Complex and vice versa.
So while I like the link between Complicated/ Complex and Lean/ Agile, local software practice need to be tailored to the environment.
Unfortunately, there is no one size fits all approach - something I want to explore further in the next episode.
The key takeaway from this episode, however, is that Software Development is not in the same domain as moving pig iron or flipping hamburgers - and the Cynefin framework helps us to understand that difference.
The deterministic practice, like moving pig iron and flipping hamburgers, falls nicely into that Clear domain, which is best served by "Best Practice" - very much the Scientific Management approach advocated by Frederick Winslow Taylor.
Software Development, however, will never be within the Clear domain. It will always be within the Complicated or Complex domain - with hopefully vanishingly rare occurrences into the Chaotic. And being in those different domains requires different approaches, different ways of thinking than the Scientific Management approach is suited for. If anything, as I covered in the last episode, attempting to use Scientific Management approaches will actually produce dysfunctional behaviour and negative outcomes.
In next week's episode, I want to explore the idea that there is no one size fits all approach to Software Development.
Thank you for taking the time to listen to this podcast and look forward to speaking to you again next week.