Reflections 13th July
The Power of Heuristics.
There are times in our lives when the walls we shelter behind disappear for a moment, and we find ourselves with a direct, high-speed connection to awe and mystery. The birth of our first child, the death of a loved one, a natural disaster, or sometimes, if we’re lucky, a glimpse through the mist of an insight that has been eluding us.
The idea of spaces has continued to dominate my thinking for the last week. The spaces we find ourselves in, the spaces we wish we were in, and the spaces that connect the spaces.
We need boundaries to function; without them, we would go mad. If we awoke every morning to think about the vastness of space and our part in it, we would probably never get out of bed.
We also need to remember that we have created the boundaries that are there, and have responsibility for them.
They are constraints we have chosen, and can change, if we wish.
Many of us have made a Faustian bargain with the organisations we work for. We trained for their needs and worked on their terms, in return for a career path and relative safety, but as businesses prioritised productivity and efficiency, we accepted the compromise that made us, in effect, human parts of a superhuman machine.
For decades, a century even, it worked for us. In the last couple of decades, however, the deal has changed. We have moved from aspiring to be a star of the show to an acute awareness that we are extras, hired from scene to scene, according to need and budget.
It would be reassuring if there were “bad guys”, some comforting conspiracy theory we could shelter behind, but there isn’t. We just happen to be here at this time, in the spaces we occupy, as the system we have created evolves.
Nobody in the organisations we are part of wishes us harm, because there is no benefit for them (other than the odd sociopath or two).
As Donella Meadows reminds us, tinkering with parts of the system is nowhere near as powerful as changing the mindset of those who comprise it.
Maybe we need to stop worrying about the parts and pay more attention to the whole.
The system we have created is bent out of shape. I enjoy Roger Martin’s work on strategy, and perhaps the most memorable element for me is not the processes he advocates, but a very simple model of a knowledge funnel. He demonstrates a process of extracting heuristics from mysteries and transforming them into algorithms.
Heuristics are strategies based on rules to generate optimal decisions.
These strategies depend on using readily accessible, though loosely applicable, information to control problem solving in human beings, machines and abstract issues. When an individual applies a heuristic in practice, it generally performs as expected.
It is, I think, at the heart of our current challenges. We have become very good at converting the approximation of heuristics into algorithms that offer the seductive precision and ease of automation that efficiency loves, but have lost the essential curiosity and scepticism that gives them their power.
We have overdosed on algorithms and largely deserted the creative work of creating new heuristics. The result is that we are losing our intuitive connection to mystery. We have concentrated on the extraction of what we know and forgotten the joy of discovery.
Algorithms offer lower risk returns than playing with mysteries. A quick look at the money being made in angel and venture capital underlines it - people creating marginally better digital widgets to sell to larger companies who will buy them once proven, rather than developing their own. Some are “Killer” acquisitions, where start-ups are bought to prevent them from disrupting established businesses. Others are “fillers”, where there is a marked reduction in development activity once the space the acquisition was needed for has been filled.
Market pressure from investors encourages “safe” startups designed to sell, in a way that reminds me of new build homes.
All highly profitable, but stunningly boring.
I feel that we have fallen into a polarity of sorts. At one end, we have the algorithmists, focused on data, measurement and short-term performance. Organisations that crave productivity and efficiency, with scale as one of the most effective ways of achieving it.
At the other end are those who embrace mysteries, from the scientific to the spiritual, but do not want to contaminate their approaches with commercialisation. It can make them insular and defensive, creating shelters by doubling down on the esoteric nature of their approaches, pulling up the drawbridge that might connect them to those who can do the work of making them productive.
There is a whole industry that offers these shelters, encouraging us to be more spiritual at work, but without bridging it effectively to the brutal reality of the performance necessary to survive in the algorithmic world of measurable short-term performance.
There is a conflict. We cannot embrace mystery at scale. Entertaining the unknown requires trust, relationships and a willingness to experiment, fail and try again.
The bridge from mystery to algorithm is heuristics, and, as I will come to, heuristics are the domain of a particular type of person, and potentially, organisation.
I think it is one of the reasons we have moribund economies. We are torn between the safety of certainty, the attraction of connecting to something larger, and the risks involved in the true discovery to be found in adventurous speculation.
No matter the rhetoric or the values on the company website, Capital is voting for safe certainty every time, and Capital calls the shots.
We are missing those who develop heuristics, those messy, approximate models that can capture aspects of mystery and move us forward as society. Heuristics are both the starting point of science and the defining property of craft. Science seeks to remove the uncertainties, whilst craft works with them. Science and Craft are complementary. Unfortunately, scale doesn’t see it that way.
Rather than invest in the discoveries to be made heuristically, we have a class of investors who seem to prefer to invest in the way we strip mine for minerals - laying waste to the land in search of only a tiny part of what is to be found there. They discard the potential of partly understood heuristics in favour of the quick hit offered by marketable certainty.
It concerns me that until those who provide capital can embrace the uncertainty of heuristics as eagerly as they do the certainty of algorithms, we are unlikely to make progress. It’s why I believe in the power of
.There are those operating heuristically out there, but interestingly, they seem to be more in the public sector than in industry. There is the UK’s Policy Lab, Singapore’s Smart Nation Initiative, and individuals like Luiz Antonio Joia running experiments in Brazil. These areas, though, are funded, and as we have seen spectacularly in the US, and to a lesser extent in the UK, vulnerable to those who worship the guaranteed returns of private capital investment. There have been initiatives, like BCorps, that claim to offer a gentler approach, although they increasingly seem to have been captured by those large corporations who want to go along for the PR ride rather than the change in attitude it requires.
Which is more than a shame, because the Heuristic space matters.
I’ve enjoyed some good, and often challenging, conversations this week around boundaries. Across, I’ve created a NOTICE framework as a way of navigating inside the space that is work. In doing so, I’ve created a boundary that excludes a good chunk of the “mystery” space. The exclusion feels uncomfortable, but the experiment I want to follow requires focus, specifically in the awareness of what it excludes. In this respect, it is a “stepping stone”, not a wall.
Outside the Walls is about exploration without boundaries.
That said, for the next few weeks, I’m going to “make camp” in the “heuristics” space. I want to explore where heuristics are welcomed and encouraged, who is doing it, what they are uncovering and what they are offering the prospect of. I am keen to see whether there are spaces that New Artisans might find sustenance in bridging the space between mystery and algorithmic scale through the intimacy and originality of craft.
I suspect those who live in heuristic spaces exhibit particular attitudes and dispositions. They are those who move beyond the boundaries of “respectable” uncertainty, where risks are tamed through probabilities, knowledge, and even “known unknowns” - to spaces characterised by radical uncertainty, facing up to mystery on unknowables.
Going back to space for a moment, people like Chris Hadfield, whose book “An Astronaut’s View of Life on Earth” is a personal favourite, personify a heuristic approach.
You may recall him as the man who sang Bowie’s “Ground Control to Major Tom” from the Space Station, and who learned Russian in order to ride a Russian rocket to get there.
There are some lines in the book that stay with me. One involves learning to relax as you focus on working out what, in the next action you take, might kill you.
There is a chapter entitled “Square Astronaut, Round Hole” describing what he learned when trying to do his first space walk.
“Square astronaut, round hole. It’s the story of my life, really: trying to figure out how to get where I want to go when just getting out the door seems impossible. On paper, my career trajectory looks preordained: engineer, fighter pilot, test pilot, astronaut. Typical path for someone in this line of work, straight as a ruler. But that’s not how it really was. There were hairpin curves and dead ends all the way along. I wasn’t destined to be an astronaut. I had to turn myself into one.”
Chris Hadfield, An Astronaut's Guide to Life on Earth
Another is his notion of “being a zero”:
“In any new situation, whether it involves an elevator or a rocket ship, you will almost certainly be viewed in one of three ways. As a minus one: actively harmful, someone who creates problems. Or as a zero: your impact is neutral and doesn't tip the balance one way or the other. Or you'll be seen as a plus one: someone who actively adds value. Everyone wants to be a plus one, of course. But proclaiming your plus-oneness at the outset almost guarantees you'll be perceived as a minus one, regardless of the skills you bring to the table or how you actually perform…
When you have some skills but don't fully understand your environment, there is no way you can be plus one. At best, you can be a zero. But a zero isn't a bad thing to be. You're competent enough not to create problems or make more work for everyone else. And you have to be competent, and prove to others that you are, before you can be extraordinary. There are no short-cuts, unfortunately.”
Chris Hadfield, An Astronaut's Guide to Life on Earth
I think Angela Merkel, Tim Berners Lee, Jonny Ive of Apple, James Dyson, David Attenborough, and Helen Sharman, among others, were all zeros.
They are, or were, heuristic people:
Successes teach you nothing. Failures teach you everything. Making mistakes is the most important thing you can do." James Dyson
"We're taught to do things the right way. But if you want to discover something that other people haven't, you need to do things the wrong way." James Dyson
"Be conservative in what you do and liberal in what you expect" Tim Berners Lee”
No one will protect what they don't care about; and no one will care about what they have never experienced. David Attenborough
Meaningful progress comes from “zeros”. Those who work heuristically to make the paths we need by walking.
There are zeros all around us, individuals and businesses, doing the work that can take us beyond the damage done by all the self-proclaimed “plus ones” who noisily and self-interestedly surround us.
We have to not only encourage them, but also be them. For those working in algorithmically oriented businesses, the writing is on the wall, penned by AI and approved by investors. For those sheltering in the mysteries, nobody is coming to rescue you.
If we fail to make the necessary compromises to bridge the gap between mystery and algorithm, we will pay the price.
Articles and Posts I have appreciated this week
Craftsmanship on Tinkering
Margaret Heffernan - Ideas are Cheap
Parker Palmer on the Contemplation of Catastrophe
Ted Gioia on Indie Culture. I think this matters
Hadfield's idea of a zero reminds me of the empty cup in Zen stories. You need to have a functioning cup (not a minus one) but not to fill it with certainty (not a one). That seems like a good heuristic to me.
Love this Richard - since you introduced me to the understanding of heuristics I've had a field day! Hope all well.