Reducing friction in structures
Organizational structures have existed since one caveman told the other to go hunting while he fished. It was a natural distribution of work where resources needed to be organized to produce greater output. Then a third dude strolled along and the first guy asked him if he could make fire. Turns out he could, and he could hunt too, but the thing he did best was to make fire so he was called the fire guy. And so it was that human beings learned to collaborate and coordinate in a flexible manner to achieve something they couldn’t achieve on their own. At its essence, this “organism“ of three people isn’t terribly different than what we today see in complex organizations. Replace skill with fire/hunting/fishing, problem with starvation, and opportunity with a chance to eat, and the basic archetype hasn’t changed.
The key difference is that where structures were forced to change as new skills, problems, and opportunities developed, the modern organization’s default approach is to account for new skills, problems, and opportunities using existing structures. This results in a square-peg-round-hole situation as the current structures were designed to account for a different set of skills, problems, and opportunities.
It is expensive to change structures and given the aversion to change that people generally have, it’s not a surprise that organization structures rarely change significantly, and if they do, the results aren’t great.
The room for improvement here is vast, and this post isn’t going to pretend to try and solve any problems — we’ll leave that to the expensive consultants. What is far more interesting to me is how the structures exacerbate more than alleviate the very problems they were designed to solve. I’ve picked three to talk about: influence-based networks, wasted talent, and communication quality.
The influence-based organizational chart
Many structures are put in place so that communication channels are clear and information is shared seamlessly. This formal structure of communication is a starting point, but over time, informal communication structures develop and people figure out how to get things done. Whether it’s chatting up the Linux admin so he can do your firewall change first, to squeezing in an extra feature because you‘re friends with the product owner. This is normal and most collections of people develop these sort of relationships. If something takes 2 days to do formally and 5 minutes to do informally, all other things being equal, why would you not take the 5 minute option? Formal structures, that’s why.
Over time and under the covers, the influence-based organizational chart replaces the hierarchical organizational chart, and if the two are not aligned, it makes for an imperfect marketplace for getting things done. In this imperfect marketplace, the original structure is a hindrance and causes loss of value to the organization.
The seven wastes of software development are fairly well-known and rooted in lean principles. There is an eighth one though, and it’s human creativity. Arguably, it is the most expensive one of them all because we go through great lengths to hire people, only to put boxes around them to the point where their true value to the organization is never realized. This rings especially true in the knowledge-based economy where mechanical tasks are rare, and the rigid structures which support them are not needed. This is the classic glass ceiling effect which destroys potential and erodes motivation.
People have good intentions when they place other people under a particular structure, and implicitly trust the structure to promote or bubble up talent. In theory, this is great. In practice, it is very difficult for static structures to adapt to dynamic human beings whose tastes and talents change with time.
It is hard to believe that the pockets in which people were deployed a year ago remains the best way of doing business, simply because it doesn’t account for an individual’s growth (or stagnation). The restrictions in mobility that organizational structures impose prevent the seamless transfer of skills from one place to another, implying that the people working on activities are not always the best ones to be working on them as there is a good chance there is someone more suited in another pocket who could be better, but hasn’t had the chance to move. This leads to organizational value loss, or if you’re of the economics mindset, a dead-weight loss.
Structures are often created to make communication easier in the face of growth. The PMP’s famed n (n — 1) /2 communication channels for n people is a scary thought, so we reduce n by creating hierarchies where the “managers” of each team communicate on behalf of the team. Again, a noble idea grounded in reality and practicality. Now complement this with the broken telephone concept, and you can imagine the quality of communication that ends up getting propagated throughout the organization.
My personal favorite is the “prep meeting” where we have a meeting to ensure what we’re going to say when speaking with someone up the hierarchy. This is now considered a necessarily evil because communication has now been reduced to PowerPoint decks (or unless you’re a sick person, Prezi). My view is that there is a risk in communicating upward because face-time with leadership is rare and you want to make sure you get the message right the first time, because communication has become less of a dialogue and far more formal with weasel words dominating the material.
There are explicit time delays (e.g., biweekly checkpoints) and implicit bottlenecks (e.g., only managers communicating particular messages) which inhibit the overall communication in the structure, even though the purpose of the structure was to improve quality and frequency of communication. Instead, we have surrendered quality and reduced frequency.
OK, so what do we do? We can’t just do away with age-old structures overnight, and even if we did, what would replace them? These are tough questions and very context-sensitive. In a software development context, we’ve played around with extremes — ranging from waterfall to programmer anarchy, with results all over the map. These are difficult problems to solve because growth asks questions of us which we aren’t always equipped to answer. We don’t know what works and what doesn’t in our context until we have played around with it.
And I suppose that is the underlying point here, once we get feedback from our context (e.g., team, department, board of directors), how often are we willing to change either the structure itself or the content of the edges connecting the nodes in the structure so that structures help and not inhibit. The frequency of this change depends on the cost of change, which poses deeper questions discussed in change management literature. Mostly, how can we reduce the cost of change so we change often? That is for you to decide in your own context.
Moving on to the practical part of this post, three things that I’ve done in large software development teams that I’ve lead to reduce this cost of change are:
Do not have a formal structure until you need one, and make that structure as leaderless as possible, or more correctly, use shared leadership — difficult, but possible. If there’s no predefined structure, then change is inevitable, and it’ll be the type the people feel most comfortable with, not what is imposed upon them.
Encourage direct horizontal and vertical communication, i.e., just because there is an implicit or explicit leader/manager/whatever, that should not prevent people from communicating with anyone else. If this message is hammered home to everyone, it can have liberating effects on team cohesion and willingness to listen to each other. Apply with caution and always consider your own context.
Manage the outcome, not the process. Let people decide on how they want to work, and then manage the outcomes, not the process. You’ll find that given the right set of guiding principles (or goals, or vision), the structures will be flatter and more fluid than what they’d be if you had designed them on paper. This self-organization reduces friction when the inevitable change comes.
These are difficult problems to solve, and the only thing I know for sure is that solutions need to be context-specific and principles-based. There are no boilerplate solutions here.