The theory behind hierarchies, markets and market makers

Vladimir Lenin (1870-1924) famously said that the economic system in Russia after the revolution would be run as one big factory. Many economists at the time said that this was impossible. Yet there were already big factories in the West then, and there still are, so why not? Is there a limit to the size of a factory that cannot be surpassed, or is it because the factory logic cannot be used outside a real factory?

The typical hierarchical, factory, form of an organization is meant to simplify communication, accountability and the coordination of tasks. In theory an employee needs only one connection, to the boss. This is far easier than communicating with all and trying to coordinate actions with everyone. And what about accountability? The worker is accountable only to her manager. That manager reports to her manager on the next level up, and the chain goes further, leading in the end to – Lenin.

During the centuries since the publication of “The Wealth of Nations” in 1776 by Adam Smith (1723-1790), the principal theme of most economists has been that government regulation or centralized planning were not necessary, or even welcome, in order to make an economic system function well. The coordination would be the result of pricing mechanisms in markets. Lenin and the communists were advised to move to a market economy. The parties in this system follow their own self-interest and are “governed”, when it comes to the actual choices they make, by the system of prices and information they possess. This is the polar opposite of centralized planning. Adam Smith was a proponent of extreme decentralization and something that did not really have a name at the time – democracy.

Ronald Coase (1910-2013) was one of the first economists who started to question mainstream thinking in economics. If a system of prices and competition could perform all the coordination that is necessary, why did we have centralized planning, not only in the now bygone communist countries, but also in well-functioning and successful firms? Why did we need management, whose function was to coordinate? Why didn’t we rely on markets?

Ronald Coase set out to bring these two different views together. It is almost impossible now to fathom that he found the answer as early as during the summer of 1932, at the age of 22. He realized that there were costs involved in using the pricing mechanism. The needs and offerings have to find one another. The prices have to be discovered. Negotiations need to be undertaken. Contracts have to be made. There may be disputes that later have to be settled. Adam Smith did not see this. These costs were not part of “the invisible hand” equation. Ronald Coase called these costs transaction costs.

The first revolutionary argument was that a firm would emerge, exist and continue to exist successfully if it performed its planning, coordination and management functions at a lower cost than would be incurred by means of market transactions, and also at a lower cost than would apply if the same things could be performed by another firm. This is where competition keeps firms internally efficient and where non-competition in the public sector creates complex, non-efficient governance models and units that are too big.

The second revolutionary argument was that a well-functioning economic system needs both markets and planning. This depends on the size of the organization and the level of the market side transaction costs. Increasing the size increases the internal transaction costs. Running a sizable organization is difficult and running an even bigger organization is even more difficult.

Managerial overheads increase as the organization grows. Management, communication and coordination are all transaction costs. Every sales call, every offer, every agreement and every meeting also consumes limited resources and increases transaction costs. As the corporation grows, all its energy finally goes into maintaining the corporation and does not benefit external stakeholders.

Whenever the transaction costs inside the organization reach the level of the transaction costs in the (outside) markets, markets outperform firms and outperform central planning/coordination in general. This was the main theoretical argument against Lenin. The same thing is clearly still evident today in companies like GM or organizations like large health care units. Communist countries learned their lesson, but we still haven’t.

The existence of high transaction costs outside firms led to the emergence of the firm as we know it, and management as we know it. A large part of corporate economic activity is designed to accomplish what high market transaction costs prevented earlier.

If the (transaction) costs of exchanging value in the society at large go down drastically as is happening today, the form and logic of economic and organizational entities necessarily need to change! The firm, as we have known it, becomes the more expensive alternative. Accordingly, a very different kind of management is needed when coordination can be performed without intermediaries with the help of new technologies.

Today, we stand on the threshold of an economy where the familiar economic entities are becoming increasingly irrelevant. The Internet, and new Internet based firms, rather than the traditional organizations, are becoming the most efficient means to create and exchange value.

For most of the developed world, corporate hierarchies, as much as markets, make up the dominant economic pattern. The Internet is nothing less than an extinction-level event for the traditional firm as we have known it for the past 100 years. The Internet makes it possible to create totally new forms of economic entities.

The Internet changes our views of markets and hierarchies in ways that Adam Smith or Vladimir Lenin could never have imagined. But Ronald Coase did much to explain the theory behind Uber, Airbnb and other new market makers as early as 1932.

Thank you @cs

Communication and diversity secure system-wide performance

In our competitive view of the world, we often think that the most capable are those who are the most competitive, and accordingly that competition creates and secures efficiency. It may be that high performance is incorrectly attributed to competition and is more a result of diversity, self-organizing communication and non-competitive processes of collaboration.

Competitive processes lead to the handicapping of the higher-level system that these processes are part of. This is because competitive selection leads to exclusion: something is left outside. Leaving something out always means a reduction of diversity. The resulting less diverse system is efficient in the short term, but always at the expense of flexibility. Agility and complex problem solving require diversity. Everything goes fine if nothing changes and if there are only easy problems to take care of!

Self-organizing, non-competitive processes are about interdependent individuals and groups solving problems in a shared context. Interaction creates capability beyond what could ever be predicted just by looking at the performance of the individuals involved. The higher performance and robustness are emergent properties of interaction. They are not attributable to the parts of the system. Social networks provide problem-solving capability that results directly from the amount of communication and level of diversity of communication.

Most organizations would soon fail if all their employees thought alike or had little or no contact. There are two new challenges. The first is to understand the need for networking with views and values that are different. The second challenge is even bigger because of the mainstream reductionist thinking: our assumption has been that by understanding the parts in detail, we understand the whole. This is simply not possible! What happens in interaction between the parts is more important than the parts. The whole is the emergent pattern of that interaction, not the sum of the parts.

Diversity here means the degree of unique information in the network. If all contribute the same information, then diversity is low. If each agent contributes relevant, unique information that is not shared by others, then the diversity measure is high.

Networks with a wide spectrum of filtering information are resilient to noise. This facilitating effect of diversity is critical when dealing with difficult problems where false information can lead to expensive consequences. Higher system performance and robustness occur through the simple collaborative combination of the different experiences of individuals, even though each individual takes part in communicative interaction from their own limited perspective.

The importance of self-organization and diversity is unfortunately still greatly underestimated today, particularly in centralized, monoculture systems – like firms. One of the great societal promises of social media is that interaction in wide-area networks, with enough diversity, can solve problems beyond the awareness of the individuals involved.

.

Background: On serendipity

Thank you Stuart Kauffman, Sari Baldauf and Norman Johnson

The new world between chance and choice

Nonlinear dynamics are concerned with messy systems. Examples for these systems are the human brain, the weather and the evolution of life itself. There is not a single science of non-linearity, but there are different streams of research such as chaos theory or the theory of complex adaptive systems. The latter strand takes up an agent- and rules of interaction-based approach to modeling what is going on. The first explains the behavior of systems that can be modeled by complex mathematical equations where the output of one calculation is taken as the input for the next.

Chaos theory explains how the parameters in the equations cause patterns in time. These patterns can also be described as phase diagrams. Then they are called attractors. A parameter might be the flow of information or the amount of energy in the system. At low rates the system moves forward displaying a repetitive, stuck behavior. This pattern, “spatially”, is called a point attractor. At higher rates the pattern changes. At very high rates of, for example information flow, the system displays a totally random behavior. The pattern is highly unstable. However, there is a level between repetition and stability or randomness and instability. This level is called the edge of chaos. The pattern in time is called a strange attractor when it is described as a phase diagram.

The strange thing with a strange attractor is that the ongoing movement is never the same but always recognizable. The pattern is paradoxically stable and unstable, predictable and unpredictable at the same time. Chaos theory describes a paradoxical dynamic that is not a synthesis of order and disorder. It is about orderly disorder or disorderly order. The very meaning of these words is new.

The weather is often used as an example of a system that displays this pattern. The overall weather patterns can be (almost) predicted over short periods of time. Over longer periods, the behavior cannot be predicted. The long-term behavior of a system like this is determined as much by the smallest changes in the smallest of parts of the system, as it is determined by the laws governing it. The conclusion is very clear. Predictability, if it is possible, is always short-term. Longer-term predictions would only be possible if absolutely all the variables in the system could be measured with absolutely infinite accuracy.

As it is impossible to know all the variables and as it is totally impossible to measure the variables with the infinite accuracy needed, the smallest overlooked variable or the most minute change can escalate up by non-linear iterations into a major transformative change in the later life of the system. Another conclusion is that from a chaos theory perspective, movement towards equilibrium is always movement towards death. If a system is healthy, successful and alive, it is “at the edge of chaos” where the long-term cannot be seen.

The scientists at the Santa Fe Institute developed another strand of research: the complex adaptive systems (CAS) approach. A CAS consists of a large number of agents. Each agent behaves according to its own intentions and rules for local interaction. Local here means that no agent can interact with the whole population of agents at the same time. No individual agent can determine or cause the pattern of behavior that the system as a whole displays. These adaptive systems display the same dynamics as the chaos theorists found: stable equilibrium at one end of the spectrum, random chaos at the other, and in-between the newly found complex dynamic of simultaneous stability and instability or predictability and unpredictability, paradoxically at the same time.

The conclusions are hugely important for us. Firstly, novelty always emerges in a radically unpredictable way. Secondly, the patterns of healthy behavior are not just caused by competitive selection or independent choices made by independent agents. Instead, what is happening, happens in interaction, not by chance or by choice, but as a result of the competitive/collaborative interaction itself.

The Internet changes the patterns of connectivity, transforms our understanding what “local” is, and makes possible new enriching variety in interaction. The changed dynamics we experience every day through social media may have the very characteristics of the edge of chaos.

The sciences of complexity change our perspective and thinking. Perhaps, as a result we should, especially in management, focus more attention on what we are doing than what we should be doing. The important question to answer is not what should happen in the future, what the goals are, but what is happening now that creates the continuously developing pattern that is our life.

.

Thank you Stu Kauffman and W Brian Arthur. Special thanks to Ralph Stacey and Doug Griffin.

Complexity and management.

Learning from Caravaggio

We have a curious habit of thinking that what we are accustomed to is the way things should be. We are inclined to accept conventional forms as facts, and as meaningful reference points, when facing novelty.

Artists are often the people who want to see the world afresh. It is not easy to get rid of preconceived ideas, but the artist who best discards accepted notions and prejudices often produces the most remarkable works of art. However, a painting that represents a traditional subject in an unexpected way is often condemned. Normally there is no good reason, apart from the work of art just not following tradition.

There have been a few times in history that a great artist looked carefully at what was visible to everybody, but in fact saw things very different from the way others saw them.

Very few artists have caused as much shock and outrage as Caravaggio (1571 – 1610). His full name was Michelangelo Merisi da Caravaggio A famous story is told about the commissioning of a painting of Saint Matthew for the altar of a church in Rome. This happened around 1600. The saint was going to be represented writing the gospel that was later going to be named after him. As gospels were the word of God, there was also going to be an angel in the picture, giving guidance to Matthew. Caravaggio read the old texts very carefully and tried to figure out what it must have been like when an old workingman suddenly had to sit down and do something that he had never done, or never dreamed of doing: writing, and writing a book, and being guided by an angel!

Caravaggio painted an elderly man, seemingly from a poor background. A workingman with a bald head and muscular legs, writing awkwardly! Beside him was a strange child with beautiful white wings. A young angel was guiding Matthew’s hand just like a teacher would do to a young pupil doing something for the very first time. Caravaggio’s painting was a fantastic portrait of a human being in a very, very special situation. But it was more than that, it was also a completely new way of expressing a familiar topic. The painting was to be placed on the altar of a church in Rome.  This never happened because Caravaggio’s work created a huge scandal. The painting was not accepted because it was claimed that it showed a lack of respect for the topic! What Caravaggio had done, for the first time in history, was to create a new expression and give a human face to something that was previously highly formalized. Caravaggio may have been the first human-centric painter.

People thought that Caravaggio was out to shock them. They also thought that he had no understanding for beauty and tradition. In fact, Caravaggio may have been the first artist in known history who was labeled both a rebel and a naturalist at the same time. Anyway, he was the first painter of the “ugly” true reality. The curious thing is that seeing Caravaggio’s paintings today; one still encounters the same boldness, energy and power that must have shocked people over four hundred years ago.

Caravaggio’s art is still very much alive today, although the altar painting described here was never accepted for the altar and was eventually destroyed in Germany during the Second World War. It has been missing since.

The question remains whether the human-centric approaches today meet the same kind of opposition as Caravaggio encountered. This time, it is not about challenging the conventions carried forward by the church, or representations of what is considered holy. Human-centric thinking today questions the conventions carried forward by organization-centric thinking and the idealized representations of leadership/management. But the pattern is the same. Not much has changed, when it comes to the importance of Caravaggio! Being human-centric may be today as difficult as it was in 1602.

The challenge is the same: to look carefully at what is visible to everybody, but in fact see things very different from the way others see them.

Thank you @cshirky