Friday, 10 October 2014

COMPLEXITY TIME BOMB: When systems get out of control


 by Dirk Helbing

                                                                                                                                                                                Photo: RenateWernli

This is second in  a  series of blog posts that form chapters of my forthcoming book Digital Society. Last week's chapter was titled:  GENIE OUT OF THE BOTTLE: The digital revolution on its way.

Financial crises, terrorism, conflict, crime: it turns out, the conventional ‘medicines’ to tackle global problems are often inefficient or even counter-productive. The reason for this is surprisingly simple: we approach these problems with an outdated understanding of our world. While the world might still look similar to how it has looked for a long time, I will argue that it has, in fact, inconspicuously but fundamentally changed over time.

We are used to the idea that societies must be protected from external threats such as earthquakes, volcanic eruptions, hurricanes, and military attacks by enemies. However, we are increasingly threatened by another kind of problems: those that come from within the system, such as financial instabilities, economic crises, social and political unrest, organized crime and cybercrime, environmental change, and spreading diseases. These threats have become some of our greatest worries. According to the World Economic Forum's Risk Map, the largest risks today are of a socio-economic nature such as inequality or governance failure. These global 21st century problems cannot be solved with 20th century wisdom, because they are of a different scale and result from a new level of complexity in today's socio-economic systems. We must therefore better understand what complex systems are, and what are their properties. To this end, I will discuss the main reasons why things go wrong: unstable dynamics, cascading failures in networks, and systemic interdependencies. I will illustrate these problems by examples such as traffic jams, crowd disasters, blackouts, financial crises, crime, wars, and revolutions.

Phantom traffic jams


Complex systems include phenomena ranging from turbulent flows and the global weather system to decision-making, opinion formation in groups, financial and economic markets, and the evolution and spread of languages. But we must take care to distinguish complex systems from complicated ones. A car is complicated: it consists of thousands of parts, yet is easy to control (when it works properly). Traffic flow, on the other hand, which depends on the interactions of many cars, is a complex dynamical system, which produces counter-intuitive, individually uncontrollable behaviors such as "phantom traffic jams" that seem to have no cause. While many traffic jams do occur for a specific, identifiable reason, such as an accident or a building site, everyone has also encountered situations where a vehicle queue appeared "out of nothing" – and where there is no visible cause - see  visualisation

To explore the true reasons for these phantom traffic jams, Yuki Sugiyama from the Nagoya University in Japan and his colleagues carried out an experiment, in which they asked many people to drive their cars around a circular track - see visualisation  The task sounds simple, and indeed all vehicles moved smoothly for some time. But then a random perturbation in the traffic flow, an unexpected slow-down of a car, triggered the appearance of “stop-and-go” traffic – a traffic jam that travelled backwards around the track, against the driving direction.

While we often blame others for poor driving skills to explain such "phantom traffic jams," studies in complexity science have shown that they rather emerge as a collective phenomenon unavoidably resulting from the interactions between vehicles. A detailed analysis shows that, if the density of cars exceeds a certain "critical" threshold – that is, if their average separation is smaller than a certain value – then the smallest perturbation in the speed of any car will be amplified to cause a breakdown of the entire flow. Because drivers need some time to respond to such a disturbance, the next driver in line will have to brake harder to avoid an accident. Then the following driver will have to break even harder, and so on. This chain reaction amplifies the small initial perturbation and eventually produces the jam – which of course every individual would prefer to avoid.

Recessions - traffic jams in the world economy?


Economic supply chains might exhibit a similar kind of behavior. As known from John Sterman's "beer distribution game," supply chains are also hard to control. Even experienced managers will often end up ordering too much beer, or will run out of it. This is a situation that is as difficult to avoid as stop-and-go traffic. In fact, our scientific work suggests that economic recessions may be regarded as a kind of traffic jam in the global supply network (see figure below). This is actually somewhat heartening news, since it implies that, just as with traffic flow, engineered solutions may exist that can mitigate economic recessions, provided that we have access to real-time data on the world's supplies and materials flows. Such solutions will be discussed later in the chapter on Socio-Inspired Technologies.

Instability and self-organization in strongly interacting systems


A shocking example for systemic instabilities discussed later is the occurrence of crowd disasters. Here, even when everyone is peacefully minded and tries to avoid harming others, many people might die. What do all these examples tell us? Our experience will often not inform us well, and our intuition may fail, since complex dynamical systems tend to behave in unexpected or even counter-intuitive ways. Such systems are typically made up from many interacting components, which respond to the behavior of other system components. As a consequence of these interactions, complex dynamical systems tend to self-organize, i.e. to develop a collective behavior that is different from what the components would do in separation. Then, the components’ individual properties are often no longer characteristic for the system. "Chaotic" or "turbulent" dynamics are possible outcomes, but complex systems can show many other phenomena.

When self-organization occurs, one often speaks of emergent phenomena that are characterized by new system properties, which cannot be understood from the properties of the single components. For example, the facts that water feels wet, extinguishes fires, and freezes at a particular temperature are properties, which cannot be understood from the properties of single water molecules.

As a consequence of the above, we have to shift our attention from the components of our world to their interactions. In other words, we need a change from a component-oriented to an interaction-oriented, systemic view, which is at the heart of complexity science. I claim that this change in perspective, once it becomes common wisdom, will be of similar importance as the transition from the geocentric to the heliocentric worldview. The related paradigm shift has fundamental implications for the way in which complex techno-socio-economic systems must be managed and, hence, also for politics and our economy. Focusing on the interactions in a system and the multi-level emergent dynamics resulting from them, opens up fundamentally new solutions to long-standing problems.

Instability is one possible behavior of complex dynamical systems, which results when the characteristic system parameters cross certain critical thresholds. If a system behaves unstable, i.e. perturbations are amplified, a random, small deviation from the normal system state may trigger a domino effect that cannot be stopped, even if people have the best intentions to do so and have enough information, good technology, and proper training. In such situations of systemic instability, the system will inevitably get out of control sooner or later, no matter how hard we try to avoid this. As a consequence, we need to know the conditions under which systems will behave in an unstable way, in order to avoid such conditions. In many cases, too strong interactions are a recipe for disaster or other undesirable outcomes.

Group dynamics and mass psychology may be seen as typical examples of collective dynamics. People have often wondered what makes a crowd turn "mad", violent, or cruel. After the London riots in the year 2011, people asked how it was possible that teachers and daughters of millionaires – people one would not expect to be criminals – were participating in the lootings. Did they become criminal minds when their demonstrations against police violence suddenly turned into riots? Possibly, but not necessarily so. In the above traffic flow example, people wanted to do one thing: drive continuously at reasonably high speed, but a phantom traffic jam occurred instead. We found that, while individual cars are well controllable, the traffic flow – a result of the interactions of many cars – is often not. The take home message may be formulated as follows: complex systems cannot be steered like a car. Even if everyone has the latest technology, is well-informed and well-trained, and has the best intentions, an unstable complex system will sooner or later get out of control.

Therefore, while our intuition works well for weakly coupled systems, in which the system properties can be understood as sum of the component properties, complex dynamical systems behave often in counter-intuitive, hardly predictable ways. Frequently, the collective, macro-level outcome in a complex system can't be understood from and controlled by the system components. (Such system components might also be individuals or companies, for example.)

Beware of strongly coupled systems


Thus, what tends to be different in strongly coupled systems as compared to weakly interacting ones? First, the dynamics of strongly connected systems with positive feedbacks is often faster. Second, self-organization and strong correlations tend to dominate the dynamics of the system. Third, the system behavior is often counter-intuitive – unwanted feedback or side effects are common. Conventional wisdom tends to fail. In particular, extreme events occur more often than expected, and they may impact the entire system. Furthermore, the system behavior can be hard to predict, and planning for the future may not be useful. Opportunities for external control are also typically quite limited, as the system-immanent interactions tend to dominate. Finally, the loss of predictability and control may lead to an erosion of trust in private and public institutions, which in turn can create social, political, or economic instabilities.

In spite of all this, many people still have a component-oriented and individual-centric view, which can be quite misleading. We praise heroes when things run well and search for scapegoats when something goes wrong. But the discussion above has shown how difficult it is for individuals to control the outcome of a complex dynamical system, if its components' interactions are strong. This fact may be illustrated by the example of politics. Why do politicians, besides managers, have among the worst reputations among all professions? This is probably because we vote them to make politics according to the positions they publicly voice, but then we often find them doing something else. This, again, is a consequence of the fact that politicians are exposed to many strong interactions due to lobbyists and pressure groups with various points of view. Each one is trying to push the politician in a different direction. In many cases, this will force the politician to take a decision that is not compatible with his or her own points of view, which is hard for the voters to accept. Managers of companies find themselves in similar situations. But not only they: think of the decision-dynamics in many families. If it were easy to control, we would not see so many divorces...

Crime is another good example for unwanted outcomes of complex dynamics, even though a controversial one. We must ask ourselves: Are we interested in sustaining social order, or are we interested in filling prisons? If we decide for the first option, we must confront ourselves with the question: Should we really see all crime as deeds of criminal minds, as we often do? Or should we pay more attention to the circumstances that happen to cause crime? In cases, where individuals plan crimes such as the theft of a famous diamond, the conventional picture of crime is certainly appropriate. But do these cases give a representative picture?

Classically, it is assumed that crimes are committed, if the expected advantage is larger than the punishment, multiplied with the probability of being convicted. Therefore, raising punishments and discovery rates should theoretically eliminate all crime. Such punishment would make crime a lossful experience and, therefore, "unattractive." However, empirical evidence questions this simple picture. On the one hand, people usually don't pick pockets, even though they could often get away without a punishment. On the other hand, deterrence strategies are surprisingly ineffective in most countries, and high crime rates are often recurrent. For example, even though the USA have 10 times more prisoners than most European countries, rates of various crimes, including homicides, are still much higher. So, what is wrong with our common understanding of crime?

Surprisingly, many crimes, including murders, are committed by average people, not by people with criminal careers. A closer inspection shows that many crimes result from situations, over which the involved individuals lose their control. Frequently, group dynamics plays an important role, and many scientific studies indicate that the socio-economic context is a strong determining factor of crime. Therefore, in order to counter crime, it might be more effective to change these socio-economic conditions rather than sending more people to jail. I am saying this also with an eye on the price we have to pay for this: A single prisoner costs more than the salary of a postdoctoral researcher with a PhD degree, some even more than a professor!

Cascade effects in complex networks


Making things worse, complex systems may show further problems besides dynamic instabilities based on amplification effects. Thanks to globalization and technological progress, we have now a global exchange of people, goods, money, and information. Worldwide trade, air traffic, the Internet, mobile phones, and social media have made everything much more comfortable – and connected. This has created many new opportunities, but everything now depends on a lot more things. What are the implications of this increased interdependency? Today, a single tweet can send stock markets to hell. A youtube movie can trigger a riot that kills dozens of people. Our decisions can have impacts on the other side of the globe more easily than ever – and sometimes unintentionally so. For example, today’s quick spreading of emerging epidemics is largely a result of global air traffic, and can seriously affect global health, social welfare, and economic systems.

By networking our world, have we inadvertently built highways for disaster spreading? In 2011 alone, three major cascading failures occurred, which are changing the face of the world and the global balance of power: The financial crisis, the Arab spring and the combined earthquake, tsunami and nuclear disaster in Japan. In the following, I will discuss some examples of cascade effects in more detail.

Large-scale power blackouts


On November 4, 2006, a power line was temporarily turned off in Ems, Germany, to facilitate the transfer of a Norwegian ship. Within minutes, this caused a blackout in many regions all over Europe – from Germany to Portugal! Nobody expected this. Before the line was switched off, of course, a computer simulation was performed to verify that the power grid would still operate well. But the scenario analysis did not check for the coincidence of a spontaneous failure of another line. In the end, a local overload of the grid caused emergency switch-offs in the neighborhood, creating a cascade effect with pretty astonishing outcomes: some blackouts occurred in regions thousands of kilometers away, while other areas in the neighborhood were not affected at all. Is it possible to understand this strange behavior?

Indeed, a computer-based simulation study of the European power grid recently managed to reproduce such effects. It demonstrated that the failure of a few network nodes in Spain could create a surprising blackout in Eastern Europe, several thousand kilometers away, while the electricity network in Spain would still work - see visualisation

Furthermore, increasing the capacities of certain parts of the power grid would unexpectedly make things worse. The cascading failure would be even bigger! Therefore, weak elements in the system have an important function: they act as circuit breakers, thereby interrupting the failure cascade. This is an important fact to remember.

Bankruptcy cascades


The sudden financial meltdown in 2008 is another example, which hit many companies and people by surprise. In a presidential address to the American Economic Association in 2003, Robert Lucas said:
"[The] central problem of depression-prevention has been solved."
Similarly, Ben Barnenke, as chairman of the Federal Reserve Board, long believed that the economy was well understood, and doing well. In September 2007, Ric Mishkin, a professor at Columbia Business School and then a member of the Board of Governors of the US Federal Reserve System, made a statement reflecting widespread beliefs at this time:
"Fortunately, the overall financial system appears to be in good health, and the U.S. banking system is well positioned to withstand stressful market conditions."

As we all know, things came very different. A banking crisis occurred only shortly later. It started locally, with the bursting of a real estate bubble in the West of the USA. Because of this locality, most people thought this problem was easy to contain. But the mortgage crises had spill-over effects to the stock markets, where certain financial derivatives could not be sold anymore (now called "toxic assets"). Eventually, more than 400 banks all over the United States went bankrupt. How could this happen? The video presents an impressive visualisation of the bankruptcies of banks in the USA after Lehman Brothers collapsed. Apparently, one bank's default triggered further ones, and these triggered even more. In the end, hundreds of billion dollars were lost.

The above video reminds of another video which I often use to illustrate cascade effects: It shows an experiment with many table tennis balls placed on top of mouse traps. The experiment demonstrates impressively that a single local perturbation can mess up the entire system. It illustrates chain reactions, which are the basis of atomic bombs or of nuclear fission reactors. As we know, such cascade effects are technologically controllable in principle, if we stay below the critical interaction strength (sometimes called the "critical mass"). Nevertheless, these processes can sometimes get out of control, mostly in unexpected ways. The nuclear disasters in Chernobyl or in Fukushima are well-known examples for this. So, we must be extremely careful with systems showing cascade effects.

The financial crisis


As we know, the above-mentioned cascading failure of banks was just the beginning of an even bigger crisis. It subsequently caused an economic crisis and a public spending crisis in major areas of the world. Eventually, the events even threatened the stability of the Euro currency and the European Union. The crisis brought several countries (including Greece, Ireland, Portugal, Spain, Italy and the US) at the verge of bankruptcy. As a consequence, many countries have seen historical heights in unemployment rates. In some countries, more than 50 percent of young people do not have a job. In many regions, this has caused social unrests, political extremism and increased crime and violence. Unfortunately, it seems that the cascade effect has not been stopped yet. There is a long way to go until we fully recover from the financial crisis and from the public and private debts accumulated in the past years. If we can't overcome this problem soon, it has even the potential to endanger peace, democratic principles and cultural values, as I pointed out in a letter to George Soros in 2010. Looking at the situation in Ukraine, we are perhaps seeing this scenario already.

While all of this is now plausible from hindsight, the lack of advance understanding by conventional wisdom becomes clear by the following quote from November 2010, going back to the former president of the European Central Bank, Jean-Claude Trichet:

"When the crisis came, the serious limitations of existing economic and financial models immediately became apparent. Arbitrage broke down in many market segments, as markets froze and market participants were gripped by panic. Macro models failed to predict the crisis and seemed incapable of explaining what was happening to the economy in a convincing manner. As a policy-maker during the crisis, I found the available models of limited help. In fact, I would go further: in the face of the crisis, we felt abandoned by conventional tools." Similarly, Ben Bernanke summarized in May 2010: “The brief market plunge was just an example of how complex and chaotic, in a formal sense, these systems have become… What happened in the stock market is just a little example of how things can cascade, or how technology can interact with market panic.”

Leading scientists as well had problems making sense of the crisis. In a letter dated 22 July 2009 to the Queen of England, the British Academy came to the conclusion:

"When Your Majesty visited the London School of Economics last November, you quite rightly asked: why had nobody noticed that the credit crunch was on its way? ... So where was the problem? Everyone seemed to be doing their own job properly on its own merit. And according to standard measures of success, they were often doing it well. The failure was to see how collectively this added up to a series of interconnected imbalances over which no single authority had jurisdiction. ... Individual risks may rightly have been viewed as small, but the risk to the system as a whole was vast. ... So in summary ... the failure to foresee the timing, extent and severity of the crisis … was principally the failure of the collective imagination of many bright people to understand the risks to the systems as a whole."

Thus, nobody was responsible for the financial mess? I don't want to judge, but we should remember that it's often not possible to point the finger at the exact person who caused a phantom traffic jam. Therefore, given that these are collectively produced outcomes, do we have to accept collective responsibility for them? And how to enumerate everyone's share of responsibility? This is certainly an important question worth thinking about.

It is also interesting to ask, whether complexity science could have forecasted the financial crisis? In fact, before the crash, I followed the stock markets pretty closely, as I noticed strong price fluctuations, which I interpreted as "critical fluctuations," i.e. an advance warning signal of an impending financial crash. Therefore, I sold my stocks in the business launch of an airport in 2007, while waiting for the departure of my airplane. In spring 2008, about half a year before the collapse of Lehman brothers, I wrote an article together with Markus Christen and James Breiding, taking a complexity science perspective on the financial system. We came to the conclusion that the financial system was in a process of destabilization. Pretty much as Andrew Haldane, Chief Economist and Executive Director at the Bank of England, formulated it later, we believed that the increased level of complexity in the financial system was a major problem. It made the financial system more vulnerable to cascade effects than most experts thought. In spring 2008, we were so worried about this that we felt we had to alert the public, but none of the newspapers we contacted was ready to publish our essay at that time. "Too complicated for our readers" was the response, while we replied "if you cannot make this understandable to your readers, then there is nothing that can prevent the financial crisis." And so the financial crisis came! Six month after the crisis, a manager of McKinsey in the United Kingdom commented on our analysis that it was the best he had ever seen.

But there were much more prominent people who saw the financial crisis coming. For example, legendary investor Warren Buffet warned of mega-catastrophic risks created by large-scale investments into financial derivatives. Back in 2002 he wrote:

"Many people argue that derivatives reduce systemic problems, in that participants who can't bear certain risks are able to transfer them to stronger hands. These people believe that derivatives act to stabilize the economy, facilitate trade, and eliminate bumps for individual participants. On a micro level, what they say is often true. I believe, however, that the macro picture is dangerous and getting more so. ... The derivatives genie is now well out of the bottle, and these instruments will almost certainly multiply in variety and number until some event makes their toxicity clear. Central banks and governments have so far found no effective way to control, or even monitor, the risks posed by these contracts. In my view, derivatives are financial weapons of mass destruction, carrying dangers that, while now latent, are potentially lethal."
As we know, it still took five years until the "investment time bomb" exploded, causing losses of trillions of dollars to our economy.

Fundamental uncertainty


In liquid financial markets and many other hardly predictable systems such as the weather, we can still determine the probability of events, at least approximately. Thus, we make a probabilistic forecast similar to: "there is a 5 percent chance to lose more than half of my money when selling my stocks in 6 months, but a 70 percent chance that I will make a good profit, etc." It is then possible to determine the expected loss (or gain) implied by the likely actions and events. For this purpose, the damage or gain of each possible event is multiplied with its probability, and the numbers are added up to give the expected damage or gain. In principle, one could do this for all actions we might take, in order to determine the one that minimizes the damage or maximizes the gain. The only problem involved in this exercise seems to be the practical determination of the probabilities and of the likely damages or gains involved. With the increasing availability of data, this problem might, in fact, be attacked, but it will remain difficult or impossible to determine the probabilities of "extreme events," as the empirical basis for rare events is too small.

It turns out, however, that there are problems where the expected damage in large (global) systems cannot be determined at all for principal reasons. Such "fundamental" or "radical" uncertainty can occur in case of cascade effects, where one failure is likely to trigger other failures, and where the damage related to subsequent events times their likelihood is increasing. In such cases, the sum of losses may be unbounded, in principle, i.e. it may not be possible anymore to enumerate the expected loss. In practice, this means that the actual damage can be small, big, or practically unbounded, where the latter might lead to the collapse of the entire system.

Explosive pandemic outbreaks


The threat by cascade effects might be even worse if the damage occurring in an early phase of the cascade process reduces the probability of resisting failures that are triggered later. A health system, in which financial or medical resources are limited, may be considered as an example for this. How will this system deal with emergent diseases? A computer-based study that I performed together with Lucas Böttcher, Nuno Araujo, Olivia Woolley Meza and Hans Hermann shows that the outcome very much depends on the connectivity between people who may infect each other. A few additional airline connections can make the difference between a case, where the disease will be contained, and where it turns into a devastating global pandemics. The problem is that crossing a certain connectivity threshold will change the system dynamics dramatically and unexpectedly. Thus, have we built global networks that behave in unpredictable and uncontrollable ways?

Systemic interdependencies


Recently, Shlomo Havlin and others made a further important discovery: they revealed that networks of networks can be particularly vulnerable to failures. A typical example is the interdependency between electrical and communication networks. Another example, which illustrates the global interdependencies between natural, energy, climate, financial, and political systems is the following: In 2011, the Tohoku earthquake in Japan caused a tsunami that triggered chain reactions and nuclear disasters in several reactors at Fukushima. Soon after this, Germany and Switzerland decided to exit nuclear power generation over the next decade(s). However, alternative energy scenarios turn out to be problematic as well. European gas deliveries depend on some regions, which we cannot fully rely on. Likewise, Europe’s DESERTEC project, a planned 1000 billion Euro investment into infrastructure to supply solar energy for Europe – has an uncertain future due to another unexpected event, the Arab Spring. This was triggered by high food prices, which were no longer affordable to many people. These high food prices, in turn, resulted partly from biofuel production, which intended to improve the global CO2 balance, but competed with food production. The increasing food prices were further amplified by financial speculation. Hence, the energy system, the political system, the social system, the food system, the financial system – they have all become closely interdependent systems, which makes our world ever more vulnerable to perturbations.

Have humans unintentionally created a "complexity time bomb"?


We have seen that, when systems are too much connected, they might get out of control sooner or later, despite advanced knowledge and technology, and best intentions to keep things under control. Therefore, as we have created more and more links and interdependencies in the world, we must ask ourselves: have humans inadvertently produced a "complexity time bomb", a system that will ultimately get out of control?

For a long time, problems such as crowd disasters and financial crashes have been seen as puzzling, ‘God-given’ phenomena or "black swans" one had to live with. However, problems like these should not be considered “bad luck.” They are often the consequence of a flawed understanding of counter-intuitive system behaviors. While conventional thinking can cause fateful decisions and the repetition of previous mistakes, complexity science allows us to understand the mechanisms that cause complex systems to get out of control. Amplification effects can result and promote failure cascades, when the interactions of system components become stronger than the frictional effects or when the damaging impact of impaired system components on other components occurs faster than the recovery to their normal state. That is, time scales of processes largely determine the controllability of a system as well. Delayed adaptation processes are often responsible for systemic instabilities and losses of control (see the related Information Box at the end).

For certain kinds of networks, the similarity of related cascade effects with those of chain reactions in nuclear fission is quite disturbing. Such processes are difficult to control. Catastrophic damage is a realistic scenario. Therefore, given the similarity of the cascading mechanisms, is it possible that our worldwide anthropogenic system will get out of control sooner or later? When analyzing this possibility, one must bear in mind that the speed of destructive cascade effects might be slow, and the process may not appear like an explosion. Nevertheless, the process may be hard to stop and lead to an ultimate systemic failure. For example, the dynamics underlying crowd disasters is slow, but deadly. So, what kinds of global catastrophic scenarios might we face in complex societies? A collapse of the global information and communication systems or of the world economy? Global pandemics? Unsustainable growth, demographic or environmental change? A global food or energy crisis? A cultural clash? Another global-scale war? A societal shift, driven by technological innovations? Or, more likely, a combination of several of these contagious phenomena? The World Economic Forum calls this the "perfect storm," and the OECD has formulated similar concerns.

Unintended wars and revolutions


Last but not least, it is important to realize that large-scale conflicts, revolutions, and wars can also be unintended results of systemic instabilities and interdependencies. Interpreting them as deeds of historical figures personalizes these phenomena in a way that distracts from their true, systemic nature. It is important to recognize that complex systems such as our economy or societies usually resist attempts to change them at large, namely when they are close to a stable equilibrium. This is also known as Goodhart's law (1975), principle of Le Chatelier (1850-1936), or as "illusion of control." Individual factors and randomness can only have a large impact on the path taken by the complex system, when the system is driven to a tipping point (also known as "critical point"). In other words, instability is a precondition for individuals to have a historical impact. For example, the historical sciences increasingly recognize that World War I was pretty much an unintended, emergent outcome of a chain reaction of events. Moreover, World War II was preceded by a financial crisis and recession, which destabilized the German economic, social, and political system. This finally made it possible that an individual could become influential enough to drive the world to the edge.

Unfortunately, civilization is vulnerable, and a large-scale war may happen again – I would say, it is even likely. A typical unintended path towards war looks as follows: The resource situation deteriorates, for example, because of a serious economic crisis. The resulting fierce competition for limited resources lets competition, violence, crime, and corruption rise, while solidarity and tolerance go down, so that the society is fragmented into groups. This causes conflict, further dissatisfaction and social turmoil. People get frustrated about the system, calling for leadership and order. Political extremism emerges, scapegoats are searched, and minorities are put under pressure. As a consequence, socio-economic diversity is lost, which further reduces the economic success of the system. Eventually, the well-balanced "socio-economic ecosystem" collapses, such that the resource situation (the apparent "carrying capacity") deteriorates. This destabilizes the system further, such that an external enemy is "needed" to re-stabilize the country. Finally, nationalism rises, and war may seem to be the only "solution" to keep the country together.

Note that a revolution, too, can be the result of systemic instability. Hence, it does not need to be initiated by an individual, "revolutionary" leader, who challenges an established political system. The breakdown of the former German Democratic Republic (GDR) and some Arab spring revolutions (for example, in Libya) have shown that revolutions may start even without the existence of a clearly identifiable political opponent leading the revolution. On the one hand, this is the reason, why such revolutions cannot be stopped by targeting a few individuals and sending them to jail. On the other hand, the absence of revolutionary leaders has puzzled secret services around the world – the Arabic spring took them by surprise. It was also irritating for sympathetic countries, which could not easily provide support for democratic civil movements. Whom should they have talked or given money to?

It provides a better picture to imagine such revolutions as a result of situations, in which the interest of government representatives and the people (or the interests of different societal groups) have drifted away from each other. Similar to tensions created by the drift of the Earth's tectonic plates, this would sooner or later lead to an unstable situation and an "earthquake-like" stress release (the "revolution"), resulting in a re-balancing of forces. Again, it is a systemic instability, which allows individuals or small groups to become influential eventually, while the conventional picture suggests that the instability of a political regime is caused by a revolutionary leader. Putting it differently, a revolution isn't usually the result of the new political leaders, but of the politics that was made before, which destabilized the system. So, we should ask ourselves, how well are our societies doing in terms of balancing the different interests in our societies, and in terms of adapting to a quickly changing world, due to demographic change, environmental change, technological change?

Conclusion


It is obvious that there are many problems ahead of us. Most of them result from the complexity of the systems humans have created. But how can we master all these problems? Is it a lost battle against complexity? Or do we have to pursue a new, entirely different strategy? Do we perhaps even need to change our way of thinking? And how can we generate the innovations needed, before it's too late? The next chapters will let you know...


Information Box: How harmless behavior can turn critical

In the traffic flow example and for the case of crowd disasters, we have seen that a system can get out of control when the interaction strength (e.g. the density) is too large.

How a change in density can turn harmless behavior of system components uncontrollable, is illustrated by the following example: Together with Roman Mani, Lucas Böttcher, and Hans J. Herrmann, I studied collisions in a system of equally sized particles moving in one dimension, similar to Newton's Cradle see video. We assumed that the particles tended to oscillate elastically around equally spaced equilibrium points, while being exposed to random forces generated by the environment. If the distance between the equilibrium points of neighboring particles was large enough, each particle oscillated around its equilibrium point with normally distributed speeds, and all particles had the same small variance in speeds.

However, as the separation of equilibrium points approached the particle diameter, we found a cascade-like transmission of momentum between particles see video. Surprisingly, towards the boundary particles, the variance of speeds was rapidly increasing. In energy-conserving systems, the speed variance of the outer particles would even tend towards infinity with increasing system size. Due to cascading particle interactions, this makes their speeds unpredictable and uncontrollable, even though every particle follows a simple and harmless dynamics.

Information Box: Loss of Synchronization

There is another puzzling kind of systemic instability that is highly relevant for our societies, as many socio-economic processes accelerate. It occurs when the separation of time scales gets lost. For example, hierarchical systems in physics and biology are characterized by the fact that adjustment processes on higher hierarchical levels are typically much slower than on lower hierarchical levels. Therefore, lower level variables adjust quickly to the constraints set by the higher level ones, and that is why the higher levels basically control the lower ones. For example, groups tend to take decisions more slowly than the individuals forming them, and the organizations and states made up from them change even more slowly (at least it has been like this in the past).
Time scale separation implies that the system dynamics is determined by a few variables only, which are typically related to the higher hierarchy levels. Monarchies and oligarchies are good examples for this. In current socio-political and economic systems, however, we observe the trend that higher hierarchical levels show accelerating speeds of adjustment, such that the lower levels can no longer adjust more quickly than the higher levels. This may eventually destroy time scale separation, such that many more variables start to influence the system dynamics. The result of such mutual adjustment attempts on different hierarchical levels could be turbulence, "chaos," or a breakdown of synchronization. In fact, systems often get out of control, if the adjustment processes are not quick enough and responses to changed conditions are delayed.




Friday, 3 October 2014

GENIE OUT OF THE BOTTLE: The digital revolution on its way

by Dirk Helbing

The information revolution, or "digital" revolution, may fundamentally transform our economy and societies within a short time. We need to prepare ourselves for this transformation and the new era to come. We might just have 20 years for this. Even though we may dislike such socio-economic change, we will probably not be able to stop it. While the breath-taking advances in information and communication technologies are challenging us, they also create great opportunities we haven’t had in hundred years. If we understand the fundamentally new logic of this digital future world and use its forces for our purposes, we will be able to benefit tremendously and fix a number of problems humanity has long been struggling with!



Never before have we so urgently needed politicians, business leaders, scientists, and citizens to master the challenges ahead of us. In the past we have had the agricultural society (with its "economy 1.0"), the industrial society (with its "economy 2.0"), and the service society (with its "economy 3.0"). But now, driven by computers, the Internet, social media, cognitive computing, and robotics, we are currently experiencing a third socio-economic transformation, the "digital revolution," which is creating the "economy 4.0". While we see the symptoms, such as mass unemployment, mass surveillance, cybercrime and cyberwar, we haven't understood the implications well. The emergence of movements such as the Occupy, Wikileaks, Anonymous, and the Pirate party has been controversially discussed in countless newspaper columns, but no one agrees on what they imply or where they are heading. My judgment is that we are in the middle of a socio-economic transformation. We are entering a world that will be ruled by a different logic. But who will rule it? Big business, governments empowered by Big Data, a computer-based super-human artificial intelligence, or its citizens?



The digital revolution arguably deserves even more attention than climate change. It will have dramatic impacts on almost every single aspect of our economy and societies within our lifetimes, and the pace of technological advances in the area of information and communication technologies is exponentially accelerating. Ten years ago, most of us had not the slightest idea that Facebook, Twitter, and iPhones would emerge. Even Microsoft and Yahoo didn't see these things coming. But now we talk about phenomena like "Twitter revolutions." What can we expect to happen in the next ten, fifty or hundred years? In the following, I offer some suggestions – not wide-eyed futurology, but considerations based on evidence and trends existing already. But before we discuss these trends, let’s look how the current situation emerged.



How the digital revolution came about




In 1943, Thomas John Watson Sr. (1874-1956), the chairman of IBM, is claimed to have said: "I think there is a world market for maybe five computers." At that time, almost nobody could imagine a mass market for computers. Today, we know how wrong these people were, but for decades, computers were barely useful to anyone. Even in 1968, an engineer at the Advanced Computing Systems Division of IBM asked of the microchip: "what ... is it good for?" Bill Gates is said to have suggested in 1981 that a computer memory of 640 K “ought to be enough for anybody." Today, our smartphones have a hundred thousand times more memory than that, and an iPhone has more processing power than the Apollo rocket required to fly to the moon and back.

The innovators of information technology did not have an easy life. The founder of Apple Computers Inc., Steve Job (1955-2011), remembers the failed attempts to get Atari and Hewlett-Packard interested in his and Steve Wozniak's personal computer: "So we went to Atari and said, 'Hey, we've got this amazing thing, even built with some of your parts, and what do you think about funding us? Or we'll give it to you. We just want to do it. Pay our salary, we'll come work for you.' And they said, 'No.' So then we went to Hewlett-Packard, and they said, 'Hey, we don't need you. You haven't got through college yet.' " Later, Steve Jobs even had to leave Apple for some time, before he came back to build the iPod, iPhone, iPad as well as iTunes and the AppStore, which made Apple, for several years, the most valuable company in the world.

But this is still the smaller part of the story. There was, in fact, another important development. In the late sixties, Arpanet was created to allow a few military computers to exchange information with each other. Arpanet was later opened up for civilian use, and thereby became the Internet. This "going public" eventually unleashed the power of information. Later, to support the collaboration between Swiss and French teams contributing to the CERN elementary particle accelerator, physicist Tim Berners-Lee (*1955) invented a hyperlink protocol allowing web pages to be linked with each other. This gave rise to the World Wide Web (WWW) and eventually made computers useful for ordinary people rather than just experts. It also made the Internet attractive for doing business. However, Berners-Lee’s ideas did not have an easy start either. At CERN he did not get the support he was asking for, so he went to the Massachusetts Institute of Technology (MIT), where he founded the World Wide Web Consortium (W3C).

Eventually, with the creation of Facebook in 2004 and Twitter in 2006, a billion people became part of a giant global information system. We are no longer just users of this system; we are also generators of data and services, and are in effect embedded in the system as “human processors.”



Computers more intelligent than humans?




Since decades, the processing power of computers is doubling roughly every 18 months. If this trend continues, then in 10 to 15 years from now we will have computers exceeding the processing capacity of the human brain. Computers are already out-performing humans in ever more ways. They are far better at doing calculations – and at chess. In 1996 IBM's Deep Blue computer defeated the best chess player in the world at that time, Garry Kasparov (*1963). In 2011, IBM's Watson, a “cognitive computer” that is able to judge the relevance of information from the Internet, beat human players in answering questions in the game show "Jeopardy." Now Watson is taking care of customer hotlines, as the computer is better at managing all the knowledge required to answer customer questions. Watson understands natural language and comes increasingly close to what humans can do – and in many respects even better. Now, by investing 1 billion dollars in Watson’s technology, IBM hopes to earn 100 times that amount in the years to come.

Europe wants to invest a comparable sum – 1 billion Euros – to build a supercomputer performing like the “human brain.” The USA decided to spend even 3 billion dollars on a brain project. The Google Brain project, led by the technology guru Ray Kurzweil, is a further attempt to turn the Internet into a intelligent entity that can think and decide.

While building brain-like performance will still take a few years, the Google car, which does not need a driver anymore, is already there. Furthermore, about 70 percent of all financial transactions in the world's stock markets are now performed by autonomous trading algorithms. Therefore, computers and robots are increasingly doing our work, and they will replace many jobs that can be performed according to logical routines and rules. We may even lose highly qualified jobs, which depend on skilled judgments, including medical doctors, care workers, scientists, lawyers, managers, and politicians, to some degree even teachers – and parents, with Korea engaging into robot-based child care. So what role will humans play in the future?

What are the socio-economic forces that will be created when, within the next 10 to 30 years, computers, algorithms and robots increasingly take over our roles and jobs? Remember that the first sector of the economy to emerge historically was agriculture, followed by industry and services. Now we are seeing the emergence of a fourth, "digital" sector – that of information and knowledge production. But history tells us that the emergence of any new economic sector often leads, decades later, to social and economic unrest, because the innovations behind the rise of the new sector undermine the basis of the previous sectors. As a result, not only are societies and the economy transformed, but our established worldviews are questioned as well: new socio-economic and political systems emerge. We must expect the same from the rise of the information sector.



The digital revolution is coming




We can already be sure that the digital revolution will fundamentally change many of our institutions and modes of action: how we educate (with open on-line courses and personalized education), how we do research (with the analysis of Big Data), how we move around (with self-driving cars) or transport goods (with drones), how we shop (with Amazon and eBay, for example), how we produce goods (with 3D printers), and also our health systems (with personalized medicine). Our political institutions (with evidence-based decision-making and citizen participation) and the basis of our economy (with production and sharing on the side of the consumer) will change as well. Financial business, which used to be the domain of banks, is increasingly replaced by algorithmic trading, Paypal, Bitcoin and Google Wallet, and so on. Therefore, banks such as the Credit Suisse are now building on advice of experts from the Silicon Valley. Moreover, the biggest share of the insurance business is now in financial derivatives traded at the stock markets. By now, they amount to a multiple of the global gross domestic product. Military experts believe that even wars may change from conventional to cyberwars.

Are our societies well prepared for this digital revolution? Let's look at the problem of the coming economic transformation more closely. In some industrialized countries, the proportion of agricultural jobs is now just 3 percent, while 300 years ago, it stood at 80 percent or more. How did we get there? Among other developments, it was the great success of James Watt's steam engine invented in 1769, which gave rise to the industrial revolution. As a result, many people lost their jobs to automation – in the textiles sector, for example. However, with the industrial revolution came consumerism, such that industrial production eventually created many new jobs. In fact, during its heyday, industrial production provided more than 30 percent of all jobs in developed countries. But later, due to the ever-increasing efficiency of industrial production, half of all industrial jobs were lost. And the number of industrial jobs will continue to fall, probably below 10 percent. Very soon, therefore, only 10 to 15 percent or so of jobs will be provided by the first and second sectors of our economy.

The third sector – services – is the result of another big innovation: the spreading of education at all levels of society from 1870ies onwards. As a consequence, reading and writing were not skills restricted to just a few people, and many more people could perform jobs requiring special qualifications. Therefore, the service sector developed, coming along with an increase of planning, administrative and management tasks. In modern societies, up to 70 percent of jobs are currently in the service sector. But again, more than half of them will be replaced – this time by robots, algorithms and computers. If we don't find a new way to produce jobs quickly, we will face unemployment rates much beyond the levels that our current socio-economic system can handle. In fact, all previous socio-economic transitions were accompanied by mass unemployment – one of the major forces transforming societies. Given that computers will reach human brainpower in about 10 to 15 years, we have very short time to adapt to the current societal transition. Technology experts say that automation will dramatically progress, leading to a “second machine age.” If we don't adapt fast enough, the transition will be sudden, harsh, and discontinuous.

It is known that delayed adaptation causes systems to become unstable. Financial crises, conflicts, or wars may be the consequences. In fact, there are current signs of destabilization in many countries around the world. These could be advance warning signals of socio-economic transitions to come. The list of countries that have lately seen social unrest includes Afghanistan, Pakistan, Iraq, Syria, Somalia, Nigeria, South-Sudan, Uganda, Congo, Central African Republic, Egypt, Bahrain, Libya, Libanon, Yemen, Tunisia, Turkey, China, India, Malaysia, Thailand, Brazil, Mexico, Ukraine, Spain, Greece, Romania, United Kingdom, and even Sweden and Switzerland. However, these countries are faced with different scenarios. Many African countries are now in a transition from an agricultural to an industrial society; some Asian countries are in a transition from an industrial to a service society; and the USA, Europe, and Japan, for example, are in a transition from a service to a digital society. Of course, these transitions may interact with each other. In the worst case, they might all come at about the same time, and soon.

Can we avoid or suppress these transitions? I don't think so. We may delay historical developments but we cannot stop them, and it wouldn’t be reasonable to try this. No country could afford to miss the long-term opportunities resulting from these transitions, which will lead to more efficient systems and a higher quality of life. This is all part of the cultural evolution as societies progress, which is driven by socio-economic forces such as the increase of complexity, diversity, and "collective intelligence." With the digital revolution, the genie is out of the bottle. Information and communication systems have grown ever more powerful, but a single mistake can be potentially disastrous. We must, therefore, learn to make the genie work for us. How can we do this? We must learn to understand the new logic of the digital societies we will live in. And we must learn it soon.



Two scenarios: coercion or freedom




Does the digital revolution imply that we will lose human rights such as privacy and freedom, dignity, or informational self-control? Later in this book I will explore whether classical democracies are likely to be replaced by "big governments" or "super-governments" that, in the very best case, would rule like a "benevolent dictator." Or are there any better ways to create social order and socio-economic well-being? Ways that don't use sensitive personal data and mass surveillance to control all individual actions, through methods ranging from speed control to Internet control and, one day, perhaps even thought control?

Hence, the crucial question is: what kind of society will the digital revolution produce? Given that there is ever more data, evidence-based decision-making will certainly spread. Moreover, as computers, algorithms and robots get ever more powerful, our socio-economic institutions will change. Many of our current jobs will be lost, and it will take time to create new jobs and new institutions. This brings us to a crossroads – or a tipping point, as complexity scientists would call it. Hence, we must make up our mind what path to choose. Do we want a super-government that takes all decisions for us in a top-down way or do we want a participatory market society with a large deal of bottom-up involvement?

We will explore the implications of both, and it is now time to publicly voice our choice to our political representatives. In the future, a super-government might actually tell us what to do, and it would be able to control everyone’s compliance by means of mass surveillance systems. In fact, we are increasingly using surveillance systems in our tax and traffic systems and in a quickly increasing number of other systems. It, therefore, seems that we have currently a trend towards an implementation of super-governments. But do we want to live in such a society? And would it be stable? I personally don't think so. The approach would reduce the innovation rate, and we would, therefore, eventually fail to adapt to technological, demographic, environmental and other change. In the end, a top-down controlled society would fall apart.



A better future ahead of us




Despite the many challenges, I am quite optimistic about our long-term future. We have mastered societal transitions already several times in human history. Therefore, we can master the coming one as well. The industrial societies can help the agricultural societies, the service societies can help the industrial societies. Of course, the service societies also have to manage their own transition towards digital societies. But we can master this.

Each age had brilliant minds, who helped to understand the transformations faced by society and offered their guidance. The philosopher Voltaire (1694-1778) provided the framework for the absolutism of the almighty King Ludwig XIV (1638-1715). The father of modern economics, Adam Smith (1723-1790), worked out the intellectual framework for industrial society. And later, the sociologists Emile Durkheim (1858-1917) and Max Weber (1864-1920) offered a way of thinking about a service society, built on the notion of administrations that try to plan and optimize systems. There will also be people, whose imagination will guide us into the digital age.

This book hopes to make a contribution to a necessary public debate, by discussing the two main frameworks for the digital society to come. One of the frameworks is based on the vision of a super-government, which takes evidence-based top-down decisions like a "wise king," empowered by huge masses of data ("Big Data") or even by a digital "Crystal Ball." This might be seen as a futuristic instantiation of the "Leviathan," the powerful state promoted by Thomas Hobbes (1588-1679). His credo was that social order could not exist without a powerful state, otherwise we would all behave like wild beasts ("homo hominis lupus").

The alternative framework for the digital society is based on the creation of self-regulating systems, empowered by real-time measurements and feedbacks. This vision takes the concept of the "invisible hand" forward, for which Adam Smith (1723-1790) is known. But financial meltdowns and "tragedies of the commons" such as environmental pollution suggest that the invisible hand often fails, which questions libertarian approaches.

However, will modern information and communication technologies finally allow us to reach desirable systemic outcomes based on decentralized bottom-up decision-making? What would it take? In the past, we have been pretty bad at finding suitable "rules of the game" ensuring desirable outcomes. Can we now make self-organization and self-regulation work? This is an exciting vision, as it could lead to a novel paradigm how to reach socio-economic order and success in the 21st century. It might lead us into an era of "collective intelligence" and an age of creativity, participation, and well-being.

On the long run, I am very confident about our future, because I believe in the power of ideas in the digital age. But no doubt, we could make serious mistakes on the way. Our financial system may fail, democracies may – intentionally or accidentally – turn into surveillance societies, or we may end up fighting wars against each other. Therefore, with this book, I am trying to explain the opportunities and risks ahead of us. For sure, the future world will have properties different from ours. But even though we can't predict this future well, can we get a glimpse of it at least? I believe so, to a certain extent. The properties of our future world will be emergent outcomes of interactions, and even though often counter-intuitive, we will be able to explain and shape many of them. To be beneficial, we need to understand, how such interactions can be chosen and designed – a capacity that will decide about our future fate. In a sense, this is the holy grail.



The new logic of prosperity and leadership




Most likely, the 21st century will be governed by fundamentally different principles than the 20th century, and I think the organization of our future socio-economic system will be based on collective intelligence rather than planning and optimization. That is why we need to change our way of thinking about our socio-economic systems. We need to recognize some fundamental trends and truths.

Information is ubiquitous and everywhere instantly available, such that borders tend to dissolve. The "second machine age" is coming quickly, challenging established jobs and routines. Most of the knowledge about our world is outdated by the time we learn it. Technological, social and economic system are becoming more variable, less predictable, and less controllable. Their increased connectivity means that they are more complex. The quick growth in the amount of data means that we are overloaded by it – yet we need ultimately to convert it into information and into knowledge we can act on. Furthermore, the more data we produce, the more difficult it will be to keep secrets, and the cheaper will data become. This means that we will make ever less profit from data itself, but more from algorithms that turn data into useful information and knowledge.

In a digital world, ideas can spread more quickly. Digitally literate people will be better informed than experts used to be, and therefore classical hierarchies may dissolve. Moreover, we are entering an increasingly immaterial age, determined by an abundance of information and ideas rather than objects. Data can be replicated as often as we like. It's a virtually unlimited resource, which may help us to overcome conflicts that scarce resources used to imply.

Services and products will be more individualized, personalized, and user-centric. Moreover, what used to be science fiction may become reality one day, actually more quickly than we think. The countries first recognizing these new principles and turning them into their advantage will be the ones that lead. Those failing to adapt to these trends in a timely manner will be in trouble. Given that computers may soon reach brain power, our societies may just have about 20 years for this adaptation – a very short time considering that planning and building a road often takes 30 years or more.



Creating a resilient society




The best way to prepare for the hardly predictable future to come is to create a society that is able to flexibly adapt to new kinds of situations. The digital revolution can certainly cause disruptive change. Hence, are our societies well prepared for shocks? Our response to September 11, 2001, questions this. Even though it was initially a local event, it has changed the face of our entire world. In the aftermath, we have built a security architecture to prevent us from terrorism. But with mass surveillance and armed police, are we now safe?[2] Less than 10 years later, we saw another, probably even more serious event that changed the face of the world: the financial crisis. Again, this started locally, but it had a major impact on the entire world.

Therefore, what can we do to be better prepared for future challenges? There is a whole bundle of measures, ranging from risk assessment, prediction, prevention, intervention, insurance, and hedging. However, we must recognize that, in a world that is not totally predictable, problems will always occur, and accidents will always happen. That is why we need resilient, "survivable" systems.

Increasing resilience will simultaneously fix a number of problems our world is currently suffering from. But what exactly is resilience about? It means the ability of a system to absorb shocks and to recover from them quickly and well. If we fall and get wounded, we will usually recover from this, because our body is resilient to such shocks. So, how to increase societal resilience? First of all, we need systems with redundancies, reserves, backup strategies, and alternatives (a "plan B", "plan C"). Second, we must have mechanisms to deal with speed and complexity. Third, diversity increases the survivability of a system, and its adaptability to new circumstances. Forth, one may reduce system size and interdependencies, i.e. use modular designs. Otherwise dynamic decoupling strategies, engineered breaking points, or shock absorbers are a "must have" to avoid cascading failures. Think of the electrical fuses at home or the crush zone of your car, which are there to protect the sensitive parts of the system (your home, your life). Fifth, adaptive self-regulation mechanisms or, in other words, stabilizing real-time feedback, can increase resilience tremendously. And finally, transparency, accountability, responsibility, awareness are important to increase resilience as well.

Thus, are we well prepared to master the digital revolution? Does mass surveillance, together with armed police, create a resilient society? They are certainly intended to be tools to prevent bad things from happening, but is this good enough? Or is it even counter-productive, as they suppress societal change and, hence, needed societal adaptation to a changing reality? Aren't we, in fact, trying to fight the outcome of innovation, or even innovation itself? I personally believe that we must – and can – prepare ourselves much better, in a different way.

Let us for a moment discuss the illustrative problem of traffic safety. We know that accidents keep happening, and this is bad. But how would it be to live in a world without traffic? Our economy would not work, and we would not be able to live in modern societies. Therefore, we have come up with measures to reduce the number and impact of accidents. How have we done this? We have improved traffic rules. We have developed better technology. We have created emergency services and built hospitals. We have developed better brakes and driver assistance systems that empower drivers to do a better job. We provide traffic information to prevent single accidents from messing up the entire system. And we have designed cars in better ways. Decades ago, a car's body was often little damaged by an accident, but driver and passengers frequently died. Now, cars are constructed to be shock absorbers: even a small accident may destroy the entire car, but the risks of injuries and deaths have been dramatically reduced.

This can guide our thinking of how to deal with challenges of society. It shows us that the idea to protect institutions from changing is just the opposite of what would make societies resilient. This is like the stiff car body, which in case of an accident is not made to serve the life of the driver. For resilience, our institutions must be able to flexibly adapt in ways that serve the citizens best.

But why at all do we see so many global crises now? In just a few months, our world was confronted with three new problems: the Ebola epidemics, a crisis in Ukraine endangering world peace, and the Islamic State fighters. These problems, too, started locally, but their implications might become relevant for all of us.


[1] 
Dear Reader,
thank you for your interest in this chapter, which is thought to stimulate debate.
What you are seeing here is work in progress, a chapter of a book on the emerging Digital Society
I am currently writing. My plan was to elaborate and polish this further, before I share this with anybody else. However, I often feel that it is more important to share my thoughts with the public now than trying to perfect the book first while keeping my analysis and insights for myself in times requiring new ideas.
So, please apologize if this does not look 100% ready. Updates will follow. Your critical thoughts and constructive feedback are very welcome. You can reach me via dhelbing (at) ethz.ch or @dirkhelbing at twitter.
I hope these materials can serve as a stepping stone towards mastering the challenges ahead of us and towards developing an open and participatory information infrastructure for the Digital Society of the 21st century that would enable everyone to take better informed decisions and more effective actions.
I believe that our society is heading towards a tipping point, and that this creates the opportunity for a better future.
But it will take many of us to work it out. Let’s do this together!
Thank you very much, I wish you an enjoyable reading,
Dirk Helbing
PS: Special thanks go to the FuturICT community and to Philip Ball.
[2] The events in Ferguson rather suggest that they create new problems.

Friday, 26 September 2014

BIG DATA: A Powerful New Resource for the 21st Century

by Dirk Helbing

This chapter is a free translation of an introductory article on "Big Data - Zauberstab und Rohstoff des 21.Jahrhunderts" originally published in Die Volkswirtschaft - Das Magazin für Wirtschaftspolitik (5/2014), 

Abstract


Information and communication technology (ICT) is the economic sector that is developing most rapidly in the USA and Asia and generates the greatest value added per employee. Big Data - the algorithmic discovery of hidden treasures in large data sets - creates new economic value. The development is increasingly understood as a new technological revolution. Switzerland could establish itself as data bank and Open Data pioneer in Europe and turn into a leading place in the area of information technologies.


What is Big Data?

When the social media portal WhatsApp with its 450 million users was recently sold to Facebook for$19 billion - almost half a billion dollars was made per employee. "Big Data" is changing our world. The term, coined more than 15 years ago, means data sets so big that one can no longer cope with them with standard computational methods. Big Data is increasingly referred to as the oil of the 21st century. To benefit from it, we must learn to "drill" and "refine" data, i.e. to transform them into useful information and knowledge. The global data volume doubles every 12 months. Therefore, in just two years, we produce as much data as in the entire history of humankind.

Tremendous amounts of data have been created by four technological innovations:
  • the Internet, which enables our global communication
  • the World Wide Web, a network of globally accessible websites that evolved after the invention of hypertext protocol (HTTP) at CERN in Geneva
  • the emergence of social media such as Facebook, Google+, Whatsup, or Twitter, which have created social communication networks, and 
  • the emergence of the "Internet of Things'', which also allows sensors and machines to connect to the Internet. Soon there will be more machines than human users in the Internet.


Data sets bigger than the largest library

Meanwhile, the data sets collected by companies such as eBay, Walmart or Facebook, reach the size of petabytes (1 million billion bytes) - one hundred times the information content of the largest library in the world: the U.S. Library of Congress. The mining of Big Data opens up entirely new possibilities for process optimization, identification of interdependencies, and decision support. However, Big Data also comes with new challenges, which are often characterized by four criteria:
  • volume: the file sizes and number of records are huge,
  • velocity: the data evaluation has often to be done in real-time,
  • variety: the data is often very heterogeneous and unstructured,
  • veracity: the data is probably incomplete, not representative, and contains errors

Therefore, one had to develop completely new algorithms: new computational methods. Because it is inefficient for Big Data processing to load all relevant data into a shared memory, the processing must take place locally, where the data resides, on potentially, thousands of computers. This is accomplished with massively parallel computing approaches such as: MapReduce or Hadoop. Big Data algorithms detect interesting interdependencies in the data ("correlations"), which may be of commercial value, for example, between weather and consumption or between health and credit risks. Today, even the prosecution of crime and terrorism is based on the analysis of large amounts of behavioral data.


What do applications look like?

Big Data applications are spreading like wildfire. They facilitate personalized offers, services and products. One of the greatest successes of Big Data is automatic speech recognition and processing. Apple's Siri understands you when asking for a Spanish restaurant, and Google Maps can lead you there. Google Translate interprets foreign languages by comparing them with a huge collection of translated texts. IBM's Watson computer even understands human language. It can not only beat experienced quiz show players, but even take care of customer hotlines - often better than humans. IBM has recently decided to invest $1 billion to further develop and commercialize the system.

Of course, Big Data plays an important role in the financial sector. Approximately seventy percent of all financial market transactions are now made by automated trading algorithms. In just one day, the entire money supply of the world is traded. Such quantities of money also attract organized crime and financial transactions are scanned by Big Data algorithms for abnormalities to detect suspicious activities. The company Blackrock uses a similar software called "Aladdin", to successfully speculate with funds amounting to multiple times the gross domestic product (GDP) of Switzerland.

Box 1:
To get an overview of the ICT trends, it is worthwhile to look at Google with over 50 software platforms. The company invests nearly $6 billion in research and development annually. Within just one year, Google has introduced self-driving cars, invested heavily in robotics, and started a Google Brain project to add intelligence to the Internet. Through the purchase of Nest Labs, Google has also invested $3.2 billion in the "Internet of Things". Furthermore, Google X has been reported to have around 100 secret projects in the pipeline.


The potential is great...

No country today can afford to ignore the potentials of Big Data. The additional economic potential of Open Data alone - i.e. of data sets that are made ​​available to everyone - is estimated by McKinsey to be between 3,000 to 5,000 billion dollars globally each year [2]. This can benefit almost all sectors of society. For example, energy production and consumption can be better matched with "smart metering", and energy peaks can be avoided. More generally, new information and communication technologies allow us to build "smart cities". Resources can be managed more efficiently and the environment protected better. Risks can be better recognized and avoided, thereby reducing unintended consequences of decisions and identifying opportunities that would otherwise have been missed. Medicine can be better adapted to the patients, and disease prevention may become more important than curing diseases.


... but also the implicit risks

Like all technologies, Big Data also implies risks. The security of digital communication has been undermined. Cyber ​​crime, including data, identity and financial theft, quickly spread on ever greater dimensions. Critical infrastructures such as energy, financial and communication systems are threatened by cyber attacks. They could, in principle, be made dysfunctional for an extended time period.

Moreover, while common Big Data algorithms are used to reveal optimization potentials, their results may be unreliable or may not reflect causal relationships. Therefore, a naive application of Big Data algorithms can easily lead to wrong conclusions. The error rate in classification problems (e.g. the distinction between "good" and "bad" risks) is often relevant. Issues such as wrong decisions or discrimination must be seriously considered. Therefore, one much find effective procedures for quality control. In this connection, universities will likely play an important role. One must also find effective mechanisms to protect privacy and the right of informational self-determination, for example, by applying the Personal Data Purse [1] concept.


The digital revolution creates an urgency to act

Information and communication technologies are going to change most of our traditional institutions: our educational system (personalized learning), science (Data Science), mobility (self-driving cars), the transport of goods (drones), consumption (see amazon and ebay), production (3D printers), the health system (personalized medicine), politics (more transparency), and the entire economy (with co-producing consumers, so-called prosumers). Banks are losing more and more ground to algorithmic trading, alternative payment systems such as Bitcoins, Paypal and Google Wallet. Moreover, a substantial part of the insurance business takes place in financial products such as credit default swaps. For the economic and social transformation into a ``digital society'', we may perhaps just have 20 years. This is an extremely short time period, considering that the planning and construction of a road often requires 30 years or more.

The foregoing implies an urgent need for action on the technological, legal and socio-economic level. Some years ago, the United States started a Big Data research initiative amounting to 200 million dollars followed by further substantial investments. In Europe, the FuturICT project (www.futurict.eu) has developed concepts for the digital society within the context of the EU flagship competition. Other countries have already started to implement this concept, for example, Japan has recently launched a $100 million 10-year project at the Tokyo Institute of Technology. In addition, numerous other projects exist, particularly in the military and security sector, which often have multiples of the budgets mentioned above.


Switzerland can become a European driver of innovation for the digital era

Switzerland is well positioned to benefit from the digital age. However, it is insufficient to reinvent and build upon already existing technologies in Switzerland. New inventions that will shape the digital age must be invented. The World Wide Web was once invented in Switzerland, the largest civil Big Data competence in the world exists at CERN, however the USA and Asian countries have the lead in commercializing Big Data to date. With the NSA controversy, the ubiquity of wireless communication sensors as well as the "Internet of Things",  a new opportunity is emerging.

With targeted support of ICT activities at its universities, Switzerland could take the lead in Europe's research and development. Swiss academia has excelled with the scientific coordination of three out of six finalists of the EU FET flagship competition.
At the moment, however, there is only a focus on the digital modeling of the human brain and robotics. However from 2017 onwards, the ETH domain plans to increasingly invest into the area of Data Science, the emerging research field centered around the scientific analysis of data.

In view of the fast development of the ICT area, the huge economic potential as well as the transformative power of these technologies, a prioritized, broad and substantial financial support is a matter of Swiss national interest. With its basic democratic values, legal framework and ICT focus, Switzerland is well prepared to become Europe's innovation driver for the digital age.

Box 2:
How will the digital revolution change our economy and society? How can we use this as an opportunity for us and reduce the related risks? For illustration, it is helpful to recall the factors that enabled the success of the automobile age: the invention of cars and of systems of mass production; the construction of public roads, gas stations, and parking lots; the creation of driving schools and driver licenses; and last but not least, the establishment of traffic rules, traffic signs, speed controls, and traffic police.
What are the technological infrastructures and the legal, economic and societal institutions needed to make the digital age a big success? This question would set the agenda of the Innovation Alliance. A partial answer is already clear: we need trustworthy, transparent, open, and participatory ICT systems, which are compatible with our values. For example, it would make sense to establish the emergent "Internet of Things" as a Citizen Web. This would enable self-regulating systems through real-time measurements of the state of the world, which would be possible with a public information platform called the "Planetary Nervous System". It would also facilitate a real-time measurement and search engine: an open and participatory "Google 2.0."


To protect privacy, all data collected about individuals should be stored in a Personal Data Purse and, given informed consent, processed in a decentralized way by third-party Trustable Information Brokers, allowing everyone to control the use of their sensitive data. A Micro-Payment System would allow data providers, intellectual property right holders, and innovators to get rewards for their services. It would also encourage the exploration of new and timely intellectual property right paradigms ("Innovation Accelerator"). A pluralistic, User-centric Reputation System would promote responsible behavior in the virtual (and real) world. It would even enable the establishment of a new value exchange system called "Qualified Money," which would overcome weaknesses of the current financial system by providing additional adaptability.
A Global Participatory Platform would empower everyone to contribute data, computer algorithms and related ratings, and to benefit from the contributions of others (either free of charge or for a fee). It would also enable the generation of Social Capital such as trust and cooperativeness, using next-generation User-controlled Social Media. A Job and Project Platform would support crowdsourcing, collaboration, and socio-economic co-creation. Altogether, this would build a quickly growing Information and Innovation Ecosystem, unleashing the potential of data for everyone: business, politics, science, and citizens alike.

Further Reading

[1] Y.-A. de Montjoye, E. Shmueli, S. S. Wang, and A. S. Pentland (2014) openPDS: Protecting the Privacy of Metadata through SafeAnswers,

[2] McKinsey & Company (2013) Open data: Unlocking innovation and performance with liquid information,


Tuesday, 23 September 2014

Creating ("Making") a Planetary Nervous System as Citizen Web

by Dirk Helbing

The goal of the Planetary Nervous System is to create an open, public, intelligent software layer on top of the "Internet of Things" as the basic information infrastructure for the emerging digital societies of the 21st century.

After the development of the Computer, Internet, the World Wide Web, Smartphones and Social Media, the evolution of our global information and communication systems will now be driven by the "Internet of Things" (IoT). Based on wirelessly connected sensors and actuators,[1] it will connect "things" (such as machines, devices, gadgets, robots, sensors, and algorithms) with things, and things with people.

Already now, more things than people are connected to the Internet. In 10 years time, it is expected that something like 150 billion sensors will be connected to the IoT. Given such masses of sensors everywhere around us -- sensors in our coffee machine, our fridge, our tooth brush, our shoes, our fire alarm etc. -- the IoT could easily turn into a dystopian surveillance nightmare, if largely controlled by one company or by the state. For the IoT to be successful, people need to be able to trust the new information and communication system, and they need to be able to exert their right of informational self-determination, which also requires the possibility to protect privacy.

Most likely, the only way to establish such a trustable, privacy-respecting IoT is to build it as a Citizen Web. Citizens would deploy the sensors in their homes, gardens, and offices themselves, and they would decide themselves what sensor information to open up (i.e. decrypt), and for whom (and for how long). In other words, the citizens would be in control of the information streams. A software platform such as open Personal Data Store (openPDS)[2] would allow everyone to manage the access to personal data produced by the IoT.

What are the benefits of having an "Internet of Things"?

  • One can perform real-time measurements of the (biological, technological, social and economic) world around us
  • This information can be turned into (real-time) maps of our world[3] and serve as compasses for decision-makers, enabling them to take better decisions and more effective actions, considering externalities
  • One can build self-organizing and self-regulating systems, based on real-time feedback and adaptation[4]
  • Sensor Kits and Smartphones, to measure the environment
  • Algorithms and filters to encrypt information or degrade it such that it is not sensitive anymore[5]
  • Ad hoc network / mesh net (e.g. firechat) to enable direct communication between wirelessly communicating sensors
  • Server architecture to collect, manage and process data
  •  A data analytics layer and possibly a search engine and Collective Intelligence/Cognitive Computing layer on top
  • An open Personal Data Store (such as openPDS) to empower users to exercise their right of informational self-determination
  • An app-store-like Global Participatory Platform (GPP) to share data, algorithms, and ratings
  • An editor allowing non-expert users to combine inputs and outputs in playful, creative ways 
  • A multi-dimensional reputation and micro-payment system
  •  A project platform to allow the Nervous community to coordinate and self-organize their activities and projects
Both Planetary Nervous System Apps would offer a rich Open Data stream accessible for everyone. They would build something like a "real-time data streaming Wikipedia", offering people and companies to build services and products on top. The PNS is hence an attempt to enable and catalyze new creative jobs in times where the digital revolution is expected to eliminate about 50% of the conventional jobs of today.
Uses of these kinds will be enabled by a software layer that we call the "Planetary Nervous System" (PNS) or just "Nervous". It offers new possibilities that will allow humanity to overcome some long-standing problems (such as systemic instabilities or "tragedies of the commons" like environmental degradation, etc.), and to change the world to the better.

Basic Elements of the Planetary Nervous System
We will build two variants of the Planetary Nervous System App for smart devices such as smartphones: Nervous and Nervous+. While Nervous would not save original sensor data, Nervous+ would potentially do so. Nervous is thought to be for users that are concerned about their personal data, while Nervous+ offers additional functionality for people who are happy to share data of all kinds. Hence, the users can choose the system they prefer.


Creating a public good, and business and non-profit opportunities for everyone by maximum openness, transparency, and participation
The main goal of the PNS project is to create a public good, namely the basic information infrastructure for the emerging digital societies of the 21st century. Besides providing Open Data streams, the Planetary Nervous System may nevertheless offer some premium services to people and/or institutions, who pay for the services or have qualified to receive them for free (such as committed scientists or citizens). "Qualification" means contributions made to the components of the Planetary Nervous System, but also a responsible use of the information services. In this way, we want to reduce malicious uses of the powerful functionality of Nervous+ as much as possible.

The profits created by the PNS would be managed, for example, by a benefit corporation, which is committed to improving social and/or environmental conditions. The largest share of the profits should be used to promote the science, research and development promoting the PNS and services built on top of it. Profits created with inventions of the PNS shall be also used to support the PNS project.

As the PNS project wants to grow a public good for everyone, the Planetary Nervous System project is committed to opening up its source codes, as much as this is not expected to create security issues or dangers to human rights. Depending on the competitive situation the PNS is in, the publication may be done with a delay (usually less than 2 years). To minimize delays we will create incentives for early sharing.

The goal of this strategy is to catalyze an open information and innovation ecosystem. Others will be able to use our codes (and other people's open source codes), modify them and share them back. The same will apply to data, Apps, and other contributions. In this way, the Nervous community will benefit maximally from contributions of other Nervous members, and everyone can build on functionality that has been created by others.

Contributions of volunteers will be acknowledged by mentioning the respective creators by name (if they don't prefer to stay anonymous or pseudonymous). In addition, contributions will be rewarded by ratings, reputational values, or scores, which may be later used to get access to premium services. These would include larger query or data volumes ("power users") or an earlier access to codes that will be publicly released with a delay, or further benefits. The PNS project may also hand out medals or prizes for outstanding contributions, or highlight them in social or public media.

The role of Citizen Science
For the Planetary Nervous System to be successful, it is crucial to grow a large community of users, but the underlying logic of sharing, bottom-up involvement and informational self-determination demands that everyone is encouraged to contribute to the creation of the system itself. The system would hence be built similar to Wikipedia or OpenStreetMap. In fact, the success of OpenStreetMap is based on the contributions of 1.5 million volunteers worldwide.

This is, why the Nervous project wants to engage with Citizen Science, to grow the Planetary Nervous System as a Citizen Web. As basis of citizen engagement, the Nervous Team will provide (a) kits containing sets of sensors[6] and actuators (e.g. a basic kit, and several extension kits) and (b) a GPP portal, where people can download (and upload) algorithms ("Apps"), which will run on the sensors and thereby produce certain kinds of functionalities.

The CitizenScience community will be engaged in certain measurement tasks (e.g. "measure the noise distribution in your city as a function of time", or "measure data enabling weather predictions"). It will also be engaged to come up with innovative ways to use sensor data and turn them into outputs (i.e. to produce new codes or modify existing ones, thereby creating new Apps). For this, the PNS team will provide tools (such as an editor), allowing non-expert users to transform inputs into outputs in playful, creative ways. Playfulness, fun and reputation are hence offered in exchange for contributing to the development and spreading of the PNS. As a result, we will get new measurement procedures for science, and adaptive feedback processes to create self-regulating systems.


The Planetary Nervous System is already being built, see the @NervousNet on Twitter. To join our development team as a volunteer in a similar way as Linux, Wikipedia or OpenStreetMap are being created, please contact Dirk Helbing dhelbing (at) ethz.ch







[1] Actuators are devices that can induce change, for example, motors.
[2] see http://openpds.media.mit.edu/ and http://www.plosone.org/article/info%3Adoi%2F10.1371%2Fjournal.pone.0098790
[3] Such maps may include environmental maps depicting environmental changes and who causes them, resource maps visualizing resources and who uses them, as well as risk maps, crisis maps, or conflict maps.
[4] Smart city applications are often of this kind.
[5] For example, one may average over personal data, and throw away the original data. Microphone data may be degraded by using a lowpass filter, such that spoken words cannot be anymore understood, remaining basically with a noise level measurement. Access to unfiltered video or microphone data would require explicit approval, similar to accepting a call. Data shared with a restricted circle will be encrypted.
[6] Such sensors may measure temperature, noise, humidity, health, or anything else.