Responsible IT Innovation: How to digitally upgrade our society?

by Dirk Helbing (ETH Zurich/Delft University of Technology) and 
Jeroen van den Hoven (Delft University of Technology)

Why is the world confronted with so many problems, even though we have more data and better technology than ever? We are not using it rightly.

When Chris Anderson postulated "the end of theory" in 2008, there was no public outcry. While the new field of Big Data analytics spread, altogether it seems that scientists did not pay much attention to the claim that "the data deluge makes the scientific method obsolete".[1] This was a big mistake - not only because it weakened the role of science, but also because of the implications for society at large.  

In the meantime, IT companies and governments around the world have made huge investments in infrastructures and machine learning tools to store and mine Big Data. There are entirely new approaches such as predictive policing, "big nudging"[2] (the large-scale manipulation of individual emotions, opinions, decisions and behaviors[3] using knowledge from behavioral economics and massive amounts of personal data), and citizen scores[4] (rewarding and punishing citizens for specific behaviors and those of their friends). Even the idea of predictive sentencing (based on the likelihood of future offenses) is spreading.[5] This questions human rights, the separation of powers, democracy and justice - achievements of hundreds of years of human cultural, political and institutional history, lessons learnt from many conflicts and wars.

It is time to make up our minds and respond to the novel, data-driven reality with a constructive operational framework that enables sustainable new modes of political thinking. Digital technologies and data science are now used to shape our societies, to constitute the very fabric of our sociality, often circumventing science and democratic decision-making.[6] Data-driven updates of historical ways of running societies are now in the making, for example, fascism 2.0 (a big brother and brave new world society), communism 2.0 (distributing rights and resources based on a paternalistic "benevolent dictator" approach), feudalism 2.0 (based on a few monopolies and a new kind of caste system), and capitalism 2.0 (discussed below). Moreover, cybernetic concepts of society, which use automated feedback loops, have spread from Chile to Singapore to other countries.[7] Google is aiming to build the operating system of the digital society,[8] and IBM's Watson is proposed for US president.[9] Some experts even suggest, Artificial Intelligence will solve the world's problems, once machine intelligence surpasses human intelligence.[10] Is this a reasonable expectation?

We must question such overly optimistic claims and projections. Even strong supporters of new technologies such as Big Data and Artificial Intelligence should better acknowledge their limitations and side effects. One would think that, before one tries to turn the world into a "data-driven paradise", one would demonstrate the feasibility of this idea and the superiority of the new approach. However, none of the leading IT nations has succeeded in contributing to the top 10 most livable cities on our planet. In addition, the world is in pretty bad shape, in spite of the fact there are more data and better technologies than ever. Financial crises, violent conflicts, mass migration around the planet, increasing cyber threats, dangerous global warming and emerging shortages of resources are just a few examples of problems, for which data-driven solutions have not been found. We also have not managed to overcome the last financial, economic, and public spending crisis, while the next one seems to be looming already. 

The fact that we are far from solving these problems is actually not by chance.

First, conventional Big Data analytics must be improved in various aspects. So far, there is a lack of quantitative indicators to judge the quality of Big Data analyses. The bigger a data set the more spurious correlations and meaningless patterns occur. For instance, most patterns of stars one sees in the sky do not have scientific relevance. Given such "over-fitting" problems, how significant are Big Data results from a statistical point of view? How much are they biased by the data providers and those who build and choose the algorithms? To what extent do information systems help people to take well-informed, conscious and responsible decisions, and to what extent do they opinionate or manipulate them?

Second, how robust are the results of Big Data analyses, given the issue of parameter sensitivity? For example, how much do classification results and error rates depend on the classification method and model parameters applied? In fact, for every terrorist there are hundreds of people on the lists of terror suspects, who will never commit a serious crime. More generally, how often do Big Data analyses discriminate people, and how to protect or compensate them?[11] For instance, what if health insurance rates would depend on the food that people eat: would it be fair to charge women and men, Jews, Christians and Muslims different amounts (on average)?

Third, going one step further, how much trust, how much solidarity and other social capital is destroyed or exploited by today's data-driven business models?¹¹ This is important to know if we want to avoid the risk that public interests are undermined and public values corroded. For example, there is increasing awareness that "big nudging" weakens critical thinking and resistance to propaganda, social cohesion, informational self-determination, and the self-control of our lives.² Thus, how well are the implications of Big Data algorithms compatible with democracy, freedom, fairness, human rights, and the separation of powers? Scientists should start to measure these things. One urgently needs ways to assess the quality of data and algorithms - to be precise: various kinds of qualities. And one needs different standardized ways to look at data: digital filters providing pluralistic perspectives.

A system that is oriented at a single indicator (such as gross national product per capita or a citizen score) does not serve humans well. It is crucial to create tools and institutions, which support the transparent design of technological systems that are compatible with the moral, social and cultural values of a society. For example, system designs for efficiency may look different from designs for sustainability, usability, flexibility, resilience, fairness, equality, justice, transparency, accountability, (informational) self-determination, self-organization, democracy, inclusion, dignity, happiness, well-being, inspiration, creativity, innovation, safety, security, health, empathy, friendship, solidarity, participation, or peace. Virtual Reality may now help to make these various perspectives understandable.

Importantly, Responsible Data Science and Responsible AI require value pluralism.[12] Therefore, the use of Big Data and AI should be opened up beyond the circles of big corporations, secret services and the military, as the openAI initiative demanded as well. The complexity of modern societies requires the ability to see and judge the world from various perspectives and to bring them in a suitable balance, otherwise a society becomes dysfunctional sooner or later. To achieve a high quality of life, one needs pluralistic, flexible, adaptive, context-dependent and culturally fitting schemes of governance, which offer diverse opportunities for many different groups of people. Compared to this, today's data-driven schemes to govern societies are overly simplistic. While they claim to be human-centric, the great majority of people has actually no say in how the new operating system would work and what would be its goals.

However, it is important to understand that society is not a machine. Automating and running it like a modern production line is not adequate. For example, it would not be enough to simulate majority decisions in a computer using digital doubles. This would mean to turn average behaviors into social norms or standards, which would constitute something like a majority dictatorship with little protection of minorities - certainly not a desirable governance approach. The long-term consequences could be a loss of socio-economic diversity, a decline in the innovation rate, the loss of economic efficiency, and socio-political instability, potentially leading to war. To achieve culturally fitting, sustainable and legitimate results, deliberation processes are crucial. By identifying solutions that are satisfactory from various perspectives they manage to serve different functionalities at the same time. Moreover, by combining and integrating various perspectives, they create innovative solutions that manage to bring different interests under one roof.[13] This calls for suitable deliberation platforms enabling democracy 2.0,[14] which support the emergence of collective intelligence, i.e. of solutions that are better than any single solution by unleashing the potential of diversity. Civil society is still an underused resource.

Of course, governments should utilize the possibilities of data and information technologies. However, it is highly non-trivial what constitues a responsible and good use. The statement "code is law" reflects the fact that algorithms increasingly shape our reality.[15] While the behavior of people is regulated by hundreds of laws, algorithms are subject to very few regulations, even though they may have super-human powers. This is inappropriate and dangerous, as actors without legitimate grounds can increasingly interfere with the lives of millions of people, often without their knowledge. Altogether, this does not seem to be helpful. Humans are increasingly losing control over the information and communication technologies they have created. For instance, cybercrime costs us 3 trillion dollars annually, and it is increasing exponentially. Therefore, our judgment is that companies and institutions are just at the beginning of making good use of Big Data and AI. The application of these technologies is not yet mature. They lack participatory opportunities, transparency, and mechanisms to promote fact-based, responsible, ethical and legitimate use.

While many countries and companies may have put in place IT infrastructures to control society with the very best intentions, these imply serious new threats. Imagine, for example, that political extremists would gain control over today's powerful IT tools, which enable effective and hard-to notice censorship and propaganda. It is clear that this could have catastrophic and irreversible consequences. For such reasons, one must ensure that the use of powerful IT tools will be soon embedded in a suitable institutional framework. Improvements are urgently needed, including parliamentary control, scientific use, ethical oversight, transparency, and compensation of victims. In some democratic countries one can already see what can happen if one does not take care of these things. A gradual restriction of democratic principles or even a transition to a totalitarian state appear to be entirely possible.

In summary, super-intelligent systems will certainly help us to solve a number of difficult problems, but not all. Social capital, culture, security, peace, financial stability, mass innovation, and thriving lives (including jobs for all) seem to be out of scope. To address these issues, there is a need to digitally upgrade our socio-economic system. It is important to understand that the digital transformation of societies takes place in two phases: The first phase is based on Big Data, Artificial Intelligence, and top-down solutions, but it is likely to destroy more jobs than it creates. The second phase will add open and participatory bottom-up structures and create a decentralized information, innovation, production and service ecosystem, using a combination of the Internet of Things, Complexity Science, FinTech (e.g. Bitcoin-inspired block chain technologies), and new kinds of Social Technologies such as personal digital assistants. 

As Albert Einstein said: "One can not solve our problems with the same kind of thinking that created them." To fix our problems, there is a need for a new paradigm, otherwise societies will fail. Therefore, we propose to digitally upgrade capitalism, the so far most successful economic system, which is based on bottom-up self-organization, entrepreneurial and individual freedom, competition, high innovation rates, efficiency, flexibility, and resilience. Now, one needs to add a liberal and democratic framework supporting "(eco-) systems thinking", co-creation, co-evolution, and collective intelligence. 

The Internet of Things will finally allow us to measure external effects of interactions, to price such positive and negative externalities, and to trade them in a multi-dimensional way - a system we propose to call "finance 4.0". This will create entirely new markets, but first of all, promote resource efficiency, a circular economy, sharing, and cooperation. In this multi-dimensional exchange system, additional kinds of "money" can be created in a bottom-up way. This can generate a living for everyone and public resources in times, where progressive automation is threatening incomes, taxes, and consumption, i.e. the basis of our economy. Let us now build the framework for the digital society to come. There is no time to lose.

---

[1] C. Anderson, The end of theory: The data deluge makes the scientific method obsolete, Wired Magazine (June 23, 2008), see http://www.wired.com/2008/06/pb-theory/

[2] D. Helbing et al., Digitale Demokratie statt Datendiktatur, Spektrum der Wissenschaft (January 2016), see http://www.spektrum.de/pdf/digital-manifest/1376682

[3] R.M. Bond et al., A 61-million-person experiment in social influence and political mobilization, Nature 489, 295-298 (2012); A.D.I. Kramer et al. Experimental evidence of massive-scale emotional contagion through social networks, PNAS 11, 8788-8790 (2014); R. Epstein and R.D. Robertson, The search engine manipulation effect (SEME) and its possible impact on the outcome of elections, PNAS 112, E4512-E4521 (2015).

[4] ACLU: Orwellian citizen score, China's credit score system, is a warning for Americans, Computerworld (October 7, 2015), see http://www.computerworld.com/article/2990203/security/aclu-orwellian-citizen-score-chinas-credit-score-system-is-a-warning-for-americans.html

[5] A.M. Barry-Jester et al., Should prison sentences be based on crimes that haven't been committed yet? FiveThirtyEight (August 4, 2015), see http://fivethirtyeight.com/features/prison-reform-risk-assessment/

[6] D. Lazer, The rise of the social algorithm, Science 348, 1090-1091 (2015).

[7] E. Medina, Cybernetic Revolutionaries: Technology and Politics in Allende's Chile, MIT Press (2014); S. Harris, The social laboratory, Foreign Policy (July 29, 2014), see http://foreignpolicy.com/2014/07/29/the-social-laboratory/

[8] A. Lobe, Google möchte den Staat neu programmieren [Google wants to re-program the state], Frankfurter Allgemeine Zeitung (October 14, 2015), see http://www.faz.net/aktuell/feuilleton/medien/google-gruendet-in-den-usa-government-innovaton-lab-13852715.html

[9] K. Noyes, Forget Trump and Clinton: IBM's Watson is running for president, PCWorld (February 8, 2016), see http://www.pcworld.com/article/3031137/forget-trump-and-clinton-ibms-watson-is-running-for-president.html

[10] S. Dörner, In 29 Jahren sind die Probleme der Menschheit gelöst [In 29 year the problems of humanity will be solved], Die Welt (February 14, 2016), see http://m.welt.de/wirtschaft/webwelt/article152198869/In-29-Jahren-sind-die-Probleme-der-Menschheit-geloest.html

[11] D. Helbing, Thinking Ahead: Essays on Big Data, Digital Revolution, and Participatory Market Society (Springer, 2015), Chapter 10: Big Data Society.

[12] J. van den Hoven, The use of normative theories in computer ethics, in: The Cambridge Handbook of Information and Computer Ethics, edited by L. Floridi (Cambridge University Press, 2010), pp. 59-76

[13] J. van den Hoven, Engineering and the problem of moral overload, Sci. Eng. Ethics (2011), see http://link.springer.com/article/10.1007/s11948-011-9277-z#/page-1

[14] D. Helbing and E. Pournaras, Build digital democracy, Nature 527, 33-34 (2015).

[15] L. Lessig, Code and other Laws of Cyberspace, Version 2.0 (Basic Books, 2006).