ETHICS FOR TIMES OF CRISIS

by Jan Nagler,1,2 Jeroen van den Hoven³, and Dirk Helbing1,3,4

(The Pdf of this article can be downloaded here)

Affiliations: 

1. Computational Social Science, Department of Humanities, Social and Political Sciences, ETH Zurich, Clausiusstrasse 50, CH 8092 Zurich, Switzerland 
2. Computational Physics for Engineering Materials, IfB, ETH Zurich, Wolfgang-Pauli-Strasse 27, CH 8093 Zurich, Switzerland
3.  TU Delft, The Netherlands

4.  Complexity Science Hub, Vienna, Austria
   
   

E-mail addresses:  

jnagler@ethz.ch

m.j.vandenhoven@tudelft.nl

dhelbing@ethz.ch

What will happen in a crisis when Artificial Intelligence systems will decide about increasingly many issues, including life and death? 

Will a Citizen Score based on Big Data determine our chances of survival? 

How should autonomous systems make decisions that are ethically aligned with what is morally required from humans? 

We argue that in times of permanent crisis the dominant approach should be innovation instead of optimization.
 

These days, everyone is talking about artificial intelligence (AI), robots and self-driving cars. We absolutely agree that these technologies have promising applications and perspectives. So, why then are people like Elon Musk warning us that AI may pose the biggest existential threat to humanity? (1) And what does this have to do with mundane things such as autonomous cars, if anything at all?
 

Autonomous cars will certainly be involved in accidents where people may die. Then the question is (2-4) when fatalities cannot be prevented – should an autonomous vehicle be programmed to run over a crowd of people on the street, swerve into a smaller group of pedestrians on the walkway or sacrifice the lives of the car passengers by ramming into a concrete wall? Should particular kinds of people be privileged, giving them higher chances of survival? For example, should luxury cars be allowed to offer a higher degree of self-protection as compared to cars in the lower price segment, to create incentives to buy a more expensive car? Would this be ethically justified, given that expensive cars cause more accidents and already impose higher risks on others (5)?

These are the kind of ethical dilemmas that are now frequently discussed. But there is a far bigger problem nobody is openly talking about: In the non-sustainable world we are living in, when there are not enough resources left for everyone, will autonomous systems be used to decide about life and death? If yes, how should they decide? Therefore, when we talk about ethical principles governing how autonomous vehicles deal with matters of life and death, one should always keep in mind the implications for scaled up and generic applications in times of crises. Or to put it in a Kantian form: what if the maxims or policies of these types of machines were to be become the universal principles for all machines?

AI systems knowing “who is who”

With better sensor and video technologies and powerful information systems, artificial intelligence (AI) systems are increasingly capable of distinguishing between one person and many, a child and an elderly person, an average person and a famous politician, a white person and a person of colour, a person with a job or without, a rich and a poor person, a convicted criminal and a saint, a healthy person and one who may die soon, a person with health or life insurance and without?

Should people with higher status or life expectancy be protected, because they may contribute more to society? Should a Citizen Score decide, which represents the value of a person from the point of view of the government, as it is currently being tested in China in other areas of life? (6) Should a person who pays a higher insurance premium have a higher chance of survival and others be sacrificed? This sounds like a profitable business model, but it would fundamentally contradict the principle of equality and human dignity, on which the United Nations’ Universal Declaration of Human Rights is built.

Criteria such as health, age, or social status are not suitable criteria to decide who should come to harm, live or die, not even from a narrow utilitarian perspective. Recall that Kant, the father of Enlightenment, who inspired modern democratic constitutions, wrote his masterpieces at old age. Van Gogh had a very low social status during his lifetime. Mozart died poor. Beethoven was almost deaf when he wrote his 9th symphony. Degas and Toulouse-Lautrec were handicapped and Monet had impaired sight, but they became three of the most important painters of Impressionism. These individuals have created some of the greatest cultural achievements in the history of humankind. Whatever measure is taken to distinguish the value of people, there are always examples that show the inappropriateness of such a measure.
 

Utilitarian thinking can be inappropriate

The current rulings of many constitutional courts and ethical committees largely agree that people should not be valued differently, but share a common humanity and human dignity. This is also a lesson learned from the history of fascism and the Holocaust. Utilitarian “optimization” appears to be highly immoral, if it intentionally exposes different kinds of people to different life-threatening risks. For example, the new Hippocratic oath (7) requires doctors to swear: “I will not permit considerations of age, disease or disability, creed, ethnic origin, gender, nationality, political affiliation, race, sexual orientation, social standing or any other factor to intervene between my duty and my patient.” Perhaps, given that “code is law”, we should require a similar oath from computer scientists and software engineers to ensure design for values (8).

Also note that autonomous systems based on utilitarian principles would be easily exploitable – both by criminals and authorities. Deterministic decisions could be successfully instrumentalized to harm people in cases of manipulation or hacking. For example, someone may jump on the street and force a deterministically deciding vehicle to crash into a concrete wall, or someone may trigger or hack the vehicle sensors to trick it into a dangerous manoeuvre that might put passengers at risk. Other drivers may anticipate the safety behaviour of the type of car you are driving and exploit its response repertoire for personal advantage. In such cases, a probabilistic decision rule would make it less likely that an autonomous system could be successfully instrumentalized against other people.

Overall, if ethical dilemmas cannot be avoided, decisions should be randomized, giving each person the same weight. A society with ubiquitous AI requires a framework that is impartial, as proposed by the Harvard political philosopher John Rawls with his concept of “the veil of ignorance” (9). This implies that, in deciding about the basic normative principles of a society, one should ignore properties that could be tailored to serve self-interests. This, again, suggests that humans should not be treated differently in a critical situation and solutions based on utilitarian grounds should be rejected.
 

Killing algorithms are not science-fiction

Very soon, the ethics of autonomous systems may affect all of our lives every day. In turbulent times, as we may encounter them in an unsustainable world, decisions about life and death may become commonplace. Today, robocops are being tested, drones are being used to kill dissidents, and a number of autonomous weapons are in the making (10, 11). Some experts even think about AI-based euthanasia (12) and the use of palliative means. Soon, computer-controlled implants may be used to release drugs to our bodies, but such devices would be vulnerable to hacking and may cause overdoses (13, 14).

Let us assume or a moment, one would apply the Citizen Score, as it is currently tested in China, or the United Kingdom’s KARMA POLICE program (15) to make decisions about life and death – saving those who have a higher score. Such a “digital judgment day” approach would create one of the most serious moral hazards imaginable. “The elite”, i.e. the people with the highest scores, would always have the lowest risks and the greatest opportunities. Therefore, why should they make a serious effort to improve the opportunities and risks of all the others, if it will not improve their own lives? In contrast, an unbiased probabilistic decision rule in combination with a fair veil of ignorance, would put everyone at the same level of risk, and hence everybody would have an incentive to reduce the number of ethical dilemmas as much as possible. 

Humanity has a moral obligation to prevent the occurrence of ethical dilemmas, i.e. choice situations where one cannot fulfil all moral norms at the same time. Furthermore, if ethical dilemmas occur nevertheless, there is an obligation to transform them into situations which expand the set of obligations one can satisfy, whenever possible. A Citizen-Score-based system would certainly miss this goal. It provides a framework suggesting we can fulfill our moral duties by optimization that weighs and counts lives and deaths, and that such a decision can be automated and dealt with by a machine.

To minimize the number of critical situations, we do not only need the best use of human and artificial intelligence, but creativity as well. In a crisis, innovation may be more important than optimization. To successfully address the sustainability challenges of our planet, we may have to fundamentally change the monetary, financial, and economic system, or even the organization of society altogether (16,17). Given the limitations of optimization, Citizen-Scores-based systems and utility maximization, we should spend much more resources on systemic innovation, e.g. on participatory resilience and City Olympics, where cities all over the world and the regions around them would regularly compete for the best environmental-friendly, energy-efficient, resource-saving, crisis-proof, and peace-promoting solutions (18). Such approaches could dramatically improve the future prospects of humanity within a short period of time.

References

1. S. Gibbs, The Guardian, Monday 27 October 2014, available at https://www.theguardian.com/technology/2014/oct/27/elon-musk-artificial-intelligence-ai-biggest-existential-threat
2. B. Deng, Nature 523, 24-26 (2015). 
 
3. J.-F. Bonnefon et al., Science 352, 1573-1576 (2016). 
4. D. Leben, Ethics Inf. Technol., 19:107-115 (2017). 
5. The Telegraph, Nov 18, 2015 (available at http://www.telegraph.co.uk/finance/personalfinance/insurance/motorinsurance/11993627/Its-official-drivers-of-luxury-cars-cause-more-accidents-insurers-say.html). 
6. D. Storm, ACLU: Orwellian Citizen Score, China’s credit score system, is a warning for Americans. Computerworld (7 October 2015); available at http://go.nature.com/3pq8b4, and at http://www.independent.co.uk/news/world/asia/china-surveillance-big-data-score-censorship-a7375221.html 
7. https://www.wma.net/policies-post/wma-declaration-of-geneva/ 
8. J. van den Hoven, P. E. Vermaas, I. van de Poel, Handbook of Ethics, Values, and Technological Design: Sources, Theory, Values and Application Domains, Springer (2015). 
9. K. B. Rasmussen, Philos. Stud. 159:205-218 (2012). 
10. S. Russell, S. Hauert, R. Altman, and M. Veloso, Nature 521, 415-418 (2015). 
11. IJCAI conference, July 28 (2015). Open letter initiative; available at https://futureoflife.org/open-letter-autonomous-weapons/ 
12. F. Hamburg, Een computermodel voor het ondersteunen van euthanasiebeslissingen (E.M. Meijers Reeks) 
13. Hackers remotely kill jeep highway, Wired, July 24, 2015; available at https://www.wired.com/2015/07/hackers-remotely- kill-jeep-highway/ 
14. Hackers reveal nasty new car attacks, Forbes, July 24, 2013; available at https://www.forbes.com/sites/andygreenberg/2013/ 07/24/hackers-reveal-nasty-new-car-attacks-with-me-behind-the-wheel-video/#45c73d7b228c.
15. https://www.theverge.com/2015/9/25/9397119/gchq-karma-police-web-surveillance http://www.dailymail.co.uk/news/article-3249568/GCHQ-spooks-spied-internet-user-operation-called-Karma-Police-according-leaked-documents.html 
16. D. Helbing, Nature 497, 51-59 (2013). 
17. D. Helbing and E. Pournaras, Nature 527, 33-34 (2015) 
18. https://www.theglobalist.com/technology-big-data-artificial-intelligence-future-peace-rooms/

A new, global fascism, based on mass surveillance is on the rise

By Dirk Helbing
 

Last weekend, I spoke to business leaders at the Petersberger Dialogue in the Villa Hammerschmidt. I warned that a new, global fascism, based on mass surveillance, appears to be on the rise.

The German talk can be viewed here

 

Petersburger Dialogue

A similar, English video can be found here

Recently, an increasing number of experts and intellectuals have warned of an emerging technological totalitarianism, enabled by what some people call “surveillance capitalism.” 

A recent, much discussed contribution in Scientific American raised the question “Will democracy survive Big Data and Artificial Intelligence”, pointed to the dangers of new kinds of behavioural manipulation. Even Cass Sunstein, one of the fathers of “Nudging”, has recently issued grave words of warning.

Today, secret services and Big Data companies possess much more data about us than were needed to run totalitarian states in the past. It is unlikely that such power will not be misused at some point in time. 

Moreover, it becomes increasingly evident that all the features of fascism have been implemented digitally or are currently being implemented. They could be used on a society-wide scale at any time. 

This includes:

• mass surveillance, 
• unethical experiments with humans,  
• social engineering,  
• forced conformity (“Gleichschaltung”),
•  propaganda and censorship,  
• “benevolent” dictatorship,
•  (predictive) policing,  
• different valuation of people,  
• relativity of human rights,  
• and, it seems, even euthanasia for the expected times of crisis in our unsustainable world.
  

The signs are clear. We are faced with the emergence of a new kind of totalitarianism of global dimensions that must be stopped immediately. “An emergency operation is inevitable, if we want to save democracy, freedom, and human dignity,” I warned. “Arguments such as terrorism, cyber threats and climate change have been used to undermine our privacy, our rights, and our democracy.”

The emergence of mass surveillance after 9/11, enabled by the Patriot Act and other laws, has led to the incremental erosion of liberties and human rights. Since the Snowden revelations, we know that there is mass surveillance of billions of people around the world. But most of us still have no idea how pervasive it is, and how it may influence their lives in future. 

Billions of dollars have been spent on mass surveillance tools of secret services to hack our computers, smartphones, smart TVs and smart cars. The estimated amount of data collected about us every day ranges from millions of numbers to Gigabytes of data. As a result, we have ended up with the digital tools for a data-driven, AI-based so-called “benevolent” dictatorship, where big businesses and the state determine “what is best for us.” Moreover, we have seen that democracies in Hungary, Poland, Turkey, and elsewhere have already undergone transformations towards more autocratic regimes.

Citizens are being targeted, their data collected and consolidated. This is used to create a near complete profile of each person, their nature, habits and preferences. Each profile can contain thousands of specifiers. These digital doubles can be used to make thousands of computer experiments with our virtual self to find out how our thinking and behaviour can be manipulated.

More specifically, in today’s attention economy, our personal data is being applied to customize information such that it will influence our attention, emotions, opinions, decisions, and behaviours – often subconsciously – by a technique called big nudging or neuro-marketing. This ranges from steering our consumption behaviour to manipulating voting behaviour in elections.

In the wrong hands, the misuse of surveillance-based personal data will have catastrophic consequences for us and for society. In an explicitly or implicitly totalitarian state, this kind of information could be used to predict and identify those people who don’t agree with certain government policies and sanction them even before they can exercise their democratic rights.

The British secret service, for example, runs a program called Karma Police, which shows where our societies are heading. This Citizen Score, which is currently also tested in China, may be used to run an entirely new kind of autocratic society, or even police state. According to plans, the Citizen Score would determine the level of access to facilities, products and services. We would be scored or penalised according to our behaviours. Reading critical news or having the “wrong” kinds of social ties, for example, would get you minus points.

To counter this danger of a digital totalitarian state, I strongly suggest that we should:

• ensure a democratic framework of use for powerful cyberinfrastructures, 
• ensure scientific use by interdisciplinary teams, considering multiple perspectives,  
• ensure ethical use considering human rights and human dignity,  
• ensure transparency,  
• ensure cyber-security (by decentralization etc.),
•  prevent misuse (by effective laws/remedies and in-built self-destruction mechanisms in case of serious misuse, malfunction, or political power grabs),  
• ensure informational self-determination (e.g. with a Personal Data Store),  
• ensure informational self-determination by giving citizens the right to opt-out,  
• turn war rooms into peace rooms, as described in a recent proposal by myself and Peter Seele.
  

The purpose of these video message is to create and spread public awareness of the insidious extent that individual personal data is being used today.

The door is wide open for global fascism to take hold, unless we take action now.

I hope that you will pay attention to these well-founded concerns regarding the rise of a technological totalitarianism, potentially on a global scale. My presentation offers much evidence. 

If this is a topic of interest that you would like to investigate further, you are kindly invited to contact me at dhelbing@ethz.ch. 

An alternative vision of a better, participatory, digital future that everyone can benefit from, is offered here: A Digital World to Thrive in and here: and here: https://www.theglobalist.com/author/dirk-helbing/

Thank you for reading.

"Switzerland may be the only country to stay democratic"

These days, political scientist Parag Khanna doesn’t get tired to advertise “technocracy” as the best form of government.[1] He is not the only one. Internet pioneers in the Silicon Valley and elsewhere have often declared democracy as “outdated technology”.[2] Peter Thiel proclaims a “deadly race between politics and technology”.[3] And Google’s Larry Page complained that “[t]here’s many, many exciting and important things you could do you just can’t do because they are illegal”.[4] This shows a concerning disrespect of our laws.

Politics has promoted such thinking as well. In the wake of 9/11 and the Patriot Act, social engineers such as George W. Bush and Tony Blair have created a surveillance society, where a state of emergency was always around the corner, seemingly justifying the restriction of human rights and liberties step by step. Since then, democracies have undergone worrying transformations towards more autocratic regimes in Hungary, Poland, and Turkey. In the USA, UK and France, a transition is on the way as well. Last but not least, Austria, Switzerland, The Netherlands and Germany have also started to tumble. After one of my talks, someone with insider knowledge even told me: “Switzerland may be the only country to stay democratic.” He had concrete reasons to be concerned. And in Switzerland too, some have suggested the Chinese system would be more attractive…

Overloaded and tired by the flood of revelations, many people have not realized that we have ended up with technologies and laws, which parallel those described in George Orwell’s 1984 or Aldous Huxley’s Brave New World. For example, just a few days ago, a new mass surveillance law (the large-scale use of “Staatstrojaner” federal hacking software) has passed the German Bundestag with a trick.[5] Little by little, we have ended up with the digital tools for a data-driven, AI-based so-called “benevolent dictatorship”.[6] This technocratic state will be totalitarian for sure, but will it also be “benevolent”? How may it look like, according to the evidence so far?

Big Brother: Since the Snowden revelations,[7] we know that there is mass surveillance of billions of people around the world, but most of us still have no idea, how pervasive it is. For example, all of our clicks in the Internet are reported by cookies, and there are a lot more digital traces of our everyday activities, which are also recorded. In Great Britain, more than 40 government institutions have access to the click history, even if it was regularly deleted.[8] Recently, Wikileaks “Vault 7” revealed that the CIA – just one of more than 15 secret services in the USA – had spent many billion dollars on mass surveillance tools to hack most kinds of computers, smartphones, smart TVs and smart cars – and that it had lost control over them, i.e. dual use of hacking tools by others was now a serious problem.[9]

Surveillance Capitalism:[10] It has become known that all citizens are being targeted to create personal profiles of them. This profiling covers thousands of specifiers per person (including name, birth day, age, income, address, religious denomination, sexual preferences and personal tastes, as well as health-related data, which is being sold to other companies). You have probably no idea of how much data has been secretly collected about you by corporations (so-called “Big Others”), and what is being done with it. The data give a much more detailed picture than the files that secret services used to run totalitarian systems in the past. You would be surprised to learn what companies such as Google, Facebook, Recorded Future, Acxiom, Palantir, and others know about you – or may infer about you from your personal data. Just have a look at your Crystal Knows profile to get a first idea.[11]

Big Nudging:[12] In today’s “attention economy”,[13] our personal data is being used to personalize information such that it will influence our attention, emotions, opinions, decisions, and behaviors – often in a subconscious way. This ranges from steering our consumption behavior with so-called “Neuro-Marketing” methods[14] up to manipulating our voting behavior during elections, as it happened during the Brexit vote and US election in 2016.[15] When combined with Social Bots – armies of robotic bloggers – very effective propaganda tools result. As the debate about Fake News and the post-fact society shows, many people are falling prey to these new technologies, some of which can even edit Web contents[16] (i.e. manipulate history). Populism, polarization, and fragmentation of societies have spread and threaten social peace. However, the censorship laws and “ministries of truth” to fight the problem are probably even a greater danger to democracies.

World (or War) Simulator: The personal data collected about each of us is being fed into a machine learning algorithm that learns how we behave. It produces a “digital double” – something like a personal avatar. These digital doubles are then used, on the one hand, for global “war games” – massive computer simulations such as Sentient World,[17] which try to predict our future. On the other hand, these digital doubles can also be used to test how we would respond to certain information stimuli. This is of interest for the next generation of “predictive policing”,[18] which would work pretty much like “Minority Report”. For example, it would be possible to find out beforehand, who is likely to protest against a shutdown of the freedom of press, or of democracy altogether.

Citizen Score: All the data collected about you are boiled down to a single number, the Citizen Score, representing your value from the point of view of those in power. This system will be a future governance tool in China, where it is currently being tested.[19] However, there is a similar system called “Karma Police” in Great Britain,[20] and most likely everywhere else. The system gives plus or minus points for everything you do, for the friends you have, the links you click on the Internet, the movies you watch, and the music you hear. The overall score will then decide about interest rates for loans, the job you may get, travel visa to other countries, and (conditions of) access to products and services.

Cashless society:[21] Once people would have a unique identity chip,[22] the Citizen Score could also be used as basis of a cashless society. In particular during future resource shortages, the Citizen Score could decide about the access to resources such as long-distance travel, medicine or meat. While all of this will be claimed to comply with principles of fairness and justice, in the future the Citizen Score might decide about life or death. But even if resources were sufficient for everyone – the competition for high Citizen Scores and good services would turn many creative individuals into submissive subjects.

“Benevolent dictatorship”: This comprises all of the above elements. The World Simulator would be used to come up with a grand plan for the world, which would imply specific roles for every individual. These would then be suggested to us through our smartphones or similar devices by means of personalized information (“Big Nudging”). If we would execute the suggested actions, this would earn us plus points, otherwise we would be punished by minus points. The ideological justification for such a system is that a “superintelligent” AI system would know better what is good for us, and therefore impose it on us (this is sometimes called the “Big Mother” society). In other words, people – currently free decision-makers – would eventually become something like an “output device” of a digital command economy. Freedom and self-determination, human rights and democracy would be gone.

Technological and Legal Assessment

Some technology gurus have said that our current system has created “wealth, health and happiness for billions of people … but now we want to try something new”.[23] Unfortunately, this new solution has not been properly tested. One would think that it would be the right approach to start with the reorganization of a company, then with that of a city (e.g. San Francisco), then with the reorganization of a state (such as California), subsequently of a country (e.g. USA), and then of a continent (e.g. America), before one rolls out the system in the entire world. So far, however, no “smart city” has made it into the top 10 hit parade of most livable cities, and it has recently been concluded that the technocratic Smart Cities approaches have largely failed.[24] In other words, there is no proof that anyone has managed yet to invent anything that comes close to a digital paradise on Earth. On the contrary, we have recently seen problems such as hate speech, fake news, and cybercrime abound, producing much harm to our societies.

Moreover, the most fundamental problem is unsolved, namely how to determine the right goal function.[25] Should it be the gross domestic product or sustainability, power or peace, happiness or life expectancy, or something else? If we chose power, for example, the AI system would learn to make us addicted to information and dependent on it.[26] On the one hand, therefore, Apple boss Tim Cook distanced himself from manipulating people with personalized information.[27] Elon Musk warned of superintelligent systems as “our greatest existential threat”, potentially more dangerous than nuclear weapons.[28] Consequently, he spent a billion dollars to launch the OpenAI project to ensure AI will be “an extension of individual human wills and, in the spirit of liberty, as broadly and evenly distributed as possible.”[29]

On the other hand, the algorithmic personalization approaches by Google and others were blamed to discriminate women[30] and people of color.[31] Facebook came under pressure, as its platform seemed to contribute to the spread of extremism,[32] and Microsoft had to take down their chatbot “Tay” from the Web, as it started to produce Nazi propaganda.[33] Big Nudging has been heavily criticized in connection with the Brexit and US election campaigns.[34] The Citizen Score was condemned by The Economist and other newspapers for its totalitarian nature.[35] An ethics committee established by the German government decided that AI systems such as autonomous cars were not allowed to take life-and-death decisions, which would advantage one kind of people over another one.[36] Moreover, the US Supreme Court has declared that hate speech does not justify censorship.[37] And the United Nations Human Rights Convention demands that our protect our privacy be protected.[38] Last but not least, the European Court of Justice decided that mass data retention (Vorratsdatenspeicherung) without sufficient reasons is illegal.[39] So, the situation is clear: all of the above developments should not happen – and there is no evidence that they would create a better society. So, why is it still happening then?

The Sustainability Agenda – A Totalitarian Agenda Wrapped in Nice Words?

In the year 2015, some important decisions were taken about the future of the world. Opened by Pope Francis on September 25, the United Nations decided to pursue very ambitious Global Sustainability Goals with its Agenda 2030.[40] Strong institutions would be built to make the world sustainable by 2030, within just 15 years! On September 25, there were also a number of other notable events: Switzerland decided about a new surveillance law (Nachrichtendienstgesetz).[41] The Intercept reported about the Karma Police program,[42] and Kai Schlieter’s book “Die Herrschaftsformel” [The formula to rule the world] appeared with a delay.[43]

The Paris Climate Deal[44] followed up the Agenda 2030 and the Agenda 21. It decided concrete actions. Carbon dioxide emissions, for example, will have to be reduced by 40 percent at least. This shall be achieved by carbon trading,[45] which will create a one trillion dollar market, to be paid by the consumers. The aim to charge everyone in a fair way, may serve as an excuse for mass surveillance. One could say this requires detailed data about who is travelling how much and in what ways. Who is consuming how much, and what are the Carbon Dioxide emissions generated by these products? Moreover, carbon trading will probably compensate oil and gas companies for their expected losses, even though they have driven the planet to the edge and also suppressed alternative energy technologies. All of this is obviously neither moral nor legitimate – and that is probably the reason why the public has not be informed about the detailed plans in a transparent way, as it should be.

The Paris Climate Deal was made under the impression of the terror attack on Friday, November 13, 2015,[46] which initiated martial law and restricted constitutional rights – until today. For example, public demonstrations can now be forbidden, and private homes can be searched by police without any evidence of wrongdoing. Such measures undermining democracy have been seen also in many other countries such as Great Britain or Germany.

All of these developments become understandable if you read the “Limit to Growth” study commissioned by the Club of Rome[47] or the Global 2000 report[48] commissioned by US President Jimmy Carter, their updates, and follow-up studies. These reports try to project the future of the Earth by means of computer simulations. The shocking result of these studies was that, given the resource constraints of our planet, an economic and population collapse would be unavoidable (if we do not change our financial and socio-economic system[49] in a way that promotes a circular and sharing economy[50] – this would be the right solution!).

Source: How Many Earths? (September 27, 2015) https://thecomingmigrations.com/2015/09/27/how-many-earths/

Today’s conventional view is that the world is overpopulated and overusing its renewable resources by at least 40 percent (see graphic above). As a consequence, about one third of the world’s population would die of an unnatural death in this century (see graphic below). This would be more people than currently live in the USA, Europe, Russia and China together.

Source: http://www.smithsonianmag.com/science-nature/looking-back-on-the-limits-of-growth-125269840/#HOSPivUzCMQMbY1v.99. Chart Sources: Meadows, D.H., Meadows, D.L., Randers, J. and Behrens III, W.W. (1972) (Linda Eckstein)

From this perspective, it appears logical that governments would prepare to ration scarce resources and distribute them fairly, according to personal merit, as reflected by the Citizen Score. It also becomes understandable that preparations were taken to suppress civil unrest, ranging from mass surveillance to armed police forces. Of course, there would also be preparations for times, when resources would not suffice anymore for everyone. Therefore, the question was raised: could AI systems be used to take decisions over life and death[51] – something one could probably call digital “judgment day”.[52] Now, you probably start to understand Elon Musk’s concerns about superintelligent systems. I also think that our societies need to take a completely different path to sustainability – based on a new financial, money and economic system. I will describe this in forth-coming contributions. 

[1] Die beste Regierungsform ist die direkte Technokratie, Tagesanzeiger (Jun 9, 2017), see http://www.tagesanzeiger.ch/ausland/standard/freiheit-ist-nicht-alles/story/25180340, https://www.infosperber.ch/Artikel/FreiheitRecht/Politologe-will-Schweiz-und-Singapur-mischen


[2] „Sie [die Demokratie] hat Reichtum, Gesundheit und Glück für Milliarden Menschen auf der ganzen Welt gebracht. Aber jetzt wollen wir etwas Neues ausprobieren und dazu haben wir hier keine Chance.“ See http://www.3sat.de/page/?source=/kulturzeit/themen/176692/index.html, http://www.theeuropean.de/aleksandra-sowa--2/12369-die-schoene-neue-cyberwelt-und-die-alte-demokratie


[3] Peter Thiel is trying to save the world: The apocalyptic theory behind his actions, Business Insider (December 8, 2016), see http://uk.businessinsider.com/peter-thiel-is-trying-to-save-the-world-2016-12


[4] Google CEO Larry Page Wants A Totally Separate World Where Tech Companies Can Conduct Experiments On People, Business Insider (May 16, 2013), see http://www.businessinsider.com/google-ceo-larry-page-wants-a-place-for-experiments-2013-5


[5] Überwachung: Koalition macht Staatstrojaner zum polizeilichen Alltagswerkzeug (Heise Online, June 20, 2017) https://www.heise.de/newsticker/meldung/Ueberwachung-Koalition-macht-Staatstrojaner-zum-polizeilichen-Alltagswerkzeug-3748014.html


[6] D. Helbing, Dictatorship 4.0: How the digital revolution threatens our freedom - and what our alternatives are, see https://www.researchgate.net/publication/317533191_Dictatorship_40_How_the_digital_revolution_threatens_our_freedom_-_and_what_our_alternatives_are; The Dream to Control the World – and Why It Is Failing, see https://www.researchgate.net/publication/316989750_The_Dream_to_Control_the_World_-_and_Why_it_is_Failing


[7] for an overview see https://www.theguardian.com/us-news/the-nsa-files


[8] Revealed: The 48 organisations that can see your entire online browsing history, even if you delete it, Mail Online (Nov 25, 2016). Retrieved from http://www.dailymail.co.uk/sciencetech/article-3971214/The-48-organisations-entire-online-browsing-history-delete-it.html ; Browsers nix add-on after Web of Trust is caught selling users’ browsing histories, The Register (Nov 7, 2016). Retrieved from https://www.theregister.co.uk/2016/11/07/browsers_ban_web_of_trust_addon_after_biz_is_caught_selling_its_users_browsing_histories/ ; Apple iCloud Hoards 'Deleted' Browser History Going Back More Than A Year, Forbes (Feb 9, 2017). Retrieved from http://www.forbes.com/sites/thomasbrewster/2017/02/09/apple-safari-web-history-deleted-stored-icloud/#632df5322439


[9] see https://en.wikipedia.org/wiki/Vault_7 and https://wikileaks.org/vault7/ for details


[10] S. Zuboff, Big Other: Surveillance Capitalism and the Prospects of an Information Civilization, Journal of Information Technology 30, 75-89 (2015).


[11] see https://www.crystalknows.com


[12] see the corresponding information box in https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/


[13] T.H. Davenport and J.C. Beck, Attention Economy: Understanding the New Currency of Business (Harvard Business Review, 2002).


[14] H.M. Sola and M. Econ. Neuromarketing Armoury (Createspace, 2017), see https://www.amazon.de/dp/1543261892/; L. Zurawicki, Neuromarketing: Exploring the Brain of the Consumer (Springer, 2016).


[15] The Rise of the Weaponized AI Propaganda Machine, https://medium.com/join-scout/the-rise-of-the-weaponized-ai-propaganda-machine-86dac61668b; also watch this video of Alexander Nix talking about Psychographics: https://www.youtube.com/watch?v=n8Dd5aVXLCc


[16] Nothing is real: How German scientists control Putin's face, https://www.youtube.com/watch?v=ttGUiwfTYvg; http://www.graphics.stanford.edu/~niessner/thies2016face.html; Adobe VoCo ‘Photoshop for voice’ causes concern, http://www.bbc.com/news/technology-37899902; https://www.youtube.com/watch?v=I3l4XLZ59iw


[17] Sentient World: War games on the grandest scale. The Register (June 23, 2007) see https://www.theregister.co.uk/2007/06/23/sentient_worlds/; see also https://en.wikipedia.org/wiki/Synthetic_Environment_for_Analysis_and_Simulations and https://www.youtube.com/watch?v=MVrBZevJY3U


[18] W.L. Perry et al., Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations (RAND Corporation, 2013); see also https://en.wikipedia.org/wiki/Predictive_policing


[19] China’s Nightmarish Citizen Scores Are A Warning For Americans, https://www.aclu.org/blog/free-future/chinas-nightmarish-citizen-scores-are-warning-americans


[20] Karma Police: GCHQ spooks spied on every web user ever, The Register (Sep. 25, 2015), https://www.theregister.co.uk/2015/09/25/gchq_tracked_web_browsing_habits_karma_police/


[21] Cashless society getting closer, survey finds, see http://www.reuters.com/article/us-global-economy-cash-idUSKBN17S001


[22] Human Microchipping: An Unbiased Look at the Pros and Cons, https://medium.freecodecamp.com/human-microchipping-an-unbiased-look-at-the-pros-and-cons-ba8f979ebd96


[23] R. Hencken, in: Mikrogesellschaften. Hat die Demokratie ausgedient? (Capriccio, 2014). Video, veröffentlicht am 15.5.2014. Autor: J. Gaertner. München: Bayerischer Rundfunk.


[24] Disrupting cities through technology, Wilton Park (March 17-19, 2016), see https://www.wiltonpark.org.uk/wp-content/uploads/WP1449-Report.pdf


[25] D. Helbing and E. Pournaras, Build Digital Democracy, Nature 527, 33-34 (2015).


[26] Smartphone Addiction, Psychology Today (Jul 25, 2013) https://www.psychologytoday.com/blog/reading-between-the-headlines/201307/smartphone-addiction; see also http://www.dailymail.co.uk/news/article-2174024/Facebook-creating-generation-gambling-addicts-sites-Las-Vegas-style-games.html and http://www.business2community.com/facebook/7-ways-facebook-keeps-addicted-apply-lessons-products-01851945


[27] http://www.dailymail.co.uk/sciencetech/article-3108457/Apple-boss-Tim-Cook-launches-blistering-attack-Google-Facebook-claims-people-fundamental-right-privacy.html


[28] http://observer.com/2015/08/stephen-hawking-elon-musk-and-bill-gates-warn-about-artificial-intelligence/


[29] https://blog.openai.com/introducing-openai/


[30] http://www.techtimes.com/articles/67268/20150709/study-suggests-google-algorithms-discriminate-against-women.htm


[31] https://www.washingtonpost.com/news/morning-mix/wp/2016/06/10/google-faulted-for-racial-bias-in-image-search-results-for-black-teenagers/?utm_term=.d500edd42210; see also https://www.theguardian.com/commentisfree/2016/jun/26/algorithms-racial-bias-offenders-florida for the more general problem


[32] Beihilfe zur Volksverhetzung bei Facebook: Staatsanwaltschaft München ermittelt gegen Mark Zuckerberg, Der Tagesspiegel (Nov. 4, 2016) http://www.tagesspiegel.de/politik/beihilfe-zur-volksverhetzung-bei-facebook-staatsanwaltschaft-muenchen-ermittelt-gegen-mark-zuckerberg/14796574.html


[33] Microsoft terminates its Tay AI chatbot after she turns into a Nazi. Ars Technica (March 24, 2016) https://arstechnica.com/information-technology/2016/03/microsoft-terminates-its-tay-ai-chatbot-after-she-turns-into-a-nazi/


[34] https://www.theguardian.com/technology/2017/may/07/the-great-british-brexit-robbery-hijacked-democracy


[35] Big Data, meet Big Brother: China invents the digital totalitarian state. The Economist (Dec 17, 2016) see http://www.economist.com/news/briefing/21711902-worrying-implications-its-social-credit-project-china-invents-digital-totalitarian


[36] Ethikkommission warnt vor Totalüberwachung des Menschen, Die ZEIT (June 20, 2017) http://www.zeit.de/mobilitaet/2017-06/autonomes-fahren-totalueberwachung-ethik-kommission


[37] Supreme Court unanimously reaffirms: There is no ‘hate speech’ exception to the First Amendment, https://www.washingtonpost.com/news/volokh-conspiracy/wp/2017/06/19/supreme-court-unanimously-reaffirms-there-is-no-hate-speech-exception-to-the-first-amendment/?utm_term=.f88021163948


[38] Universal Declaration of Human Rights, http://www.un.org/en/universal-declaration-human-rights/


[39] Europäischer Gerichtshof: It's the end of Vorratsdatenspeicherung as we know it, Die ZEIT (Dec. 21, 2016) http://www.zeit.de/digital/datenschutz/2016-12/europaeischer-gerichtshof-vorratsdatenspeicherung-urteil


[40] http://www.un.org/sustainabledevelopment/blog/tag/pope-francis/


[41] https://www.admin.ch/gov/de/start/dokumentation/abstimmungen/20160925/nachrichtendienstgesetz.html


[42] https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/


[43] K. Schlieter, Die Herrschaftsformel: Wie Künstliche Intelligenz uns berechnet, steuert, und unser Leben verändert (Westend, 2015).


[44] Paris Agreement, https://en.wikipedia.org/wiki/Paris_Agreement


[45] Carbon Trading: The World’s Next Biggest Market, http://www.greenchipstocks.com/report/carbon-trading-the-worlds-next-biggest-market/107; The EU Emissions Trading System, https://ec.europa.eu/clima/policies/ets_en


[46] November 13 Paris attacks, https://en.wikipedia.org/wiki/November_2015_Paris_attacks


[47] The Limits to Growth, https://www.amazon.de/Limits-Growth-Project-Predicament-Mankind/dp/087663918X/, see also the 30-year update, https://www.amazon.de/Limits-Growth-Donella-H-Meadows/dp/193149858X/


[48] Editions of the Global 2000 Report to the [US] President, see http://www.geraldbarney.com/G2000Page.html


[49] D. Helbing, Why we need democracy 2.0 and capitalism 2.0 to survive, published in Jusletter IT, https://www.researchgate.net/publication/303684254_Why_We_Need_Democracy_20_and_Capitalism_20_To_Survive


[50] City Olympics, a decentralized, participatory, socio-ecological finance system (finance 4.0+), democratic capitalism and digital democracy would be solutions to make the world sustainable by 2030 in a democratic way, see http://futurict.blogspot.de/2017/06/propositions-on-perspective-global.html and http://futurict.blogspot.de/2017/06/digitization-20-new-game-begins.html


[51] Een computermodel voor het ondersteunen van euthanasiebeslissingen, http://www.maklu.be/MakluEnGarant/BookDetails.aspx?ID=9789046600207


[52] In this connection, it is interesting to have another look at the Terminator science fiction movies, which – among the existence of a Skynet AI system – seem to hint at a coming “judgment day”, https://en.wikipedia.org/wiki/Terminator_(franchise), http://terminator.wikia.com/wiki/Judgment_Day. Note that Cyberdyne and Genisys actually exist. See also the news about the German TV movie “Terror”, after which a telephone vote was held to decide it was ok to kill people in order to save the lives of (more) others. It’s cynical that this might imply their own death sentence in the future.