These days, political scientist Parag Khanna doesn’t get tired to advertise “technocracy” as the best form of government.[1] He is not the only one. Internet pioneers in the Silicon Valley and elsewhere have often declared democracy as “outdated technology”.[2] Peter Thiel proclaims a “deadly race between politics and technology”.[3] And Google’s Larry Page complained that “[t]here’s many, many exciting and important things you could do you just can’t do because they are illegal”.[4] This shows a concerning disrespect of our laws.
Politics
has promoted such thinking as well. In the wake of 9/11 and the Patriot
Act, social engineers such as George W. Bush and Tony Blair have
created a surveillance society, where a state of emergency was always
around the corner, seemingly justifying the restriction of human rights
and liberties step by step. Since then, democracies have undergone
worrying transformations towards more autocratic regimes in Hungary,
Poland, and Turkey. In the USA, UK and France, a transition is on the
way as well. Last but not least, Austria, Switzerland, The Netherlands
and Germany have also started to tumble. After one of my talks, someone
with insider knowledge even told me: “Switzerland may be the only
country to stay democratic.” He had concrete reasons to be concerned.
And in Switzerland too, some have suggested the Chinese system would be
more attractive…
Overloaded
and tired by the flood of revelations, many people have not realized
that we have ended up with technologies and laws, which parallel those
described in George Orwell’s 1984 or Aldous Huxley’s Brave New World.
For example, just a few days ago, a new mass surveillance law (the
large-scale use of “Staatstrojaner” federal hacking software) has passed
the German Bundestag with a trick.[5] Little by little, we have ended
up with the digital tools for a data-driven, AI-based so-called
“benevolent dictatorship”.[6] This technocratic state will be
totalitarian for sure, but will it also be “benevolent”? How may it look
like, according to the evidence so far?
Big Brother: Since
the Snowden revelations,[7] we know that there is mass surveillance of
billions of people around the world, but most of us still have no idea,
how pervasive it is. For example, all of our clicks in the Internet are
reported by cookies, and there are a lot more digital traces of our
everyday activities, which are also recorded. In Great Britain, more
than 40 government institutions have access to the click history, even
if it was regularly deleted.[8] Recently, Wikileaks “Vault 7” revealed
that the CIA – just one of more than 15 secret services in the USA – had
spent many billion dollars on mass surveillance tools to hack most
kinds of computers, smartphones, smart TVs and smart cars – and that it
had lost control over them, i.e. dual use of hacking tools by others was
now a serious problem.[9]
Surveillance Capitalism:[10]
It has become known that all citizens are being targeted to create
personal profiles of them. This profiling covers thousands of specifiers
per person (including name, birth day, age, income, address, religious
denomination, sexual preferences and personal tastes, as well as
health-related data, which is being sold to other companies). You have
probably no idea of how much data has been secretly collected about you
by corporations (so-called “Big Others”), and what is being done with
it. The data give a much more detailed picture than the files that
secret services used to run totalitarian systems in the past. You would
be surprised to learn what companies such as Google, Facebook, Recorded
Future, Acxiom, Palantir, and others know about you – or may infer about
you from your personal data. Just have a look at your Crystal Knows
profile to get a first idea.[11]
Big Nudging:[12]
In today’s “attention economy”,[13] our personal data is being used to
personalize information such that it will influence our attention,
emotions, opinions, decisions, and behaviors – often in a subconscious
way. This ranges from steering our consumption behavior with so-called
“Neuro-Marketing” methods[14] up to manipulating our voting behavior
during elections, as it happened during the Brexit vote and US election
in 2016.[15] When combined with Social Bots – armies of robotic bloggers
– very effective propaganda tools result. As the debate about Fake News
and the post-fact society shows, many people are falling prey to these
new technologies, some of which can even edit Web contents[16] (i.e.
manipulate history). Populism, polarization, and fragmentation of
societies have spread and threaten social peace. However, the censorship
laws and “ministries of truth” to fight the problem are probably even a
greater danger to democracies.
World (or War) Simulator: The
personal data collected about each of us is being fed into a machine
learning algorithm that learns how we behave. It produces a “digital
double” – something like a personal avatar. These digital doubles are
then used, on the one hand, for global “war games” – massive computer
simulations such as Sentient World,[17] which try to predict our future.
On the other hand, these digital doubles can also be used to test how
we would respond to certain information stimuli. This is of interest for
the next generation of “predictive policing”,[18] which would work
pretty much like “Minority Report”. For example, it would be possible to
find out beforehand, who is likely to protest against a shutdown of the
freedom of press, or of democracy altogether.
Citizen Score: All
the data collected about you are boiled down to a single number, the
Citizen Score, representing your value from the point of view of those
in power. This system will be a future governance tool in China, where
it is currently being tested.[19] However, there is a similar system
called “Karma Police” in Great Britain,[20] and most likely everywhere
else. The system gives plus or minus points for everything you do, for
the friends you have, the links you click on the Internet, the movies
you watch, and the music you hear. The overall score will then decide
about interest rates for loans, the job you may get, travel visa to
other countries, and (conditions of) access to products and services.
Cashless society:[21]
Once people would have a unique identity chip,[22] the Citizen Score
could also be used as basis of a cashless society. In particular during
future resource shortages, the Citizen Score could decide about the
access to resources such as long-distance travel, medicine or meat.
While all of this will be claimed to comply with principles of fairness
and justice, in the future the Citizen Score might decide about life or
death. But even if resources were sufficient for everyone – the
competition for high Citizen Scores and good services would turn many
creative individuals into submissive subjects.
“Benevolent dictatorship”: This
comprises all of the above elements. The World Simulator would be used
to come up with a grand plan for the world, which would imply specific
roles for every individual. These would then be suggested to us through
our smartphones or similar devices by means of personalized information
(“Big Nudging”). If we would execute the suggested actions, this would
earn us plus points, otherwise we would be punished by minus points. The
ideological justification for such a system is that a
“superintelligent” AI system would know better what is good for us, and
therefore impose it on us (this is sometimes called the “Big Mother”
society). In other words, people – currently free decision-makers –
would eventually become something like an “output device” of a digital
command economy. Freedom and self-determination, human rights and
democracy would be gone.
Technological and Legal Assessment
Some
technology gurus have said that our current system has created “wealth,
health and happiness for billions of people … but now we want to try
something new”.[23] Unfortunately, this new solution has not been
properly tested. One would think that it would be the right approach to
start with the reorganization of a company, then with that of a city
(e.g. San Francisco), then with the reorganization of a state (such as
California), subsequently of a country (e.g. USA), and then of a
continent (e.g. America), before one rolls out the system in the entire
world. So far, however, no “smart city” has made it into the top 10 hit
parade of most livable cities, and it has recently been concluded that
the technocratic Smart Cities approaches have largely failed.[24] In
other words, there is no proof that anyone has managed yet to invent
anything that comes close to a digital paradise on Earth. On the
contrary, we have recently seen problems such as hate speech, fake news,
and cybercrime abound, producing much harm to our societies.
Moreover,
the most fundamental problem is unsolved, namely how to determine the
right goal function.[25] Should it be the gross domestic product or
sustainability, power or peace, happiness or life expectancy, or
something else? If we chose power, for example, the AI system would
learn to make us addicted to information and dependent on it.[26] On the
one hand, therefore, Apple boss Tim Cook distanced himself from
manipulating people with personalized information.[27] Elon Musk warned
of superintelligent systems as “our greatest existential threat”,
potentially more dangerous than nuclear weapons.[28] Consequently, he
spent a billion dollars to launch the OpenAI project to ensure AI will
be “an extension of individual human wills and, in the spirit of
liberty, as broadly and evenly distributed as possible.”[29]
On
the other hand, the algorithmic personalization approaches by Google
and others were blamed to discriminate women[30] and people of
color.[31] Facebook came under pressure, as its platform seemed to
contribute to the spread of extremism,[32] and Microsoft had to take
down their chatbot “Tay” from the Web, as it started to produce Nazi
propaganda.[33] Big Nudging has been heavily criticized in connection
with the Brexit and US election campaigns.[34] The Citizen Score was
condemned by The Economist and other newspapers for its totalitarian
nature.[35] An ethics committee established by the German government
decided that AI systems such as autonomous cars were not allowed to take
life-and-death decisions, which would advantage one kind of people over
another one.[36] Moreover, the US Supreme Court has declared that hate
speech does not justify censorship.[37] And the United Nations Human
Rights Convention demands that our protect our privacy be protected.[38]
Last but not least, the European Court of Justice decided that mass
data retention (Vorratsdatenspeicherung) without sufficient reasons is
illegal.[39] So, the situation is clear: all of the above developments
should not happen – and there is no evidence that they would create a
better society. So, why is it still happening then?
The Sustainability Agenda – A Totalitarian Agenda Wrapped in Nice Words?
In
the year 2015, some important decisions were taken about the future of
the world. Opened by Pope Francis on September 25, the United Nations
decided to pursue very ambitious Global Sustainability Goals with its
Agenda 2030.[40] Strong institutions would be built to make the world
sustainable by 2030, within just 15 years! On September 25, there were
also a number of other notable events: Switzerland decided about a new
surveillance law (Nachrichtendienstgesetz).[41] The Intercept reported
about the Karma Police program,[42] and Kai Schlieter’s book “Die
Herrschaftsformel” [The formula to rule the world] appeared with a
delay.[43]
The
Paris Climate Deal[44] followed up the Agenda 2030 and the Agenda 21.
It decided concrete actions. Carbon dioxide emissions, for example, will
have to be reduced by 40 percent at least. This shall be achieved by
carbon trading,[45] which will create a one trillion dollar market, to
be paid by the consumers. The aim to charge everyone in a fair way, may
serve as an excuse for mass surveillance. One could say this requires
detailed data about who is travelling how much and in what ways. Who is
consuming how much, and what are the Carbon Dioxide emissions generated
by these products? Moreover, carbon trading will probably compensate oil
and gas companies for their expected losses, even though they have
driven the planet to the edge and also suppressed alternative energy
technologies. All of this is obviously neither moral nor legitimate –
and that is probably the reason why the public has not be informed about
the detailed plans in a transparent way, as it should be.
The
Paris Climate Deal was made under the impression of the terror attack
on Friday, November 13, 2015,[46] which initiated martial law and
restricted constitutional rights – until today. For example, public
demonstrations can now be forbidden, and private homes can be searched
by police without any evidence of wrongdoing. Such measures undermining
democracy have been seen also in many other countries such as Great
Britain or Germany.
All
of these developments become understandable if you read the “Limit to
Growth” study commissioned by the Club of Rome[47] or the Global 2000
report[48] commissioned by US President Jimmy Carter, their updates, and
follow-up studies. These reports try to project the future of the Earth
by means of computer simulations. The shocking result of these studies
was that, given the resource constraints of our planet, an economic and
population collapse would be unavoidable (if we do not change our
financial and socio-economic system[49] in a way that promotes a
circular and sharing economy[50] – this would be the right solution!).
Source: How Many Earths? (September 27, 2015) https://thecomingmigrations.com/2015/09/27/how-many-earths/
Today’s
conventional view is that the world is overpopulated and overusing its
renewable resources by at least 40 percent (see graphic above). As a
consequence, about one third of the world’s population would die of an
unnatural death in this century (see graphic below). This would be more
people than currently live in the USA, Europe, Russia and China
together.
Source: http://www.smithsonianmag.com/science-nature/looking-back-on-the-limits-of-growth-125269840/#HOSPivUzCMQMbY1v.99. Chart Sources: Meadows, D.H., Meadows, D.L., Randers, J. and Behrens III, W.W. (1972) (Linda Eckstein)
From
this perspective, it appears logical that governments would prepare to
ration scarce resources and distribute them fairly, according to
personal merit, as reflected by the Citizen Score. It also becomes
understandable that preparations were taken to suppress civil unrest,
ranging from mass surveillance to armed police forces. Of course, there
would also be preparations for times, when resources would not suffice
anymore for everyone. Therefore, the question was raised: could AI
systems be used to take decisions over life and death[51] – something
one could probably call digital “judgment day”.[52] Now, you probably
start to understand Elon Musk’s concerns about superintelligent systems.
I also think that our societies need to take a completely different
path to sustainability – based on a new financial, money and economic
system. I will describe this in forth-coming contributions.
[1] Die beste Regierungsform ist die direkte Technokratie, Tagesanzeiger (Jun 9, 2017), see http://www.tagesanzeiger.ch/ausland/standard/freiheit-ist-nicht-alles/story/25180340, https://www.infosperber.ch/Artikel/FreiheitRecht/Politologe-will-Schweiz-und-Singapur-mischen
[2] „Sie [die Demokratie] hat Reichtum, Gesundheit und Glück für Milliarden Menschen auf der ganzen Welt gebracht. Aber jetzt wollen wir etwas Neues ausprobieren und dazu haben wir hier keine Chance.“ See http://www.3sat.de/page/?source=/kulturzeit/themen/176692/index.html, http://www.theeuropean.de/aleksandra-sowa--2/12369-die-schoene-neue-cyberwelt-und-die-alte-demokratie
[3] Peter Thiel is trying to save the world: The apocalyptic theory behind his actions, Business Insider (December 8, 2016), see http://uk.businessinsider.com/peter-thiel-is-trying-to-save-the-world-2016-12
[4] Google CEO Larry Page Wants A Totally Separate World Where Tech Companies Can Conduct Experiments On People, Business Insider (May 16, 2013), see http://www.businessinsider.com/google-ceo-larry-page-wants-a-place-for-experiments-2013-5
[5] Überwachung: Koalition macht Staatstrojaner zum polizeilichen Alltagswerkzeug (Heise Online, June 20, 2017) https://www.heise.de/newsticker/meldung/Ueberwachung-Koalition-macht-Staatstrojaner-zum-polizeilichen-Alltagswerkzeug-3748014.html
[6] D. Helbing, Dictatorship 4.0: How the digital revolution threatens our freedom - and what our alternatives are, see https://www.researchgate.net/publication/317533191_Dictatorship_40_How_the_digital_revolution_threatens_our_freedom_-_and_what_our_alternatives_are; The Dream to Control the World – and Why It Is Failing, see https://www.researchgate.net/publication/316989750_The_Dream_to_Control_the_World_-_and_Why_it_is_Failing
[7] for an overview see https://www.theguardian.com/us-news/the-nsa-files
[8] Revealed: The 48 organisations that can see your entire online browsing history, even if you delete it, Mail Online (Nov 25, 2016). Retrieved from http://www.dailymail.co.uk/sciencetech/article-3971214/The-48-organisations-entire-online-browsing-history-delete-it.html ; Browsers nix add-on after Web of Trust is caught selling users’ browsing histories, The Register (Nov 7, 2016). Retrieved from https://www.theregister.co.uk/2016/11/07/browsers_ban_web_of_trust_addon_after_biz_is_caught_selling_its_users_browsing_histories/ ; Apple iCloud Hoards 'Deleted' Browser History Going Back More Than A Year, Forbes (Feb 9, 2017). Retrieved from http://www.forbes.com/sites/thomasbrewster/2017/02/09/apple-safari-web-history-deleted-stored-icloud/#632df5322439
[9] see https://en.wikipedia.org/wiki/Vault_7 and https://wikileaks.org/vault7/ for details
[10] S. Zuboff, Big Other: Surveillance Capitalism and the Prospects of an Information Civilization, Journal of Information Technology 30, 75-89 (2015).
[11] see https://www.crystalknows.com
[12] see the corresponding information box in https://www.scientificamerican.com/article/will-democracy-survive-big-data-and-artificial-intelligence/
[13] T.H. Davenport and J.C. Beck, Attention Economy: Understanding the New Currency of Business (Harvard Business Review, 2002).
[14] H.M. Sola and M. Econ. Neuromarketing Armoury (Createspace, 2017), see https://www.amazon.de/dp/1543261892/; L. Zurawicki, Neuromarketing: Exploring the Brain of the Consumer (Springer, 2016).
[15] The Rise of the Weaponized AI Propaganda Machine, https://medium.com/join-scout/the-rise-of-the-weaponized-ai-propaganda-machine-86dac61668b; also watch this video of Alexander Nix talking about Psychographics: https://www.youtube.com/watch?v=n8Dd5aVXLCc
[16] Nothing is real: How German scientists control Putin's face, https://www.youtube.com/watch?v=ttGUiwfTYvg; http://www.graphics.stanford.edu/~niessner/thies2016face.html; Adobe VoCo ‘Photoshop for voice’ causes concern, http://www.bbc.com/news/technology-37899902; https://www.youtube.com/watch?v=I3l4XLZ59iw
[17] Sentient World: War games on the grandest scale. The Register (June 23, 2007) see https://www.theregister.co.uk/2007/06/23/sentient_worlds/; see also https://en.wikipedia.org/wiki/Synthetic_Environment_for_Analysis_and_Simulations and https://www.youtube.com/watch?v=MVrBZevJY3U
[18] W.L. Perry et al., Predictive Policing: The Role of Crime Forecasting in Law Enforcement Operations (RAND Corporation, 2013); see also https://en.wikipedia.org/wiki/Predictive_policing
[19] China’s Nightmarish Citizen Scores Are A Warning For Americans, https://www.aclu.org/blog/free-future/chinas-nightmarish-citizen-scores-are-warning-americans
[20] Karma Police: GCHQ spooks spied on every web user ever, The Register (Sep. 25, 2015), https://www.theregister.co.uk/2015/09/25/gchq_tracked_web_browsing_habits_karma_police/
[21] Cashless society getting closer, survey finds, see http://www.reuters.com/article/us-global-economy-cash-idUSKBN17S001
[22] Human Microchipping: An Unbiased Look at the Pros and Cons, https://medium.freecodecamp.com/human-microchipping-an-unbiased-look-at-the-pros-and-cons-ba8f979ebd96
[23] R. Hencken, in: Mikrogesellschaften. Hat die Demokratie ausgedient? (Capriccio, 2014). Video, veröffentlicht am 15.5.2014. Autor: J. Gaertner. München: Bayerischer Rundfunk.
[24] Disrupting cities through technology, Wilton Park (March 17-19, 2016), see https://www.wiltonpark.org.uk/wp-content/uploads/WP1449-Report.pdf
[25] D. Helbing and E. Pournaras, Build Digital Democracy, Nature 527, 33-34 (2015).
[26] Smartphone Addiction, Psychology Today (Jul 25, 2013) https://www.psychologytoday.com/blog/reading-between-the-headlines/201307/smartphone-addiction; see also http://www.dailymail.co.uk/news/article-2174024/Facebook-creating-generation-gambling-addicts-sites-Las-Vegas-style-games.html and http://www.business2community.com/facebook/7-ways-facebook-keeps-addicted-apply-lessons-products-01851945
[27] http://www.dailymail.co.uk/sciencetech/article-3108457/Apple-boss-Tim-Cook-launches-blistering-attack-Google-Facebook-claims-people-fundamental-right-privacy.html
[28] http://observer.com/2015/08/stephen-hawking-elon-musk-and-bill-gates-warn-about-artificial-intelligence/
[29] https://blog.openai.com/introducing-openai/
[30] http://www.techtimes.com/articles/67268/20150709/study-suggests-google-algorithms-discriminate-against-women.htm
[31] https://www.washingtonpost.com/news/morning-mix/wp/2016/06/10/google-faulted-for-racial-bias-in-image-search-results-for-black-teenagers/?utm_term=.d500edd42210; see also https://www.theguardian.com/commentisfree/2016/jun/26/algorithms-racial-bias-offenders-florida for the more general problem
[32] Beihilfe zur Volksverhetzung bei Facebook: Staatsanwaltschaft München ermittelt gegen Mark Zuckerberg, Der Tagesspiegel (Nov. 4, 2016) http://www.tagesspiegel.de/politik/beihilfe-zur-volksverhetzung-bei-facebook-staatsanwaltschaft-muenchen-ermittelt-gegen-mark-zuckerberg/14796574.html
[33] Microsoft terminates its Tay AI chatbot after she turns into a Nazi. Ars Technica (March 24, 2016) https://arstechnica.com/information-technology/2016/03/microsoft-terminates-its-tay-ai-chatbot-after-she-turns-into-a-nazi/
[34] https://www.theguardian.com/technology/2017/may/07/the-great-british-brexit-robbery-hijacked-democracy
[35] Big Data, meet Big Brother: China invents the digital totalitarian state. The Economist (Dec 17, 2016) see http://www.economist.com/news/briefing/21711902-worrying-implications-its-social-credit-project-china-invents-digital-totalitarian
[36] Ethikkommission warnt vor Totalüberwachung des Menschen, Die ZEIT (June 20, 2017) http://www.zeit.de/mobilitaet/2017-06/autonomes-fahren-totalueberwachung-ethik-kommission
[37] Supreme Court unanimously reaffirms: There is no ‘hate speech’ exception to the First Amendment, https://www.washingtonpost.com/news/volokh-conspiracy/wp/2017/06/19/supreme-court-unanimously-reaffirms-there-is-no-hate-speech-exception-to-the-first-amendment/?utm_term=.f88021163948
[38] Universal Declaration of Human Rights, http://www.un.org/en/universal-declaration-human-rights/
[39] Europäischer Gerichtshof: It's the end of Vorratsdatenspeicherung as we know it, Die ZEIT (Dec. 21, 2016) http://www.zeit.de/digital/datenschutz/2016-12/europaeischer-gerichtshof-vorratsdatenspeicherung-urteil
[40] http://www.un.org/sustainabledevelopment/blog/tag/pope-francis/
[41] https://www.admin.ch/gov/de/start/dokumentation/abstimmungen/20160925/nachrichtendienstgesetz.html
[42] https://theintercept.com/2015/09/25/gchq-radio-porn-spies-track-web-users-online-identities/
[43] K. Schlieter, Die Herrschaftsformel: Wie Künstliche Intelligenz uns berechnet, steuert, und unser Leben verändert (Westend, 2015).
[44] Paris Agreement, https://en.wikipedia.org/wiki/Paris_Agreement
[45] Carbon Trading: The World’s Next Biggest Market, http://www.greenchipstocks.com/report/carbon-trading-the-worlds-next-biggest-market/107; The EU Emissions Trading System, https://ec.europa.eu/clima/policies/ets_en
[46] November 13 Paris attacks, https://en.wikipedia.org/wiki/November_2015_Paris_attacks
[47] The Limits to Growth, https://www.amazon.de/Limits-Growth-Project-Predicament-Mankind/dp/087663918X/, see also the 30-year update, https://www.amazon.de/Limits-Growth-Donella-H-Meadows/dp/193149858X/
[48] Editions of the Global 2000 Report to the [US] President, see http://www.geraldbarney.com/G2000Page.html
[49] D. Helbing, Why we need democracy 2.0 and capitalism 2.0 to survive, published in Jusletter IT, https://www.researchgate.net/publication/303684254_Why_We_Need_Democracy_20_and_Capitalism_20_To_Survive
[50] City Olympics, a decentralized, participatory, socio-ecological finance system (finance 4.0+), democratic capitalism and digital democracy would be solutions to make the world sustainable by 2030 in a democratic way, see http://futurict.blogspot.de/2017/06/propositions-on-perspective-global.html and http://futurict.blogspot.de/2017/06/digitization-20-new-game-begins.html
[51] Een computermodel voor het ondersteunen van euthanasiebeslissingen, http://www.maklu.be/MakluEnGarant/BookDetails.aspx?ID=9789046600207
[52] In this connection, it is interesting to have another look at the Terminator science fiction movies, which – among the existence of a Skynet AI system – seem to hint at a coming “judgment day”, https://en.wikipedia.org/wiki/Terminator_(franchise), http://terminator.wikia.com/wiki/Judgment_Day. Note that Cyberdyne and Genisys actually exist. See also the news about the German TV movie “Terror”, after which a telephone vote was held to decide it was ok to kill people in order to save the lives of (more) others. It’s cynical that this might imply their own death sentence in the future.
