Monday, 20 September 2021

Is the Data-Driven Paradigm of Managing the World Driving Us to the Brink of Apocalypse?

The idea that more data will automatically produce a better picture of the world and of the future must be critically reflected. It is a future vision of the past. In the last 10 years, we have learned a lot about the pitfalls of big data and machine learning - many of which have not yet been fixed yet.

The biggest commercial machine learning models currently in use learn a trillion parameters or so (assuming measurements are unbiased and reliable, which is often not the case). However, whatever the maximum number of parameters may be, when modeling the world in this way, there are many orders of magnitudes of interaction effects which are not captured, hence neglegted, and that will produce a wrong picture and bad forecasts. For example, even forecasts of the COVID-19 pandemic, of global warming, or peak oil have been consistently unreliable.

Or let us take the forecast of the world's population and carrying capacity. According to the Club of Rome’s simulations and other statistics/projections,about one third of the world’s population is overpopulation and would die early of resource shortages. However, these simulations did not consider new forms of energy production. More importantly, they do not consider the possibility of reorganizing the economy towards a circular and sharing economy. If we had done this already, the world may not be overpopulated at all. Hence, „over-population“ may just be a matter of bad economic organization. That is why I am stressing the need of nature-inspired and social innovation so much [1], in particular the socio-ecological finance system [2] and interaction support processor [3], which would introduce new incentives and feedback effects that would promote the co-evolution towards a circular and sharing economy. And I stress the importance of digital democracy [1,4] and democratic capitalism [1,5] to unleash the transformative power of civil society.

If one, instead, would take a military top-down approach, say, using something alike a "digital crystal ball" and a "war room“ to decide how to manage the planet, then the positive interaction and feedback effects would be neglected or even eliminated. One would see a much worse future than we could have, and the decisions resulting from this might actually cause avoidable depopulation events. In other words, the wrong use of Big Data may not only see an „apocalyptic" future, which might be avoidable, it may even cause it, because of oversimplifications made by the tools used. The digital "crystal ball" will not see the alternative economy that might save us, because it is not in the data of the past. It requires innovation and systemic change.

[1] Next Civilization https://www.springer.com/gp/book/9783030623296 [2]

Finance 4.0: Towards a Socio-Ecological Finance System Book


[3] Interaction Support Processor, https://patents.google.com/patent/US20160350685A1/en


[4] Build Digital Democracy: https://www.nature.com/articles/527033a


[5] Socio-Economic Implications of the Digital Revolution, https://www.researchgate.net/publication/348742263

Saturday, 18 September 2021

The Planetary Health Agenda – A Global Political Problem

Abstract: The “Planetary Health” proposal discussed here describes a totalitarian vision favoring a data-driven, AI-controlled management of all resources of the world, from the planetary scale down to the microscopic processes in human bodies and in nature. The goal includes the control of behavioral, social and cultural change. However, the concept does not only ignore the concepts of ethics and democracy. It also ignores immaterial qualities such as human dignity, freedom, creativity, consciousness, or love, which are of particular importance for humans. Therefore, such an approach may easily and unintentionally result in violations of the Nuremberg code or crimes against humanity. Proper political action is urgently needed.

“The physician examines the human body and determines whether ... all organs are working for the benefit of the entire organism. We [Dehomag] are very much like the physician, in that we dissect, cell by cell, the German cultural body. We report every individual characteristic ... on a little card… We are proud that we may assist in such a task, a task that provides our nation’s Physician […] with the material he needs for his examinations. Our Physician can then determine whether the calculated values are in harmony with the health of our people. It also means that if such is not the case, our Physician can take corrective procedures to correct the sick circumstances... Our characteristics are deeply rooted in our race. Therefore, we must cherish them like a holy shrine, which we will – and must – keep pure. We have the deepest trust in our Physician and will follow his instructions in blind faith, because we know that he will lead our people to a great future. (Heidinger, 1934).” [1]

The document discussed in the following sometimes reminds of the above quote by Heidinger. It formulates not only a “requirement of preserving the life support systems”, but also makes a number of statements that deserve attention and debate, such as:

1. “Planetary health is defined as health of human civilization and the state of the natural systems on which it depends.”

2. “Health is the state of complete physical, mental and social well-being and not merely the absence of disease or infirmity”

3. “We posit that natural resources are the socioeconomic signal of planetary health.”

4. “Humans are strongly health-minded, and are individually and collectively resource-driven…”

5. “Resource justice is defined as a … process about collective and individual relationships encompassing the distribution of rights and responsibilities…”

6. The “ultimate challenge our societies are facing” is to “reconcile the basic needs of populations with the maintenance of the life-support functions of ecosystems…”

7. “Resources, including the human resource, are the matrix of economic and political power systems, of history’s ups and downs, and they have great geopolitical sensitivity.”

8. “Today, critical state shifts are expected around 2030” (talking about “societal collapse”).

9. ‘…inseparable health of nature-societies-people, known as the “planetary health” ”

10. “A socially just and environmentally responsible society ought to consider that Health is a precondition, outcome, and indicator of a sustainable society, and should be adopted as a universal value and shared social and political objective for all.”

11. “… future undertakings in all areas of human life and activities should take a resource-health stress test”

12. “To address these issues, we developed a Resource-Planetary Health Toolbox (RPHT), built on the alliance of natural sciences, legal and social studies, and data and complex system resilience”

13. “The approach is systemic and preventive and transgresses academic, cultural, and political boundaries. It has the ambition to circumvent the above-mentioned barriers…”

14. The objective is to articulate the nature-human relationship on universal health and indivisible human rights and duties, to which health is the basis for a science-informed civilization contract…”

15. “… the price of commodities reflects the true social and ecological costs…”

16. “… building consensus toward social justice and ecological responsibility.”

17. “… the ambition to frame human agency”

18. “… natural and data sciences for monitoring and reporting with institutionalized data systems”

19. “That inclusive design articulates social justice and ecological responsibility by setting the balance between the management through numbers and the government through law”

20. “… systematically monitor and evaluate … locally and globally…”

21. “Enacting resource justice to balance basic needs/poverty, limiting resources, social cohesion, demography…”

22. “cultural shift linking health and wealth”, “socioecological transitions”

While some of these statements may sound right at first, my interpretation is as follows, where the numbering refers to the statements above:

1. While claiming to aim at improving health, it becomes clear that the underlying goal is to manage nature and human civilization, globally. Hybris is one concern here, the possibility of global totalitarianism is another one.

2. It is apparently intended to not only eradicate disease entirely, but also to manage physical, mental and social aspects of everyone’s life. “State of complete physical, mental and social well-being” sounds a bit like everyone would be drugged in the future. It also potentially implies the elimination of pain and of people with diseases. Pain, however, is an evolutionary function that supports learning. Also note that people with diseases typically do not want to die. They often value their lives, as do their friends, colleagues, and relatives. Furthermore, many of the most impressive cultural achievements of our world have been produced by people who were sick, unhappy or handicapped.

3. Here, managing health is further reduced to the problem of managing natural resources. Non-material aspects such as happiness, love, creativity or human dignity disappear from the further consideration.

4. It is suggested that health is the overarching goal of humans, and that they are resource-driven. This again underlines a materialistic approach, which is highly questionable. I do not think that the strive for resources and health is the dominating goal of most people. Personal life achievements are often more idealistic.

5. Managing resources is defined as a matter of the distribution of rights and responsibilities. Consequently, there are no “God-given” human rights by birth anymore. Instead, one needs to earn his/her rights by fulfilling certain responsibilities.

6. Next, the document claims the ultimate goal of our societies would be life support. In a different place, the term “basic needs” is used, which reminds somewhat of the lowest level of Maslow’s Hierarchy of Needs. Accordingly, higher-level needs such as Safety, Love, Belonging, Esteem or Self-Fulfilment would potentially be neglected.

7. Resources are then related to economic and political power systems and geopolitics. Accordingly, human and environmental health become subjects of an economic and political battle for power.

8. It is claimed that we would see societal collapse around the year 2030, if we would not submit the goals of individuals and societies around the world to the logic of planetary health.[2]

9. It is further suggested that nature, societies, and people share one faith, while this is only expected to hold for a tightly connected globalized world.

10. Here, Health is given the role of an universal value, which in a sense replaces the role of God (and, consequently, “Health” is written with a capital “H”). All people are supposed to share Health as the overarching political and social objective.

11. It is then suggested that all areas of human life should undergo a resource-health stress test. This could be (mis-)understood as suggesting to expose societies to diseases and disasters, possibly to identify and get rid of the “unhealthy” and “vulnerable” elements. However, “vulnerability” is part of what makes us human and what makes us social.

12. Apparently, the above is not a theoretical consideration or vision of the future, but a system that has already been built (in parts) by an alliance that ranges from science to law.

13. The system also overcomes and possibly ignores or violates established scientific, cultural and political (probably also legal) frameworks, and it is tasked to prevent certain actions.

14. The goal is a science-informed civilization, in which health is the universal goal. It is reiterated that rights would be tied to duties.

15. This science-informed technocracy is based on attributing “true costs” to everything (including ecological and social aspects of society). This would, hence, imply a utilitarian approach, corresponding more or less to what is known as the “Chinese Social Credit Score” system. (It might be seen as a new kind of all-encompassing capitalism that values all externalities and all individual behaviors on an effectively one-dimensional scale.)

16. Based on this system, social justice and ecological responsibility would be enforced, and all people would have to agree with this.

17. The ambition goes so far as to control human activities.

18. A surveillance-based and data-driven approach is taken, which is institutionalized (probably UN).

19. Societies and individuals would be managed through numbers, and this system would be enforced by law.

20. People and nature would be under surveillance locally and globally.

21. Access to resources would be controlled, social cohesion undermined, and population control established.

22. The overall approach appears to be in the tradition of social Darwinism and sounds somewhat like “Death to the weak(ling); wealth to the strong” or “The rich should live on forever”.

The Anticipatory Brief on “Planetary Health and Global Resources” might, therefore, raise some serious concerns. An even better picture results, when the subtitle “The Cross-Cutting Challenge of All” is taken into account. As the vision of this Anticipatory Brief builds on the vision of basically all other Anticipatory Briefs, this picture encompasses:

I. The data-driven approach builds on a highly detailed “digital avatar” of the world, which probably includes digital twins that map our behaviors and inner workings of our bodies.

II. The digital twins will be fed by surveillance data obtained through the Internet of Things, which will be processed by extremely powerful machine learning approaches and Quantum Computers.

III. This platform will be used to manage all global resources, including our lives, with the goal to achieve “planetary health”.

IV. Such an approach will probably also include the use of nanotechnology, an Internet of Bodies, genetic engineering, as well as mind control approaches to steer thoughts, emotions, and behaviors[3] as well as the implementation of a new ethics based on the logic of the “trolley problem” (“morality engineering”).

 Hence, the other Anticipatory Briefs may be considered to specify how the “Planetary Health” goal would be technically accomplished.

Criticism

Even though the paper is relatively vague, the idea shines through to „eliminate“ or „eradicate“ disease (and probably also „misbehavior“[4]). I do not think that this would work or that such a world would be desirable to live in. The following sections will provide a detailed criticism of some of the major scientific and political issues implied by the Planetary Health vision outlined above.

Limitations of a Data-Driven Approach

First of all, we need to learn from the shortcomings of data-driven approaches in the past, which were merely based on Big Data analytics, machine learning and AI.[5] This includes

a. classification errors (false positives or negatives),

b. spurious correlations, identification of random patterns, and overfitting,

c. sensitivity to minor modifications in the dataset or choice of the algorithm applied,

d. lack of robustness, stability or reliability of results,

e. slow learning processes or lack of convergence,

f. underestimation of the danger of wrongly fitting a certain fraction of the millions, billions, or even trillions of variables of future machine learning models,

g. underestimation of the role of diversity/heterogeneity and randomness,

h. underestimation of the role of systemic, functional, or algorithmic complexity,

i. limits of predictability,

j. limits of control,

k. limits of a data-driven approach (as the data volume grows faster than the processing power, the share of “dark data” never processed is steadily growing; consequently, a scientifically informed approach is needed to decide what data should be processed and how),

l. limits of a techology-driven approach (there are many ways how to use digital technologies),

m. limits of a transhumanist view (humans are not biological robots),

n. limits of a positivistic approach (non-material and hardly quantifiable factors are often neglected, including those that make us human, such as consciousness, freedom, creativity, love, solidarity, or human dignity...).

Other Scientific Flaws

A. Our immune system is generally highly robust, because it is being challenged by diseases all the time. The absence of diseases would probably weaken our immune system. In other words: health is not the result of absence of diseases, but a result of successfully coping with diseases. It is the result of interspecies interactions.

B. The current “Planetary Health” approach dramatically underestimates the roles that self-organization and emergence play for our well-being and a healthy state of nature.

C. In complex dynamical systems, global optimization and control approaches often perform worse than approaches based on flexible adaptation to local needs.

D. The great(er) performance of natural systems is often based on coordination, cooperation, co-evolution, synergy, and symbiosis.

E. A good illustration is the role that the microbiome plays for our health. Decades ago, it was believed that it was a good idea to disinfect everything in order to kill bacteria. In the meantime it is known that we could not exist without bacteria. In fact, there are about 10 times as many bacteria in our guts than cells in our body. These bacteria form an ecosystem that benefits us: it does not only digest food and make calories, vitamins, and minerals available to us, but it also is an important part of our immune defense.

F. Rather than pursuing a surveillance-based, data-driven, AI-controlled Planetary Health approach, it appears more promising to follow a symbiotic „ecosystem“ approach, considering also environmental, social, contextual and non-material factors (including mental and psychological ones).

G. Overwriting the operation system of our society, culture, mind, and body appears to be a radical, ideology-driven approach (aiming to “play God”) rather than an approach that is scientifically sound and recommended.

Political Issues

The “Planetary Health” proposal promotes first and foremost the interests of a medical-industrial complex.[6] There is a great lack of transparency and participation. It is entirely unclear what role, if any, democracy and human rights or human dignity will have in this framework. A global utilitarian optimization of planetary health and resource use would probably lead to phenomena such as organ harvesting against the will of people and algorithm-based triage decisions, as it is already reported from certain countries.

The spread of triage methods, which were developed for wars and war-like disaster situations, in everyday contexts is highly concerning and questions civil-ization altogether. People who are not counted to be “system relevant” by the algorithms in use would not stand good chances to get access to important resources. They would probably die early, as the planetary health approach would reward health, while punishing or trying to eliminate disease. The result would be technical selection benefitting genetically and otherwise gifted people. Eugenics would be a likely side effect.

Given the above as well as the so-called over-population of planet Earth and the lack of sustainability of our current economy, the management of highly sensitive personal/health data and medical treatment by private companies would be highly problematic. There would be obvious conflicts of interest, for example, when there are budgetary or other constraints that make patients’ interest unaligned with commercial interests of insurance companies and businesses in the health sector. The danger of crimes against humanity undoubtedly exists, particularly as there seems to be a lack of accountability for the algorithms used.

Imagine, in the future, someone would (be able to) apply technologies to our bodies or brains against our will, in favor of a proposed superior goal such as “Planetary Health”. Wouldn’t that be the mother of all nightmares? Is there any way to ensure that humans would not be experimented on against their will or exposed to unnecessary health risks? Otherwise related threats may largely outweight possible benefits.

In a data-driven and surveillance-based age, humans are increasingly managed and controlled similar to things. This strongly contradicts human dignity. For example, current Corona measures are deeply traumatizing to hundreds of millions of people around the world, probably more than even 9/11.

Perhaps the reason is that many of the ideas and technologies proposed or used in the context of planetary health have originally been developed in military contexts and labs. However, a civil society or civil-ization should not be based on military technologies, principles, or thinking, because this might quickly lead to an authoritarian, if not totalitarian framework of society.

For the governance of the highly networked techno-socio-economic-environmental systems of today we urgently need new organizational/societal frameworks and a lot more social innovation.

Judgement

The Anticipatory Brief on “Planetary Health” describes the most totalitarian vision I have ever come across. It appears to favor a brute-force, data-driven, AI-controlled management of all resources of the world, from the planetary scale down to the microscopic processes in human bodies and in nature. The goal includes the control of behavioral, social and cultural change. However, the concept does not only ignore the concepts of ethics and democracy. It also ignores immaterial qualities such as human dignity, freedom, creativity, consciousness, or love, which are of particular importance for humans. Therefore, the application of such a tool may easily and unintentionally result in violations of the Nuremberg code or crimes against humanity.

The proposed approach also seems to ignore many years of complexity and systems science, which has shown that self-organizing and evolutionary processes often outperform optimization approaches, in particular if synergistic or symbiotic processes are enabled. Resource optimization follows the logic of resource scarcity and implies a competition for and redistribution of limited resources. As a consequence, the proposed approach may put a large number of lives at risk. Biological, social and cultural processes, in contrast, often manage to overcome and outperform this logic of zero sum games (as a circular and sharing economy does).

Despite the above mentioned shortcomings, the flawed thinking behind the proposal already seems to drive political processes and technological developments worldwide. Behind great promises for the future, a highly concerning picture of society and its future is revealed. As a consequence of this new thinking, the very foundations of democracies, human rights and freedoms are at stake. It seems necessary to pull the emergency brake. In the light of the above, the Big Data and AI infrastructures built for the Planetary Health project should be seized and put under the control of The People of the world or otherwise be dismantled and turned off.

[1] W. Heidinger (1934), according to D.W. McCormick and J.C. Spee (2012), see https://www.tandfonline.com/doi/abs/10.1057/omj.2008.25
[2] This is basically the narrative of the Agenda 2030.
[3] Integrating Behavioral Health – The Role of Cognitive Computing
[5] A famous example is Google Flu Trends: celebrated as one of the big success stories of Big Data Analytics in the beginning, the service has been switched off and taken off the Web, as it was not sufficiently reliable.

Monday, 13 September 2021

The Corona Crisis Reveals the Struggle for Our Future

For a long time, experts have warned of the implications of a non-sustainable world, but few have understood the implications. Today’s economic system would enter a terminal phase, before a new kind of system would raise from the ashes. For sure, the digital revolution allowed the machinery of utility maximization to reach new heights. Today’s surveillance capitalism sells our digital doubles, in other words: detailed digital copies of our lives. It became increasingly clear that we would be next. People were already talking about the value of life, which, from a market point of view, could be pretty little – considering the fact of over-population and the coming wave of robotic automation. I have warned that some have worked on systems that would autonomously decide over lives and deaths of people, based on a citizen score reflecting what their “systemic value” was claimed to be.

Then came the Corona Virus. Even though the world had been warned in advance of the next great pandemic to come, COVID-19 hit the world surprisingly unprepared. Even though it started to spread in early December 2019, there was a shortage not only of respirators, but also of disinfectants and face masks as late as April 2020. And so, many people died an early death. Some doctors took triage decisions as in war times, and old or seriously ill people did not stand good chances to be helped. Some doctors relied on “terminal care”: basically, they gave opiates and sleeping pills to patients they could not save, and put them to death.

In the past 75 years, we have perhaps never been that close to the end of civilization. The fundamental principles of civil society were at stake. From one week to the other, driven by fear, we have lost much of our human rights: the freedom of mobility, the freedom of assembly, the freedom of worship, and – some of us – even the right to live. We were all prisoners at home and could not check the validity of news. Never before did we get so much information and learned so little. Some people called for mass surveillance, for immunity certificates, for obligatory tests and (later on) for forced vaccination. We were just one millimeter away from a totalitarian state. Some countries lost their democracies over these events. This could have easily happened around the world, given that we could not exercise our constitutional rights.

We have seen that a threat, which was dangerous for many lives, but not for humanity as such, was amplified by the media into a storm of fear and mass hysteria, such that it became a serious threat for our society, civilization and culture, which we have built over centuries. For sure, the economic system has been seriously damaged.

However, in this historical situation, we have made up our minds and figured out what really matters. We have realized that lives are more important than money. We have decided for solidarity. We have protected our democracy, which actuallyso far mastered the greatest challenge it had ever seen – contrary to the expectations of many. And even in times of lockdown and fear, we have not decided to give up our privacy and informational self-determination. Instead, we have promoted a new framework for the digital society – one, which is based on decentralization, values and empowerment. One, which even Google and Apple have to respect. So, we are seeing the dawn of a new society. A civilization based on respect for each other and for nature. It may take years to realize that it was a nasty virus that helped us build a more livable, more sustainable future together.

An edited, considerably less critical version of this text from 2020 was finally published here:

https://web.archive.org/web/20200609053329/https://www.wiko-berlin.de/institution/projekte-kooperationen/projekte/working-futures/wiko-briefs-working-futures-in-corona-times/the-corona-crisis-reveals-the-struggle-for-a-sustainable-digital-future/

Sunday, 12 September 2021

INNOVATIVE GOVERNANCE APPROACHES NEEDED: Some Thoughts on How to Master Our Future Using Big Data (also ‘Avatars’, ‘Digital Twins’) with a new Kind of Multi-Lateralism

There is little doubt that we are seeing ever more useful Big Data and AI applications that can make a contribution to a better world. It is much less obvious, however, whether the use of those technologies will benefit humanity and nature altogether, i.e. on average. Below, I just mention a few of the reasons that matter in this context:


Considering all this, fundamental reforms are needed in the way personal data is being collected, managed, and used. Otherwise it is possible that the digital transformation may fail altogether. Here are a few proposals for consideration and implementation:

A high level of cybersecurity is clearly needed. In this regard, backdoors are counterproductive.

Distributed data storage and control would be a strongly recommended safety feature.

Accountability: Any processing of sensitive data must be documented for a sufficient period of time, if ever possible in a reproducible way, such that errors and dual uses can be detected and corrective measures taken.

Transparency: Data and algorithms used must be opened up[1] to a certain number of mutually independent supervisory instances such that a sufficient quality of data processing from a statistical, scientific and technical point of view can be ensured (e.g. in terms of avoiding unjustified discrimination or in terms of requiring meaningful data analysis [no spurious correlations; sufficiently small false positive and false negative error rates; reproducibility; results not sensitive to small changes in the dataset, algorithm, or hardware, etc; of course, one also needs to consider the other known issues of Big Data analytics])

Open intelligence approaches are to be favored over secret services, whenever possible.

The organization of data access may be hierarchically organized, in terms of time delays of data access and accessible data volumes. The hierarchy level should depend on demonstrated responsible use and benefits created for the world. On the highest level, there should probably be at least 7 competitive, mutually supervisory instances (probably tied to leading countries). Their best scientific institutions should also have privileged access for the sake of quality assurance. The highest levels (serving as “guardians” of the system) would have to satisfy particularly strict rules in terms of independence and avoidance of conflicts of interest. On the next levels, with a larger time delay and smaller data volumes processed, there would be a larger number of participating entities, probably big companies and other countries. Again with some additional time delay and smaller data volumes processed, there would be SMEs and other scientific institutions as well as civil society initiatives, etc. Entities providing more (useful) data to the system should be higher up in the access hierarchy, to be able to make reasonable returns on their investments. With suitable access rules, one can ensure that there would be sufficiently many competitors, such that monopolies and a dangerous concentration of power is avoided. It would further be reasonable to have clear rules according to which entities move up or down in the hierarchy, based on how useful their contributions to the state of the world and the future and humanity were.

Ethical standards and responsible innovation must be ensured. People dealing with sensitive data should probably have to swear something like a “Hippocratic Oath for data scientists” (I can make suggestions, if desired). Qualified ethics committees must be put in place to ensure the ethical use of data, in compliance with human rights.

A diversity of teams as well as an inter-, multi-, and transdisciplinary approach (properly considering knowledge from the natural and complexity sciences, from the cognitive and social sciences, from history, philosophy and so on) as well as participatory opportunities for civil society projects should be ensured, in accordance with the peace room concept: https://www.theglobalist.com/technology-big-data-artificial-intelligence-future-peace-rooms/

Note that – due to their tendency of being extremely focused – neither a utilitarian approach, as it is often common in the economy, nor a military approach are expected to be suited to successfully run advanced, complex, multi-faceted, multi-objective, thriving societies and civil-izations living in peace with each other.

Participatory governance: There should be representatives of those affected by decisions, such that they have a say regarding what is going to happen to them. For example, regarding health applications, it is required to have a representative, independent patient council, which can voice preferences and concerns.

Digital democracy is a concept that aims at fostering collective intelligence, better solutions and collective action at scale: http://futurict.blogspot.com/2020/06/digital-democracy-how-to-make-it-work.html

Informational self-determination
is needed to ensure that people will still be able to control their own lives in the future: http://futurict.blogspot.com/2019/01/a-platform-for-informational-self.html

This does not mean though that everyone should do what they like, independently of how much harm is caused to others or to nature.

The socio-ecological finance system (‘finance 4.0’) is a multi-dimensional feedback, coordination and incentive system based on a participatory use of the Internet of Things that is combined with multiple new currencies. It offers to make sure that people will ‘feel’ the good or bad impacts of their behaviors on others and on nature, thereby helping them to take better decisions. The participatory system rewards people and companies accordingly. It is a democratic system that promotes the co-evolution of symbiotic interactions and the development of a sustainable circular and sharing economy: (Click on image for link to book)


Further reading:

M. Batty (2018) Digital Twins, Environment and Planning B 45, 817-820.

E. Arcaute et al. (2021) Future Cities: Why Digital Twins Need to Take Complexity Science on Board, preprint.

D. Helbing et al. (2021) Ethics of Smart Cities, preprint.




[1] at least with a delay – using time delayed self-decrypting encryption


Tuesday, 20 July 2021

Partial List of Possible Dual Uses of Big Data Cyber-Infrastructures

Big Data and AI are being used to improve security and the state of health, nature, and the world. 

But they can also be used for bad, as the following list demonstrates:


Mass surveillance

Profiling

Scoring

Citizen score

Targeting

Hacking

Cybercrime

Cyberwar

Information war

Hybrid war

Misinformation, also personalized

Disinformation, also personalized

Fake news, deep fakes

Behavioral manipulation

Manipulation of democratic elections

Personalized marketing

Neuromarketing

Censorship

Propaganda

PsyOps

Deception

Mind control

Brain hacking

Hate speech

Blackmailing

Cybermobbing, psychological abuse

Electronic harassment

Social engineering

Insider trading

Market manipulation

Neurocapitalism

Implanting dreams

Triggering accidents

Triggering crime (e.g. “footfall”)

A new kind of police state (“predictive policing”)

Internet of bodies

Potential of totalitarian control


Many of these applications would violate Data Protection Laws, the UN Declaration of Universal Human Rights, or the Nuremberg Codex, i.e. they would be considered crimes against humanity.

The crucial questions are: Can such dual uses be prevented by suitable laws and governance frameworks? 

If yes, how would they have to look like?

Tuesday, 22 June 2021

NAVIGATE THE 21ST CENTURY WITH DIGITAL EMPOWERMENT ON ALL LEVELS AND SCALES

We have recently discussed what kind of tool would be better to have: a crystal ball, which would tell you the future, if it was possible to build, and would allow you to see in all detail what is going to happen. Or, to have a tool that would allow us to manoeuvre troubled waters and to get around the cliffs we are going to face. It is important to realize that these are two pretty different kinds of tools. We have decided it would be desirable to be able to navigate the 21st century with digital empowerment on all levels and scales (particularly of the civil society).

Let me start by stating that, of course, Big Data, AI, and Computer Simulation of Systems have a lot to offer when it comes to addressing some of our biggest challenges of humanity. So, we should, of course, make good use of the opportunities arising from them.

Recently, there is much excitement for digital avatars and digital twins. Some would go so far to aim for an exact digital copy of the real world which would perhaps even replace analogue systems entirely in the future – in accordance with transhumanist goals such as convergence and transcendence. Personally, I consider such goals not scientifically well-founded, so I suggest to focus now on the goal of creating realistic models of parts of the world instead of exact digital copies. These can certainly be useful. Envisioned applications reach from urban twins to models of the planet to models of individuals, their personality, behavior, body, and health. This is of interest in connection with “smart cities”, climate modeling, and precision medicine.

Sounds probably good to you, but things are trickier than expected, not just because of privacy and security concerns. It is not enough to collect as much data as possible and to have a powerful AI system learn the patterns in the data. Besides incomplete datasets and limited processing power, there are a number of fundamental obstacles: biases, overfitting, the role of randomness, turbulence, the “butterfly effect” of chaos theory, the uncertainty principle of quantum mechanics, and the undecidability theorem of Gödel, to mention just a few.

So far, Big Data has not made science obsolete, nor do we have a universal AI. And if we had one, this could still be pretty dangerous. Suppose, for example, we would task an intelligent system to solve the sustainability problems of the planet. It might figure out that the easiest solution would be to kill a considerable number of people. And if it was just smart enough, humans may not even understand what is going on, given that AI systems are pretty much black boxes to them.

In a highly networked, complex world, where almost everything has side effects, feedback effects or cascading effects, a powerful tool, particularly when applied on a global scale, may cause serious, large-scale damage. This is the more likely in a time where people argue with trolley problems and lesser evils. According to this logic, if there is just a big enough disaster, problem, or threat, any ethical principle or law might be overruled, including human rights and even the right to life. In view of the Covid-19 pandemic, we may, in fact, be in the middle of such a scenario already.

Obviously, such an approach can easily and unintentionally end with crimes against humanity. As history shows, neither companies nor states nor legal frameworks are reliable safeguards against terrible misuse, accidents, or side effects. Not even the UN may be enough to ensure a save use of such tools. So, to be able to unleash the full potential of Big Data and AI for humanity and nature, we would need a particular technological framework and an entirely new kind of multi-lateralism. We would need platforms that are neither owned nor controlled by individual companies nor countries nor consortia of them, and would also not stand above our fundamental values and the rule of law.

One would need safety precautions such as decentralization, distributed control, a large amount of informational self-determination, protection from misuse, and compensation if something went wrong. New forms of organizations and social innovation are needed, to unlock the potential of technology safely and responsibly. We certainly can’t afford to open Pandora’s box. We have enough problems already…

Also, we should carefully consider alternative uses of technology. I would just like to mention the idea of creating a socio-ecological finance system. A finance system, which would use the Internet of Things to measure externalities that decisions of people and companies cause. The measurement of externalities would define multiple new currencies, which could be used to incentivize positive behavioral change. This novel real-time feedback and coordination system is inspired by nature, which has already managed to develop a circular economy based on self-organization and distributed control. Hence, introducing such real-time feedbacks into our socio-economic system is expected to create forces promoting a sustainable re-organization. A sustainable circular and sharing economy would result through a co-evolutionary process. It would be a system avoiding coercion and mind control. It would be a system consistent with freedom, self-determination, creativity and innovation, with human rights and democracy. This is actually the path to sustainability I would take. Anyway, these remarks are just to point out that our situation is not desperate. We do have options – but we also have choices to make and action to take.

Monday, 17 May 2021

Angesichts von Triage und „Todesalgorithmen“

Angesichts von Triage und „Todesalgorithmen“:

Ist die heutige daten-getriebene Medizin mit der Verfassung vereinbar?[1]


„Als Mitglied der ärztlichen Profession gelobe ich feierlich, mein Leben in den Dienst der Menschlichkeit zu stellen. Die Gesundheit und das Wohlergehen meiner Patientin oder meines Patienten werden mein oberstes Anliegen sein.

Ich werde die Autonomie und die Würde meiner Patientin oder meines Patienten respektieren. Ich werde den höchsten Respekt vor menschlichem Leben wahren.

Ich werde nicht zulassen, dass Erwägungen von Alter, Krankheit oder Behinderung, Glaube, ethnischer Herkunft, Geschlecht, Staatsangehörigkeit, politischer Zugehörigkeit, Rasse, sexueller Orientierung, sozialer Stellung oder jeglicher anderer Faktoren zwischen meine Pflichten und meine Patientin oder meinen Patienten treten...“
So fordert es das Ärztliche Gelöbnis (die „Deklaration von Genf“ [2]) in seiner neuen Fassung aus dem Jahr 2017, auch bekannt als Hippokratischer Eid. Doch sind die in der Medizin eingesetzten Verfahren überhaupt kompatibel damit – oder droht in unserer Gesellschaft neben einer biologisch-genetischen und sozialen nun auch eine technische Selektion? Angesichts der jüngsten Anwendung von Triage-Methoden sind ernste Sorgen angebracht.

Die Praxis in Krankenhäusern und der Notfallmedizin muss auf den verfassungsrechtlichen Prüfstand. Im Zusammenhang mit Algorithmen-gestützten Sterbeentscheidungen droht eine gefährliche Verantwortungsdiffusion mit unabsehbaren Folgen für unsere Gesellschaft. Zur Erläuterung beschreiben wir im Folgenden vereinfacht einige kritische Elemente der Prozesskette im Gesundheitssystem und wie sie ineinander greifen:

  • Die Realität in Krankenhäusern beinhaltet bekannter Weise Kapazitätsengpässe, wie sie auch in der COVID-19 Krise offenbar geworden sind. Unter solche Umständen kommt es implizit oder explizit zu Priorisierungsentscheidungen. Im schlimmsten Fall können das Triage-Entscheidungen sein, in deren Folge Menschen sterben, die unter normalen Umständen gerettet werden könnten und würden.
  • Dabei werden relevante Vorentscheidungen unter Umständen auch von Personal getroffen, welches die möglichen ethischen und juristischen Implikationen nicht vollständig überblickt.
  • Überdies wird das medizinische Personal bei seinen Entscheidungen zunehmend von Messdaten und Expertensystemen oder gar KI-Systemen unterstützt, was oft sehr hilfreich ist. Bei bestimmten Entscheidungen kann das aber problematisch sein. Solche digitalen Hilfsmittel werden von Softwareentwicklern geschaffen, die von medizinischen, rechtlichen und gesellschaftlichen Implikationen in der Regel nur begrenzte Ahnung haben.
  • Die Algorithmen unterliegen dem Betriebsgeheimnis und werden häufig Updates unterzogen, sind also in ihrer Wirkung bis zu einem gewissen Grade nicht bekannt und mangels Algorithmentransparenz zum Teil nicht überprüfbar. Problematisch ist zudem die „epistemische Autorität“, die algorithmischen Entscheidungen anhaften: eine Befolgung algorithmischer Empfehlungen scheint oft geboten und im Schadens- oder Klagefall eher von Vorteil zu sein.
  • Den medizinischen Entscheidungen werden oftmals Kriterien wie die erwartete Wirksamkeit der fraglichen Maßnahmen zugrunde gelegt. Hierbei ist zwischen verschiedenen Behandlungsmöglichkeiten zu entscheiden, oder ob auf solche verzichtet wird. Die zugrundeliegende Entscheidungen stützen sich oft auf Mittelwerte ab, wie sie in umfangreichen medizinischen Studien erhoben werden. Diese lassen aber in der Regel keine präzisen individuellen Prognosen zu, da die Variation der Daten oft groß ist. Mit anderen Worten: eine Maßnahme, die im Mittel besser ist, kann im Einzelfall oft schlechter sein.
  • Auch Triage-Entscheidungen werden gegebenenfalls auf der Basis von Daten getroffen, die keine exakten individuellen Prognosen zulassen oder für Triage-Entscheidungen ursprünglich nicht gedacht waren.
  • Daten-getriebene Entscheidungen sind oft sehr sensitiv gegenüber Details des verwendeten Algorithmus oder der ausgewerteten Daten. Folglich kann die Auswahl der Patienten, die medizinisch priorisiert oder Opfer der Triage werden, stark mit den verwendeten Prozeduren, Algorithmen oder Datensätzen variieren. Damit sind wir mit einer unerwünschten Zufälligkeit und Willkür resultierender Entscheidungen konfrontiert. Die Objektivität und wissenschaftliche Unangreifbarkeit entsprechender Entscheidungen ist also in Frage zu stellen. Trotz dieser methodischen Mängel werden lebenswichtige Entscheidungen zunehmend auf der Basis von Algorithmen getroffen.
  • Fraglich ist auch die Rechtmäßigkeit und Sinnhaftigkeit der verwendeten persönlichen Daten und Algorithmen. Zum Beispiel spielt in einigen von ihnen die prognostizierte Lebenserwartung eine Rolle,[3] obwohl deren individuelle Variabilität ebenfalls hoch ist.[4] In der Folge werden auf der Basis der Prognose der Lebenserwartung getroffene oder unterlassene medizinischen Maßnahmen in vielen Fällen die Lebenserwartung determinieren. Daher bedingt ein Einsatz solcher Algorithmen weitgehend die Sterbereihenfolge. Die Prognosen würden quasi zur self-fulfilling Prophecy.
  • Bei den medizinischen Entscheidungen spielen auch finanzielle Ressourcen eine zunehmende Rolle. Die Big Data-gestützten medizinischen Entscheidungen sollen für einen effizienten Einsatz verfügbarer Mittel sorgen. Damit verlagert sich aber der Fokus von der Menschenwürde zunehmend zu ökonomischen Nutzenerwägungen. Algorithmen-basierte Priorisierungsentscheidungen, einschließlich Triage-Entscheidungen, werden also zunehmend durch Kostenfragen bestimmt.
  • Wird bei den Entscheidungen überdies der individuelle Versicherungsschutz (der Leistungsumfang der jeweiligen Krankenversicherung) berücksichtigt, so ist auch mit einer sozialen Selektion zu rechnen.[5] Wie man aus der KI-Forschung weiß, können solche Diskriminierungs-Effekte wegen der Intransparenz der Algorithmen und der verwendeten Trainingsdaten selbst dann resultieren, wenn sie nicht beabsichtigt oder explizit einprogrammiert sind.[6]
  • Im Rahmen der gegenwärtigen Pandemie werden Triage-Entscheidungen, in denen es um Entscheidungen über Leben und Tod geht, heute vermehrt getroffen. Sie werden mangels ausreichender medizinischer Kapazitäten oft als unvermeidlich dargestellt. Man darf aber nicht vergessen, dass sie als äußerst problematisch gelten und bisher nur im Krieg und extremsten Katastrophenlagen akzeptabel waren.
  • Dennoch entwickeln verschiedene Projekte Algorithmen, die Triage-Entscheidung treffen sollen, welche angeblich ununterscheidbar von solchen sind, die Ärzte treffen würden. Damit droht eine Mechanisierung des Sterbens.
  • Der Trend geht folglich dahin, dass problematische Entscheidungen an daten-getriebene Algorithmen und Maschinen übertragen werden, also von autonomen Systemen getroffen werden. In manchen Ländern scheinen solche Algorithmen-basierten Triage-Entscheidungen bereits im Einsatz zu sein. Damit wird der Mensch jedoch zum Objekt einer maschinellen Entscheidung gemacht, was mit der Menschenwürde nicht vereinbar ist. Außerdem wird mit der Verantwortungsverschiebung das Prinzip der Accountability untergraben.
  • Die diesen Algorithmen zugrundeliegenden Parameter werden, wie gesagt, direkt oder indirekt auch durch die Behandlungskosten mitbestimmt. Insofern führen politische Entscheidungen über medizinische Kostendeckel zwangsläufig zu Parameteränderungen, die darüber bestimmen, wieviele Menschen letztlich von Triage-Entscheidungen betroffen sind und früher sterben werden, auch, wenn das nicht beabsichtigt war.
  • Neben den Behandlungskosten spielen aber auch Daten zur gesundheitlichen Verfassung einer Person eine Rolle, die zum Teil durch Verhalten, Arbeit und Umwelt, zum Teil aber auch genetisch bedingt sind. Daher ist im Ergebnis zu erwarten, dass die Kombination von Kostenoptimierung und daten-basierter, algorithmen-gestützter Medizin verstärkt zu eugenischen Selektions-Effekten führt.
  • Mit COVID-19 drohen (teil-)automatisierte Verfahren für Sterbeentscheidungen in den Klinikalltag einzuziehen, womit Triage-Maßnahmen nicht mehr auf Kriegs- und Katastrophenfälle beschränkt wären, sondern zum Normalfall zu werden drohen. Durch die zunehmende Automatisierung der Prozesse entziehen sie sich dann zunehmend menschlichen Entscheidungen. In einer nicht nachhaltigen, „überbevölkerten“ Welt, ist zu befürchten, dass dies langfristig nicht gut ausgehen wird.
  • In einigen europäischen Ländern werden Triage-Entscheidungen offenbar bereits seit einigen Jahren zur Optimierung der Krankenhauslogistik und „zur Verbesserung des Patientenflusses“ getroffen. Es ist daher zu befürchten, dass in der medizinischen Praxis geltendes Recht schon seit geraumer Zeit nicht immer korrekt angewandt wird.

Mit Covid-19 ist die mangelnde Nachhaltigkeit unserer Gesellschaft auch in Europa zu einer Frage von Leben und Tod geworden. Die Umorganisation zu einer nachhaltigen Gesellschaft ist damit ein vordringliches politisches Problem, wofür zahlreiche umsetzbare Vorschläge vorliegen.[7] Wir glauben allerdings, dass Triage-Entscheidungen nicht zu einem selbstverständlichen Bestandteil des gesellschaftlichen Lösungs-Arsenals werden dürfen. Hoch problematische Methoden, wie sie sonst nur im Kriegs- und Katastrophenzustand Einsatz finden, dürfen nicht zu einem „new Normal“ werden. Sie sind insbesondere keine akzeptable Lösung für das Problem der globalen Überbevölkerung.

Wir haben oben erläutert, warum im heutigen Gesundheitssystem unerwünschte genetische und soziale Selektionseffekte zu erwarten sind, die durch technische Innovationen weiter verstärkt zu werden drohen. Um diese zu minimieren schlagen wir vor, dass medizinische Einrichtungen regelmäßig statistische Auswertungen über etwaige genetischen oder soziale Selektionseffekte erstellen sollten. Es besteht sonst die akute Gefahr, dass jene Menschen, die medizinische Unterstützung und gesellschaftliche Solidarität am Nötigsten hätten, von einem daten-getriebenen System, wie es sich heute abzeichnet, besonders benachteiligt werden.

Unserer Einschätzung nach erfüllen die derzeitigen daten-basierten Ansätze die verfassungsrechtlichen Anforderungen (Gleichheitsgrundsatz, Selbstbestimmung, Recht auf Leben), Wissenschaftlichkeit (angesichts der Sensitivität der Verfahren) und politische Ziele (wie Gerechtigkeit) heute nur unzureichend. Der Einsatz von Künstlicher Intelligenz und anderen digitalen Innovationen sollte aber so erfolgen, dass er fair ist. Andernfalls ist zu befürchten, dass der demokratische Gleichheitsgrundsatz zunehmend von diskriminierenden Scoring-Systemen ersetzt wird, was letzten Endes zu einer fundamental anderen Gesellschaft führen würde.

Obwohl verfassungsgemäße und faire daten-gestützte Methoden denkbar wären, scheinen die derzeit verwendeten Verfahren diesen Anforderungen nicht immer zu genügen. Nimmt man den Gleichheitsgrundsatz ernst, demzufolge kein menschliches Leben mehr als ein anderes zählen darf, dann müssten die Segnungen der modernen Medizin allen Menschen gleichermaßen zu Gute kommen. Dieses Fairnessgebot impliziert, dass gesundheitlich benachteiligte Menschen mit einer kürzeren Lebenserwartung pro Lebensjahr auch in Zukunft eine höhere Unterstützung erwarten dürfen. Das erfordert jedoch eine andere Nutzung daten-gestützter Methoden.

Statt dem oben beschriebenen System wäre beispielsweise ein Gesundheits- und Sozialsystem denkbar, in dem jeder Mensch eine bestimmte Anzahl von „Vouchern“ bekommt. Diese könnte man zum Einsatz bringen, wenn um es Entscheidungen geht, die für das eigene Leben individuell besonders wichtig sind. Insbesondere könnten sie genutzt werden, um algorithmen-basierte Entscheidungen zu „übertrumpfen“. Damit würde vermieden, dass Menschen zum Objekt von Algorithmen-Entscheidungen werden, und es entstünden neue Entscheidungsfreiräume im Sinne der Menschenwürde.

Im Laufe eines Lebens nicht genutzte Voucher (und die damit verfügbar werdenden Ressourcen) könnten zusätzlich für Personen eingesetzt werden, die besonders hilfsbedürftig sind. Dies soll nur ein Beispiel dafür sein, dass auch in einer Welt mit Ressourcen-Knappheiten individuelle Selbstbestimmung und soziale Verantwortung realisierbar sind, die in einem scheinbar optimalen, daten-getriebenen Gesundheitssystem heute wenig Platz haben.

Engpässe im Gesundheitssystem sind weder Naturgesetz noch gottgegeben, sondern ein menschengemachtes systemisches Problem. Da es hier um Leben und Tod und um fundamentalste Grundrechte geht, handelt es sich um eine zentrale, vielleicht sogar die Wichtigste aller politischen Fragen im Zusammenhang mit der digitalen Gesellschaft. Will die Politik ihrer Aufgabe gerecht werden und ihre Legitimität sicherstellen, muss sie solchen Fragen wesentlich mehr Platz einräumen. Wir sind zuversichtlich, dass eine breite gesellschaftliche Diskussion dieser Fragen zu den Einsichten, Lösungen und systemischen Veränderungen führen wird, die in einer lebenswerteren Gesellschaft und besseren Zukunft resultieren, wie wir sie uns alle wünschen.


Dirk Helbing ist Professor für Computational Social Science am Department Geistes-, Sozial- und Staatswissenschaften der ETH Zürich und am Department of Computer Science assoziiert. Seine aktuellen Studien diskutieren global vernetzte Risiken und die digitale Gesellschaft. An der TU Delft leitet er das Doktorandenprogramm "Engineering Social Technologies for a Responsible Digital Future". Er ist zudem gewähltes Mitglied der Deutschen Akademie der Wissenschaften „Leopoldina“.


Thomas Beschorner ist Professor für Wirtschaftsethik und Direktor des Instituts für Wirtschaftsethik der Universität St.Gallen. Zu seinen Arbeitsfeldern zählen vielfältige Fragenstellungen im Bereich der Unternehmensverantwortung und im Bereich einer Ethik der Digitalisierung.


Bruno Frey ist Ökonom und Ständiger Gastprofessor an der Universität Basel, wo er das Center for Research in Economics and Well-Being (CREW) leitet. Er ist außerdem Forschungsdirektor des Centers for Research in Economics, Management and the Arts (CREMA).


Andreas Diekmann ist Seniorprofessor an der Universität Leipzig und Emeritus der ETH Zürich. Er lehrt an den Universitäten Leipzig und Zürich und befasst sich in seinen empirischen Forschungen mit digitalen Märkten und experimentellen Studien zur sozialen Kooperation. Auch er ist Mitglied der „Leopoldina“.


Thilo Hagendorff ist Post-Doc im Exzellenzcluster „Machine Learning“ sowie Projektleiter am Internationalen Zentrum für Ethik in den Wissenschaften an der Universität Tübingen. Sein Forschungsschwerpunkt liegt in der KI-Ethik.


Peter Seele arbeitet als Professor für Wirtschaftsethik an der Universität der italienischen Schweiz, USI Lugano. Seine Forschungsschwerpunkte sind Greenwashing, Maschinewashing, und die Maschinisierung von Mensch und Gesellschaft.


Sarah Spiekermann leitet das Institut für Wirtschaftsinformatik und Gesellschaft an der Wirtschaftsuniversität Wien. Sie ist als Buchautorin zur digitalen Ethik bekannt und als Expertin zu den Themen digitale Privatsphäre, Überwachungskapitalismus und Wertephilosphie. Überdies ist sie Vizevorsitzende des IEEE P7000 Standards, der einen Prozess für ethisches Systemdesign definiert.


Jeroen van den Hoven ist Ethik-Professor an der TU Delft, wo er am Thema „werte-sensitives Design“ arbeitet. Er sitzt in verschiedenen hochkarätigen Ethikgremien und ist Chef-Herausgeber des Journal of Ethics and Information Technology. Seine Kompetenz in schwierigen ethischen Sachfragen ist vielerorts geschätzt.


Andrej Zwitter ist Professor für Governance and Innovation und Wissenschaftlicher Direktor des Cyan Centre on Climate Change Adaptation sowie Dekan der Fakultät Campus Fryslan der Universität Groningen. Er ist Experte im Bereich Technology Governance und ist für seine Publikationen zu digitaler Ethik und Notstandsrecht bekannt.

[1] Dieses Policy Paper wurde am 6. Mai 2021 durch die ZEIT Online in einem Artikel gefeatured:

https://www.zeit.de/digital/2021-05/triage-software-notfallmedizin-algorithmus-kuenstliche-intelligenz-ethik

[2] https://www.bundesaerztekammer.de/fileadmin/user_upload/downloads/pdf-Ordner/International/Deklaration_von_Genf_DE_2017.pdf

[3] https://daserste.ndr.de/panorama/archiv/2017/Der-Todesalgorithmus-Computer-berechnet-Lebenserwartung,todesalgorithmus112.html

[4] Insbesondere variiert die mittlere Lebenserwartung in verschiedenen Ländern beträchtlich. Wird ein solcher Algorithmus also mit internationalen Daten gefüttert, besteht die Gefahr lebensverkürzender Entscheidungen in Ländern mit einer eigentlich überdurchschnittlichen Qualität des Gesundheitssystems.

[5] https://www.tagesanzeiger.ch/wo-man-versichert-ist-kann-ueber-leben-oder-tod-entscheiden-752186910381

[6] Nicht selten kommt es vor, dass ethnische, soziale, genetische und Verhaltens-Faktoren miteinander korrelieren.

[7] Insbesondere wurden von den Unterstützern dieses Textes selber zahlreiche innovative Lösungsansätze vorgeschlagen.