Tuesday, 19 January 2016

NERVOUSNET - Towards an open and participatory, distributed big data paradigm

by Dirk Helbing (ETH Zurich and TU Delft) and collaborators

Motivation: As the development of the Internet of Things (IoT) is taking up speed, connected devices produce staggering amounts of data. Estimates by Cisco say that by 2020, there will be 7 times more connected things than people – devices which will produce hundreds of zettabytes of data every year. Moreover, according to IBM, data volumes will soon double every 12 hours, not just every 12 months. Then, it is neither possible to store all the data nor to transmit it all for centralized processing and optimization. Therefore, we expect a new, complementary big data paradigm to emerge. This will involve decentralized approaches, data aggregation, and "optimal forgetting" of data, also for reasons of costs, scalability, and security.

As the connectivity in today's techno-socio-economic systems increases steeply, systemic complexity does so, too. Taken together, this will exclude a centralized optimization of societal-scale systems, particularly due to the NP-hardness of some optimization problems and the limitations of predictability and calibration (such as over-fitting problems and parameter sensitivity). However, highly performing solutions can often be realized in real-time by means of flexible and adaptive, decentralized heuristics. These specify self-organizing systems. All that matters to reach a desired systemic outcome efficiently is a suitable choice of the interactions between the system components. The scientific disciplines of mechanism design and complexity science serve to identify these interactions.

Decentralized solutions also leave space for diversity in the locally applied goal functions. This allows for innovation and solutions that fit the respective context and local culture best. Diversity is also often favourable for innovation rates, for societal resilience to disruptive events, and for collective intelligence. For these and further reasons, decentralized and pluralistic systems can be superior to systems lacking diversity. (Note that, in biological systems, the value of diversity is already recognized since a long time.)

Managing the future data deluge wisely is currently one of the biggest challenges facing policy, industry and civil society. The project proposed here will study how big data from connected devices can be used to build a so-called Planetary Nervous System. We envision a transparent, open-access information system, which can support crowd-sourced real-time measurements of the world. A system like this could revolutionise many sectors, from urban planning and traffic control to the early detection of epidemics. Compatible with IBM's IoT paradigm of "device democracy", we propose a citizen-run participatory platform. The platform aims to have extensive features to protect user privacy and create big data as a public good. The goal is to build a participatory data ecosystem complementing classical big data approaches and to offer opportunities for everyone: science, business, politics, and the citizens.

Historical development and state of the art: In 1943, IBM chief Watson expected that there is a world market for maybe 5 computers. However, everything changed when computers were connected with each other. ARPANET networked a couple of military computers to make the USA less vulnerable to nuclear attacks, using the principle of decentralization. Soon after this, the Internet became accessible for everyone, and with the invention of the http protocol, Tim-Berners Lee made it possible to easily connect webpages with each other. This created the World Wide Web (WWW) and made it useful for non-experts, which was the precondition for the emergence of a multi-billion-dollar digital market. With social media, the Internet reached out for people, who became nodes of a global information processing system. Now, things are connected to the Internet too, creating the Internet of Everything. It is predicted that, by 2025, 150 billion sensors will be wirelessly connected to the Internet (of Things), which makes it possible to give objects senses and, together with actuators and machine learning software, even a certain level of artificial intelligence (AI).

This allows one to create a digital nervous system on a planetary scale, i.e. a global Internet-of-Things-based information system with an intelligent, learning software layer on top. The question is how to design and operate such a system to support a thriving economy and society. Answering this question requires not only knowledge from computer science and electrical engineering, but also knowledge from the social sciences and complexity science, to ensure a systemic view. Here, we propose the Nervousnet project to develop such an integrated solution.

Project overview: Unlike most Internet of Things initiatives spearheaded by big IT companies or public institutions, Nervousnet shall be run as a 'Citizen Web', built and managed by its users. Inspired by Wikipedia and OpenStreetMap, people, companies and devices will be able to interact with Nervousnet in 3 ways: by contributing data; by analysing the crowd-sourced datasets; and by sharing code and ideas. Anyone should be able to create data-driven services and products using a generic programming interface. The aim is to yield societal benefits, business opportunities and jobs.

Today, there are several Internet of Things platforms and data science projects that share Nervousnet’s vision, but none has its scope.

These projects focus on participatory data collection (http://funf.org, http://www.kaaproject.org; http://thethingsnetwork.org/, or http://www.opensense.ethz.ch/trac/);

decentralized communication services (http://maidsafe.net/, https://www.ethereum.org, http://dcentproject.eu);

or big data analytics:
(www.socialsensor.eu, http://datalook.io, https://www.amigocloud.com, http://stratosphere.eu, http://gdeltproject.org). Nervousnet aims to meet all three objectives

Moreover, Nervousnet wants to enable real-time measurement and feedback to support self-organising systems. For example, self-controlled traffic lights responding to local pedestrian and vehicle flows can reduce urban congestion and outperform today's systems based on centralised control. The ultimate challenge will be to digitally enable collective intelligence. Digital assistants can help bring knowledge, ideas and resources together. A pluralistic approach to information processing is important to view complex problems from varied perspectives and also to anticipate and assess rare and extreme events that are costly for society – such as natural disasters, blackouts or financial meltdowns.

Nervousnet uses distributed data storage and distributed control, so that it is more robust to attacks and centralized manipulation attempts, easy to scale up, and tolerant to faults. Nervousnet's approach is compatible with the principles of informational self-determination and, according to our judgment, also with the new EU Data Protection Directive.

Attracting users is a further challenge, which will be addressed by 'gamification' and a micro-payment system to reward and incentivise digital co-creation. As critics may worry about the responsible use of bottom-up systems, Nervousnet also aims to integrate reputation systems, qualification mechanisms, and self-governance by community moderators ("social technologies").

In the long run, we expect that measurements tailored to specific purposes, together with crowd-sourced data generation, curation and analysis may outperform the big data analytics approach currently in vogue. Just as the open standards of the World Wide Web created unprecedented opportunities and a multi-billion-dollar economy, the right framework for the 'Internet of Things' and digital society could foster an age of prosperity.

Project components: To provide the full perspective of the project, which may be realized long-term with complementary resources, we give an overview of the main envisaged platform components and functionalities in the following. 

Figure: Schematic illustration of some core components of the Nervousnet concept.

In the long run, the Nervousnet platform (to run on several mobile platforms such as iOS and Android, but also on embedded devices such as Arduino) will comprise a considerable number of functional elements:

Real-time measurement: The Nervousnet platform opens up various sensors for real-time measurement of the world around us (status: done). This will enable the measurement of "externalities" such as social or environmental impacts of interactions between systems and system components (e.g. of noise or other emissions). We plan to make it easy to add further, external sensors, for example, for smart home applications, using Arduino and Raspberry Pi platforms. [Has already been internally tested, needs to become easy to configure and use]

Anonymization: Data streams are not linked to personal or smartphone identifiers, but to a randomized identifier that can be specified to change over time. [Done]

Security: Data streams, communication and data sources (smart devices/their owners) shall be secured by state-of-the art encryption. Data shall be decentrally stored to reduce possible impacts of attacks. [Implementation in progress, will be further extended]

Privacy: The GPS sensor is not made accessible or turned off by default. The smartphone's microphone is not made accessible as a microphone. Its signal is turned into a virtual noise sensor by means of a digital filter. A similar strategy is applied to other sensitive sensors. [Implemented and on-going, see also "Data aggregation"]

Informational self-determination: Users can easily turn data collection through the Nervousnet platform on or off and regulate the frequency of data collection. Furthermore, they can determine separately for each virtual sensor, whether data will be logged only locally for one's own use (e.g. in smart home applications) or shared with others. [Implemented]

Decentralized data storage and processing: Nervousnet is based on distributed storage to make the platform resilient to attacks and centralized manipulation attempts, easy to scale up, and tolerant to faults. User groups can decide to join the Nervousnet community and share their data with it or decide to run their own data server. [Functional, under further development]

Forgetting: Most of the measurement data are needed for real-time feedback (as they are needed to enable self-organizing systems). Users will be able to specify a time period after which data will be non-decryptable or automatically deleted (at latest). On top of this, Nervousnet aims to determine strategies for optimal forgetting (the usefulness of most data decays quickly with time). [To be implemented]

Data aggregation ("global analytics engine"): Before their deletion, data may be aggregated to provide information such as minimum and maximum values, averages, standard deviations, or other relevant statistical quantities to provide information at various levels of granularity without revealing personal data. This is being done by a "global analytics engine". The particular novelty of our approach is that it can aggregate data in a fully decentralized fashion, even when input data changes over time [Feasibility has been successfully tested; first version available soon]

Local analytics engine: Data logged locally can be accessed for providing data-driven services to Nervousnet users. The data are managed and accessed transparently via a high-level application programming interface that hides low-level technical details. [First version exists, will be further extended]

Pluralistic data processing: Users can also process a certain volume of shared Nervousnet data for free for own applications, such as services or games. They can run their own data aggregation and visualization algorithms, such that their view of the world is not determined by others, but self-determined. [Must be made user-friendly.]

Data visualization: The Nervousnet platform will offer various possibilities to visualize data. For this, we intend to link the platform with an open source visualization platform that runs on various devices, including smartphones, such as the D3 JavaScript platform. [Feasibility has been tested; to be developed further]

Personal data store: Sensitive personal data is only stored locally on the own smartphone. A personal data store shall make it simple to administer the data according to various categories and determine whom to give access to what kind of data for what kind of purpose and period of time (e.g. for use by a bank or other company). We might integrate OpenPDS developed at the MIT or other open-source software such as digitalID. Transactions may be supported by a AI-based digital agent that learns the user preferences locally on the smartphone. [To be done.]

Personalized services: The personalization of services will be enabled by means of a matching principle. The offers by a provider of data, services or products can be customized using data from the personal data store, without revealing the identity of the customer or user (if they don't want to). [To be done.]

Social technologies: The Nervousnet platform will be equipped with social mechanisms (such as communication or social networking) in order to create social opportunities. This will consider knowledge from game theory, mechanism design, and complexity science such that the implemented mechanisms support coordination, cooperation, and responsible use. [To be done.] 

Digital assistants: AI, machine learning and other approaches (such as recommendations and best practices from experts and users) will be employed to offer assistance to users. This will be done in a distributed way (e.g. on the user's smartphone) and only if explicitly agreed by the respective user. [To be done.]

Creating a Participatory Information and Innovation Ecosystem Open source and open innovation: Most computer code of the Nervousnet platform (not necessarily though the code of potentially commercial apps running on top of it) is made open source to allow experts to check its security and functionality. This creates trust and also participatory opportunities to contribute to the further development of the platform and its functionality. [This is our policy; a dual licensing model is being considered]

Crowd-sourcing and participatory opportunities: Users can contribute data, analyse the crowd-sourced data, contribute, use and modify code (e.g. new measurement methods, "virtual sensors"), or release apps using Nervousnet data. Therefore, anyone can create data-driven services and products using a generic programming interface. [Must be made user-friendly]

Reputation and incentive systems: In order to reward users for contributions (for data, code, and apps, or also for social, healthy or environmentally friendly behaviour...) and to assess the quality of data and services, multi-dimensional reputation and incentive systems will be designed and developed for the Nervousnet platform to provide a differentiated, context- and community-specific feedback. The incentive system will also be used to promote responsible use of the Nervousnet platform. [To be done.]

Collective intelligence and digital democracy: It is also planned to develop an online deliberation platform, which allows one to collect and integrate the knowledge and ideas of many people in order to support better solutions to problems that need to be solved. [To be done.] 

Citizen and Business Engagement: One goal of the project is to reach citizen engagement with digital technologies, as this is important for the digital transformation of our society to succeed. For this, participatory opportunities and open innovation are as important as incentive systems and gamification (and a collaboration with companies and public media).

Treasure Hunt: This app allows to localize "Treasures" equipped with Bluetooth beacons, turning the smartphone into a kind of radar system measuring proximity. [App exists, must be integrated into the Nervousnet platform]

Competition Game: This app allows one to perform group competitions using the output of selected virtual sensors (measurement of acceleration, noise or distance). For example, one could perform a virtual arm wrestling or a biking competition. [App exists, must be integrated into the Nervousnet platform]

Swarmpulse: This app allows one to perform geo-located measurements of light or noise intensities and create corresponding maps. It is also possible to leave geo-located text messages, which could be links to photos or movies, thereby allowing people to map the environment around them. [To be done.] 

Disaster response and societal resilience: This app should provide functionality in case the mobile phone network is broken down, offering information exchange via ad hoc network protocols such as the one used by "firechat". The app should also offer simple local coordination of supply and demand between peers – something like a sharing economy functionality for disasters. [To be done.] 

Early warning system: Advance detection of impending risks with predictive analytics and methods from complexity and network science. [To be done.] 

Usefulness for policy decision-making: Nervousnet and apps running on top of it (particularly Swarmpulse) will be able to engage with citizens and to collect data on the environment (allowing one to generate, for example, noise maps, maps of plant and animal species, etc.).

Invitation: Everyone who is motivated and qualified to contribute to the Nervousnet platform or Apps is invited to contribute. We are reaching out to international academic institutions, but also to coding, gaming and fablab communities to establish international Nervousnet hubs (see nervousnet.info). You can also apply to join the PhD class on „Engineering Social Technologies for a Responsible Digital Future“ at TU Delft with your own funding, e.g. a stipend, see http://www.tbm.tudelft.nl/nl/onderzoek/engineering-social-technologies-for-a-responsible-digital-future/ for details. In case of interest, please send an email to nervousnet@ethz.ch

D. Helbing and E. Pournaras, Build Digital Democracy, Nature 527, 33-34 (2015): http://www.nature.com/news/society-build-digital-democracy-1.18690

E. Pournaras, J. Nikolic, P. Velasquez, M. Trovati, N. Bessis, and D. Helbing, Self-regulatory information sharing in participatory social sensing, preprint (2015).

E. Pournaras, M. Warnier, and F.M.T. Brazier, A generic and adaptive aggregation service for large-scale decentralized networks, Complex Adaptive Systems Modeling 1: 19, 1-29 (2013).

E. Pournaras, I. Moise and D. Helbing, Privacy-preserving ubiquitous social mining via modular and compositional virtual sensors, in: Proceedings of the 29th IEEE International Conference on Advanced Information Networking and Applications-AINA-2015 (Gwangju, South Korea, March 2015), pp. 332-338.
D. Helbing, Interaction Support Processor (2015) https://patentscope.wipo.int/search/en/detail.jsf?docId=WO2015118455

No comments:

Post a Comment

Note: only a member of this blog may post a comment.