Table Of ContentFree University Brussels
FACULTY OF LAW
Research group on Law, Science, Technology & Society
(LSTS)
GOA PROPOSAL
Law and Autonomic Computing: Mutual
Transformations
Promotors : Post-Doc Researchers
Prof. Dr. S. Gutwirth (spokesperson) Dr. M. Hildebrandt
Prof. Dr. P. De Hert Drs. L. Desutter
th
May 2 2006
2
I Research Proposal General Framework: .....................................................................3
I.A Autonomic Computing: Three Scenarios for an Unknown Future......................3
I.B What law got to do with it....................................................................................5
I.C Mutual transformations: Legal Subjectivity and Ambient Law ..........................7
I.D. Pertinence of the research...................................................................................8
II Research Design.........................................................................................................9
II.1. Mutual transformations: Autonomic computing and legal subjectivity............9
II.1.A. The central position of the human subject in law, in a constitutional
democracy ..............................................................................................................9
II.1.B Task 1: Exploration of the impact of autonomic computing on the central
position of the human subject in law, in a constitutional democracy ..................10
II.1.C. Tasks 2 and 3: Privacy, Data Protection and Intellectual Rights..............12
II.1.D. Task 4: Legal subjectivity for transhumans and nonhuman consciousness?
..............................................................................................................................15
II.2. Mutual transformations: Autonomic computing and ambient law...................16
II.2.A. Task 5: Regulation of human behaviour in an AmI environment............17
II.2.B Task 6: Ambient law and the 'ecology of practices' ..................................18
II.4 Methodology of the research.............................................................................19
II.4.A Classical methodologies of the relevant disciplines..................................19
II.4.B Generic vocabulary and speculative thinking............................................19
III Work Distribution and Output ................................................................................20
IV Personnel.................................................................................................................23
V References................................................................................................................24
VI. Short description of LSTS and list of relevant running projects ...........................30
VI.A. Short description of LSTS .............................................................................30
VI.B. List of relevant running projects in LSTS......................................................30
VII Curriculum Vitae applicants & postdoctoral researchers......................................32
VII.1 Serge Gutwirth ...............................................................................................32
VII.2 Paul de Hert ....................................................................................................60
VII.3 Mireille Hildebrandt .......................................................................................86
VII.4 Laurent de Sutter ............................................................................................92
3
I Research Proposal General Framework:
I.A Autonomic Computing: Three Scenarios for an Unknown
Future
Ambient Intelligence and Autonomic Computing
1
In the 2005 Report The Internet of Things (ITU 2005), we are confronted with a
detailed description of a world in which all things are wirelessly connected. Any
movement or change is registered, allowing a permanent real-time monitoring that
should enable the environment to adapt itself automatically to our inferred
preferences. We are invited to imagine a world in which ubiquitous computing is no
longer restricted to a proliferation of desktops or visible microprocessors, but
embedded in a way that takes the technology out of our sight while providing us with
unubtrusive and user friendly human-machine interfaces (HMI’s). In fact, these
2
HMI’s may even be implanted into our bodies, like the RFID-tags now used for
livestock supply chain management, or the human implants used in healthcare
institutions in the US. If ubiquitous computing meant connectivity anytime, any place
for anyone, it will now come to anything, thus turning our offline world online.
The Internet of Things seems to describe in more detail the Vision of Ambient
Intelligence, initiated by Philips (Aarts and Marzano 2003) and the European
Commission (ISTAG 2001) in recent years. Ambient Intelligence means an
environment in which human beings are surrounded by pervasive, ubiquitous and
interconnected computing devices that anticipate their preferences in order to adapt
the environment to their inferred wishes. Ambient Intelligence is directed to user
friendly human machine interfaces to such an extent that the environment itself
becomes the interface (cf. SWAMI Deliverable 1, 2005)
The most promising and challenging aspect of Ambient Intelligence and the Internet
of Things is the fact that it will run on the basis of autonomic computing. In 2001 Paul
Horn, IBM’s senior vice president, introduced the idea of autonomic computing, a
term that refers to our autonomic nervous system. He chose this term, because the
autonomic nervous system ‘governs our heart rate and body temperature, thus freeing
our conscious brain from the burden of dealing with these and many other low-level,
yet vital, functions’ (Kephart and Chess 2003). The Ambient Intelligent environment
– which should emerge if the Internet of Things catches on – will do the same for us:
providing a technological infrastructure that permanently adjusts the environment to
our needs without bothering us with the enormous amount of decisions that have to be
taken to allow such continuous adaption. Autonomic computing implies the collection
and storage of our personal and many other data, to be mined for expected and
unexpected patterns. This will allow the environment to anticipate our needs, desires
and preferences. Most importantly, it will enable real time profiling and autonomic
decision making by the machines that will ‘run’ our networked environments for us.
1
ITU stands for International Telecommunications Union
2
In FIDIS we cooperate with Kevin Warwick, who has initiated experiments with human implants and evaluated
some of the ethical implications, see Warwick 2003.
4
The technologies that will enable such autonomic adaptive environments are
described as RFID-systems, biometric applications, sensors, miniaturization and
nanotechnologies (ITU 2005). The wireless connections between them, online data
bases and available processing power will see to it that these networked environments
will start learning about us, anticipating our worries and possibly solving our
problems before they have even occurred (to us). This is often referred to as machine
intelligence, emerging not in separate machines, but out of the interconnections
between things, sensors, online databases and processing power. Such networked,
distributed intelligence may be based on algorithms, which means that in theory the
autonomic decision making process is still predictable. However, in practice, the
complexity of the software and the information to be processed, makes it virtually
impossible to check what the system will come up with. On top of that, further
development of neural networks and nanotechnologies will produce a type of
intelligence that is no longer predictable, and therefore able to find solutions that
human experts may not have discovered, yet (Kephart & Chess 2003).
Ambient Intelligence, GRIN Technologies and Autonomic Computing: Three
scenarios
Ambient Intelligence is a vision on the role of emerging technologies, like the so-
3
called GRIN's (Genetics, Robotics, Information and Nanotechs), which have seen a
drastic acceleration in the last two decades. Autonomic computing, which is a
precondition for Ambient Intelligence, nourishes on these technologies. Without the
exponential advances made in the field of GRIN technologies autonomic computing
would not stand a chance and Ambient Intelligence would remain science fiction.
All these interrelated developments have given birth to the parrallel development of a
host of questionings around the meaning of the spectacular technological
transformation of what Latour would call our common world (Latour 1999). These
questionings have been articulated in three different scenarios (Garreau 2005). The
first has been called the ‘Heaven Scenario’. Following this scenario, the development
of GRIN technologies will give to human beings the possibility to fullfill their dreams
way beyond their wildest expectations: immortality, collective intelligence,
transcendance through machines, etc., will all become reality – a reality that will
enable humanity to take a step further into its political and social unification
(Kurzweill, 1990, 1999). The second scenario has been called the ‘Hell Scenario’.
Following this second scenario, the development of GRIN technologies will, on the
contrary, drive humanity straight into a nightmarish world. In this world, there will
not be one humanity anymore – but a strict divide between those who can afford the
benefits of GRIN technologies (‘Enhanced’), and those who cannot or will not
employ them (‘Naturals’). This divide will multiply the tensions both inside and
outside societies – leading to new forms of discrimination and war (Joy, 2000). The
third scenario has been called the ‘Prevail Scenario’. Following this scenario, it is not
the development of GRIN technologies that will drive humanity to new places
(Heaven or Hell), but the interaction between humans and GRIN technologies. Other
than the other two scenarios, the ‘Prevail Scenario’ does not consider the development
of technology a stronger power than the power of human beings to discriminate
3
Other acronyms have been used, such as NBIC (nanotechnology, biological engineering, information technology,
and cognitive engineering), GRAIN (genetics, robotics, artificial intelligence, and nanotechnology), GNR
(genetics, nanotechnology, and robotics), etc.
5
between - and act upon - what they consider desirable or not (Lanier, 2000, 2004).
While the first two scenarios seem to imply some form of technological determinism,
the third seeks to articulate the possibility for human freedom (and responsibility).
These three scenarios, describing the consequences of the current development of
GRIN technologies share certain features:
- Firstly, they all assume that the key for the development of GRIN technologies lies
in the I of GRIN: in Information. It is the development of information technologies
that allows the parallel development of genetics, robotics and nanotechs.
- Secondly, they all assume that the development of technologies is never only a
scientific question: it is a question that implies political and societal consequences.
What differentiates one scenario from another is precisely the fact that these
consequences – and not the technologies by themselves - will lead humanity into one
direction or another.
- Thirdly, they all assume that, as scenarios, what they describe cannot be considered
as what is commonly called ‘science-fiction’. Scenarios differ from science-fiction
because of the fact that they use imagination within a framework of which the
constraints are different: for science-fiction, imagination is a way to reach the
possible, without being contrained by what is probable; for scenarios, it is a way to
detect both the probable and the possible. This means that the researchers that have
given birth to the Heaven, Hell and Prevail scenarios, are researchers who claim that
the consequences they describe in part depend on very good present reasons (Garreau
2005). The fact that they are about probable and possible futures, ‘imaginative stories
about what the future might be like (…) not predictions’(Garreau 2005: 78), does not
mean we should not take them serious. On the contrary, they allow us the kind of
speculation we need in a world that is predictable only to a limited extent. This
4
necessitates an imaginative anticipation of the foreseeable futures.
- Fourthly, they all nourish autonomic computing and, at the same time, depend on its
widespread implementation. Ambient, transhuman and nonhuman intelligence will
depend on the further exponential development of GRIN technologies that enable
autonomic computing on an unprecedented scale. At the same time ambient,
transhuman and nonhuman intelligence will nourish autonomic computing with vast
amounts of data and the ability to process them in real time.
I.B What law got to do with it
The Heaven, Hell and Prevail scenarios all assume that law will be transformed by the
development of GRIN technologies. The described developments will not only lead to
the creation of new legal fields, but they may also lead to an alteration of the
foundations of law. If Roman law was based on family, Medieval law on religion and
Modern law on property – what will a legal system that has to take into account the
distinction between ‘Enhanced’ and ‘Natural’ be about? As the three scenarios are not
science-fiction, this question is not juridic-fiction: from the point of view of both the
Heaven and Hell scenarios, it will very practically be impossible to continue to
maintain law as it is, to the extent that the legal objects as well as the legal subjects
may be completely transformed. How to continue to elaborate general and abstract
4
In the Safeguards in a World of Ambient Intelligence (SWAMI)-project we have experienced that working with
scenario’s, in that case ‘dark scenario’s’, is a very fruitful way of anticipating legal problems and solutions. See
(SWAMI D1. & D.2., 2005).
6
norms when humanity is divided between ‘Enhanced’ and ‘Natural’? How to continue
to use categories such as property or originality when there will be collective
intelligence (Lévy 1994 & 1997a) ? How to continue to consider law as concerning
humanity-only when the intelligence of the machines will make them more than
human? But not only do the Heaven and Hell scenarios raise such questions: so does
the Prevail scenario. Following the Prevail scenario, the type of control over
technology that humans will begin to develop in order to choose between the probable
and possible futures that are offered, may exceed substantively the type of control that
is now offered for instance by intellectual property rights or by the fundamental rights
attributed to the citizens of democratic constitutional states. In order to stop the
investigating agents of a state department to exceed their competences, new legal
means may for instance have to be invented.
The Heaven and Hell scenarios both assume that the role of law regarding the
consequences of the development of GRIN technologies will be passive. From the
point of view of the Prevail scenario, however, law should play an active role in
assessing these consequences. But whether active or passive, the role of law in the
assessment of the consequences of GRIN technologies will necessarily lead to its
transformation. Following the three scenarios, the current state of law and of legal
concepts seems unable to meet the challenge represented by GRIN technologies for
issues like legal subjectivity, agency, causality and liability. It is necessary already
today, to imagine what kind of legal means and what kind of legal concepts must be
invented, amended or put aside, for a scenario to develop its plot. Does this mean that
law has nothing to say in the choice of scenario? The first bet of the present project is,
on the contrary, that the voice of law will be crucial in the realisation of a scenario. To
define the relationship between law and GRIN technologies in terms of passivity
(following the lead of technology) or activity (co-shaping our common world) is to
commit ourselves, as lawyers, into a debate on the role of law. This debate is not free
of constraints because a choice between activity and passivity has take into account
the consequences of that choice. The second bet of the present project is that to
choose for activity is the only choice that could be faithful to what law is made for,
whatever its foundation is (Gutwirth, 1993 & 1999). This bet is a bet in favour of the
Prevail scenario, because it is the only one that integrates this faithfulness into its own
plot.
This means that law is not at the passive end of the transformations that may occur
regarding, for instance, our subjectivity. Lawyers need to anticipate these
transformations and reconstruct the legal framework. Instead of passively adapting to
the demands of technological progress, the law needs to responsively fit the demands
of a changing environment, while remaining true to the obligations lawyers consider
crucial, even if these obligations may have to be re-configured to face the challenges
detected in the set of scenarios described above. At the same time, lawyers cannot
assume that technology will be at the passive end of the transformations they invent.
This project aims to prepare active and mutual reconfigurations between the practice
of law and the practices of technologists in the broad sense (computer scientists,
engineers, mathematicians, data base experts, etc.). As the LSTS has been involved in
a variety of relevant research projects (see section VI.B), which have confronted their
researchers with the necessity of such mutual transformations, it can provide its
lawyers, legal theorists and legal philosphers with the active context in which the
ambition of this project can be succesfully pursued.
7
I.C Mutual transformations: Legal Subjectivity and Ambient
Law
Autonomic computing is what all three scenario feed on and produce. However, the
way autonomic computing is confronted differs. The first two scenarios embrace
optimistic or pessimistic technological determinism, which means they will take
autonomic computing as a given (for the better or the worse). The third scenario will
have to face the challenges of ambient, transhuman and nonhuman intelligence,
without taking them for granted. Obviously, the possible legal implications are
manyfold. In this project we will focus on two sets of implications that demand close
attention. These will structure the two parts of his research project.
Firstly, we will assess the impact of autonomic computing on the kind of human
subjectivity that is both presumed and protected by law, within the framework of
constitutional democracy. To what extent does autonomic decision making on the part
of machines or human-machine hybrids impact the freedom of a human person to act
autonomously (Bohn, Coroama et al. 2004)? How will this impact our privacy
(Solove 2004)? How will autonomic decisions taken by interconnected ‘intelligent’
machines impact the attribution of causality in cases of tort, breach of contract or
criminal responsibility, how will it impact accountability and liability in private law,
in administrative law and in criminal law? What happens if not only the critical
infrastructure of the adaptive environment runs on autonomic decision-making, but if
human persons use intelligent devices like personal digital assistents to delegate a host
of decisions to them (Clarke 1994)?
Secondly, after having assessed the possible impact, we turn to the issue of
regulation. We will investigate to what extent the technological infrastructure of
networked intelligent environments regulates the life of people, and how this
compares to the way the legal infrastructure regulates life. The discussion on code as
law, which has been initiated by Lessig (1999a) and Reidenberg (1998) and critically
assessed by Tien (2004) Leenes & Koops (2005) and Brownsword (2005) will be
mapped and its relevance for Ambient Intelligent environments will be assessed. After
having identified shortcomings in current ways of understanding regulation by law on
the one hand and technology on the other, an attempt will be made to develop a
generic vocabulary that can adequately describe both technological and legal
regulation of human interaction with the environment, after which the relevant
differences between the two can be detected. Special attention will be given to the
problems of data protection legislation to achieve adequate implementation. At the
present moment the technological infrastructure does not empower end-users to
effectively assess information on what happens to their data, let alone provide them
with the instruments to regain some control over the knowledge built on the
information that is recorded in their environments. Building on the generic vocabulary
for the regulation of human interaction a concept of ambient law will be developed,
which explores the way autonomic computing can be put to use for the
implementation of mandatory data protection legislation. As such, ambient law is a
technologically embodied law that empowers citizens to regain control over the
information and knowledge that will impact decisions taken about them.
8
The idea of a technologically embodied law may be countered by those that fear a rule
by technology. To prevent such undesired integration of law and technology the
contraints of constitutional democracy need to be translated into such an ambient law.
This raises the question how the practices of law and technology could be involved in
such a project. This question will be researched from the perspective of the 'ecology
of practices' (Stengers 1977 & 2006), in which the possibility and desirabiliy of
mutual transformations of both legal and technological practices will be investigated.
I.D. Pertinence of the research
Why should lawyers get involved in the way their legal instruments affect human
behaviour in comparison to the way technological instruments affect human
interaction? The reason why this involvement is pertinent is the fact that technologies
today affect human interaction to an extent that was previously unthinkable. At the
same time this may interfere with the capacity of legal norms to have any effective
impact on human behaviour. Turning a blind eye to this, with the excuse that as
lawyers we would outstep our expertise by getting involved with technological
regulation, may seem a return to a safe haven, but may in fact erode the safety this
haven can deliver. If an Internet of Things appears, if we start living in a networked
environment that forms the interface between human subjects and the technologies
that monitor and profile their behaviours, we must come to terms with the way these
technologies actually regulate our life (inducing certain actions and inhibiting others).
Instead, however, of viewing technological regulation as a threat to legal regulation,
we should identify the point at which effective legal regulation needs technological
embodiement. For instance, it may be the case that Data Protection legislation cannot
achieve its ends without a technological infrastructure that enforces the mandatory
legal rules while also providing access to the information that will allow us informed
negotiations with service providers. Such informed negotiations will have to be
performed between autonomic machines on our side and those on the side of the AmI
environment. The amount of trivial – but still crucial - decisions that will have to be
taken in the framework of an Internet of Things, is simply too massive to allow
continuous human intervention (like consent).
The proposed research is unique. The EC has been funding several large projects on
“ambient intelligence”. In FP5 (eHealth) and FP6 (IST Audiovisual, IST Embedded
Systems) approximately 36 projects have been funded in the area of “ambient
intelligence” involving more than 242 organisations, for a total budget 95.5 M€. Only
a few had a specific workpackage devoted to ethical, legal, social and psychological
implications.
SWAMI (Safeguards in a World of Ambient Inteligence,
http://swami.jrc.es/pages/index.htm) is a project which focuses on a possible future in
an Information Society where all ICT capacities are interconnected. The objective of
SWAMI has mainly been to identify and explore the future social, economic, legal,
technological and ethical issues and the policy options related to safeguards for
privacy, security, trust, identity and digital divide. In this project LSTS took care of
the legal aspects : LSTS made an overview of the law relevant for Ambient
Intelligence and exploringly applied this body of law to the ‘dark scenarios’ in order
to identify strengths and weaknesses of the existing law. In the (current) final phase
LSTS consequently tries to formulate relevant legal policy options.
9
FIDIS (Future of Identity in Information Society, www.fidis.net) is a project which
focuses on advanced identification technologies and its impact on the identity of the
European citizen. LSTS is workpackage leader of Profiling, which is the enabling
technique for autonomic computing. This project explicitly takes a multidisciplinary
perspective and would provide an excellent context for the researchers of the project
on autonomic computing to exchange their findings with excellent researchers from
technological and social sciences throughout the EU.
The proposed research ‘Law and Autonomic Computing: a Process of Mutual
Reconfiguration’ will complement and further develop the research efforts initiated by
SWAMI (completed in late 2006) and FIDIS (running until spring 2009). While
SWAMI addresses the general issue of AmI and FIDIS adresses profiling from a
multidisciplinary perspective, the proposed research specifically focuses on an
assessment of the impact of autonomic computing on fundamental notions of law,
specifically the notion of the legal person and the legal norm. It will thus face the
regulatory challenges seeking the development of benchmarks beyond administrative
law, in order to prepare the making of effective and legitimate regulation in the
framework of the democratic constitutional state. The research we propose now is
thus mainly conceptual and fundamental.
II Research Design
We propose a research design along the lines of the two identified interrelated
subjects: ‘Autonomic Computing and Legal Subjectivity’ and ‘Autonomic Computing
and Ambient Law’. While describing these parts of the research we systematically
identify the different research tasks we propose to undertake.
II.1. Mutual transformations: Autonomic computing and legal
subjectivity
II.1.A. The central position of the human subject in law, in a constitutional
democracy
Constitutional democracy builds on the central position of the human subject. Firstly,
democracy in the modern sense means self-rule for the citizens of a polity, whether
this self-rule is understood in a deliberative, participatory or representative way. Only
human beings qualify for citizenship and – other than in Greek times – citizenship is
not restricted on the basis of class, race or gender. The legitimacy of democratic
government depends on the fact that human beings are free to choose alternative
courses of action, to deliberate on the best public policies, to participate in political
decision making processes and to seek adequate representation for their public
interests. It has been argued that this freedom, which is presumed by political theories
of democracy, is not a given, but needs to be created by the framework that supports a
viable democracy. In line with that, secondly, constitutional democracy depends on
the rule of law that attributes a set of fundamental rights to its citizens. These
fundamental rights confer legal subjectivity to each human being within the
jurisdiction of the polity, providing the individual with a position from which she can
10
exercise her freedom to participate in public and private life and her freedom from
illegitimate intrusions by private and public actors. To actually provide such freedom
the legal persona must not be conflated with the person of flesh and blood. The legal
persona is an artifical legal construction that aims to protect and empower the human
person, whose nature is fundamentally underdetemined. This indeterminacy is one of
the essential tenets of constitutional democracy. It means that a person can never be
entirely defined by others (Foqué & ‘t Hart 1990, Gutwirth 1993 & 2002, Hildebrandt
2002, De Hert & Gutwirth, 2006).
The challenges of autonomic computing are directly connected with possible
transformations of the human subject, raising the question what this should mean for
the concept of the legal subject. Research into these issues will be the first task of the
researchers. The findings of this research implicate a second research task, namely an
investigation of the legal regulation of privacy and intellectual property, to assess to
what extent the present legal framework offers adequate protection against possible
risks, identified in the first task. The third task relating to the central position of the
human subject in constitutional democracy, refers to the question to what extent legal
subjectivity should be attributed to transhuman and nonhuman entities.
II.1.B Task 1: Exploration of the impact of autonomic computing on the
central position of the human subject in law, in a constitutional democracy
To demonstrate the pertinence of this research we will describe some of the possible
consequences of autonomic computing, discussed in the literature. All the tasks will
feed on the issues raised in the research of this first task.
Normalisation, customisation and filtering
The potential effects of GRIN technologies, networked AmI environments and the
Internet of Things may impact the human subject as it is presumed in and constituted
by a constitutional democracy. One of the aims of this project is to investigate what
this impact could be. Authors like Lessig (1999a), Solove (2004) and Sunstein (2001)
have described how the profiling technologies that are at the core autonomated
computing may engender processes of normalisation and customisation as well as
filtering (see also Deleuze 1990 & Gutwirth 2002) .
Customisation and normalisation seem to change the balance of power between large
organisations and individual citizens. These large organisations – whether public or
private – will control an enormous amount of data and have access to detailed and
dynamic profiles of categories of citizens. They can use the knowledge thus available
to make decisions regarding individual persons, even without identifying them. In
5
fact, as Lessig writes , autonomic computing implies that these decisions are taken by
machines, which are continuously profiling us, in order to provide adequate services.
The point may be that we are not necessarily facing a Big Brother, taking over our
lives, but rather that we find ourselves in a labyrinth of ‘digital dossiers’, as Solove
5
'When the system seems to know what you want better and earlier than you do, how can you know where these
desires really come from? (…) profiles will begin to normalize the population from which the norm is drawn. The
observing will affect the observed. The system watches what you do; it fits you into a pattern; the pattern is then
fed back to you in the form of options set by the pattern; the options reinforce the patterns; the cycle begins again'
(Lessig 1999: 154).
