INTRODUCTION

In their paper “Autonomous Agents”, Floridi and Sanders (2004) give us a helpful language of discourse regarding artificial agents and their potential morality.   Although they have contributed precise definitions of autonomy, moral accountability and responsibility, Floridi and Sanders have not deeply explored some essential questions that need to be answered by computer scientists as they design these agents.   One such question is:   “Should the designer of an artificial agent focus on accomplishing a specific task efficiently, or must the designer explicitly program some semblance of ethical analysis into each agent?” Another such question is, “Can an artificial agent that changes its own programming become so autonomous that the original designer is no longer responsible for the behavior of the artificial agent?”   To explore such questions, we propose to elaborate concepts of agency, levels of abstraction and the complexity of machines using Floridi and Sanders’ definitions as a starting point.

Following Floridi and Sanders, we acknowledge the importance of using levels of abstraction (LoA) when exploring artificial agency. We will focus on two particular LoA’s:  

LoA1. The level of abstraction visible to users, both human and artificial, that interact with the artificial agent under study, and

LoA2. The level of abstraction that is visible to the designer of the agent.

We distinguish between these two views to allow control of the granularity at which we analyze what Floridi and Sanders call the “observables.”

To be more precise, we will specify that the internal states Sx are not observable at LoA1 (i.e., users can’t see the internal state), but these states are available at LoA2 (i.e., the designer can see, as well as determine, the states).    In fact, if we allow a generalized definition of the state, and assume that the designer (D1) has all the details of the computation available for inspection, the mapping at LoA2 can be simplified to mapping from an initial state that includes new values due to inputs, both external and temporal, to a new state that includes values that are externally observable at LoA1. Using this view of LoA2, in the paper, we will describe the actions of any artificial agent (A1) as a mathematical function between states.

1.   ARTIFICIAL AGENT PROCESSING USING A MAPPING TABLE (AAPUMT)

Step 0. Start with an initial state, S0.

Step 1. External stimuli change S0, yielding a new state, S1.

Step 2. Map from S1 to S2 using a large mapping table, T that maps from each possible state to a new state.

Step 3. Allow output variables based on S2 to become visible. (These variables may be visible through readouts, or externally visible behaviors or effects.)

Step 4. Loop to step 1 with S2 taking on the role of S0.

Note that external stimuli only change the state of the computation during step 1. This is only an approximation of a realistic computation. Note also that the mapping table T in step 2 models the programming (and therefore the design) of A1. For the sake of simplicity, T is assumed to be outside the impact of the computation. Without this stipulation, the model becomes more complex because the mapping listed in the table is also part of the state that each table entry must store.

In any practical situation, the state table T is prohibitively large (though finite), and object code produced from source code is used to implement this mapping instead of building an explicit T. However, T is theoretically possible since any digital computer has a finite number of states and as long as A1’s program is deterministic, the “next state” relation is a function.   (We deal briefly with non-deterministic computations in our paper, but they are not central to our argument.)

In our full paper, we will describe two additional variations of the model above. In the first, any part of the table T that defines the A1’s behavior can be modified by the A1 during its execution; in other words, A1 can self-modify. In the second variation, the table T is divided into two parts: in one part, the mappings can be modified; in the other part, the mappings can not be modified. In other words, some parts of T are protected from self-modification by A1.

One of the strengths of Floridi and Sanders use of LoA’s is that it allows the simplification of otherwise highly complex issues. However, in this paper we wish to examine in more detail some of the complexities that these simplifications hide.

2.   LEARNING, INTENTIONALITY, FREE WILL AND RANDOMNESS IN ARTIFICIAL AGENTS

We will discuss four issues: learning, intentionality, free will, and randomness in artificial agents. We will first define each of these terms with respect to LoA1, LoA2 and the AAPuMT model. Next, we will examine the relationship between learning, intentionality, and free will at A1 and at A2, comparing the traditional notions of learning, intentionality, and free will in humans to those same concepts in the artificial agent A1. Then we will examine how the generalized A1 applies in the case of two actual artificial agents: an intelligent router and software bot.

In the rest of the paper, we will present and elaborate on six propositions:

Proposition 1. Artificial agents can learn. At any level of abstraction that is (in our opinion) reasonable, artificial agents can change their future behavior on the basis of their past experience. It should be noted that this learning need not be traceable. That is, once A1’s table T is altered, it may not be possible to recapture the initial T or the inputs that led to the new T. (Such tracing may be possible, but only if extra information is saved.)

Proposition 2. Artificial agents are intentional. Dennett (1981:1) noted “a particular thing is an intentional system only in relation to the strategies of someone who is trying to explain and predict its behavior.” Applying Dennett’s principle, and treating an agent as a “someone,” the question, “Did A1 intend to do that?” always requires the answer “yes.” Unless there is a mechanical breakdown, programs do what they are told to do.   Therefore, A1 is an intentional system, because the actions of A1 follow its programming, encoded in T. However, that A1 may not be intentional with respect to the designer, D1. A1 likely has behaviors that D1 did not intend. D1 has access to the table T,   and wants to make A1 intentional. However, A1 conforms to D1’s intentions if and only if the table T conforms exactly to D1’s intentions. Note that A1 can be intentional with respect to D1 only if D1 knows exactly what D1 intends for each and every entry in T, and if D1 encodes these intentions flawlessly. This omniscience and perfection is vanishingly rare.   For users on LoA1, A1 will appear to be an intentional system if users can predict A1’s actions.

Proposition 3. Because A1 can change without human intervention after it is implemented and installed, A1 can be an agent of change that exhibits behaviors that appear intelligent to observers at LoA1. This observed intelligence will become increasingly complex and subtle in the near future.

Proposition 4. A designer D1 cannot anticipate all future behaviors of a reasonably complex A1 that can self modify.

Proposition 5. Because of limited knowledge, D1 has an increased burden of care in programming A1, especially if A1 is designed to learn and to change the table T without human intervention. Likewise, anyone using A1 (especially in a novel environment) has an increased burden of care to insure that unintended consequences are not significantly harmful.

Proposition 6.   Sanctions are appropriate against artificial agents who are observed to have behaviors reasonably classified as coming from a moral bad actor. Sanctions against artificial agents will of necessity be different from sanctions against humans; however, sanctions against artificial agents who function as bad actors are appropriate.

References:

Bechtel, William (1985) “Attributing Responsibility to Computer Systems”, Metaphilosophy, vol 16, no 4, October, Blackwell, Oxford and New York. Pp. 296-306.

Bynum, Terrell Ward (1985) “AI, Biology and Intentional States”, Metaphilosophy, vol 16, no 4, October, Blackwell, Oxford and New York.pp. 355-377.

Dennett, Daniel C. (1981, original 1978), "Intentional systems", Mind design, Bradford Books, Montgomery, Vermont. http://www.cs.umu.se/kurser/TDBC12/HT99/dennett2.html . 

Floridi, L and Sanders, J.W. ( 2004 )   “On the Morality of Artificial Agents”, Minds and Machines,   vol 14, no.3,   pp. 349-379 , Springer Netherlands.   http://www.wolfson.ox.ac.uk/~floridi/pdf/omaa.pdf