Intelligent Agents, AI & Interactive Entertainment (CS7032)

130 Pages·2014·2.67 MB·English

Intelligent Agents, AI & Interactive
Entertainment (CS7032)
Lecturer: S Luz
School of Computer Science and Statistics
Trinity College Dublin
[email protected]
December 11, 2014

Contents
1 Course Overview 5
2 Agents: definition and formal architectures 11
3 Practical: Creating a simple robocode agent 21
3.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21
3.2 Robocode: Getting started . . . . . . . . . . . . . . . . . . . . . 21
3.3 Robot creation in (some) detail . . . . . . . . . . . . . . . . . . . 22
3.4 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.1 The environment . . . . . . . . . . . . . . . . . . . . . . . 23
3.4.2 Create your own robot . . . . . . . . . . . . . . . . . . . . 23
3.4.3 Describe the task environment and robot . . . . . . . . . 23
3.4.4 PROLOG Robots (optional) . . . . . . . . . . . . . . . . 24
4 Utility functions and Concrete architectures: deductive agents 25
5 Reactive agents & Simulation 33
6 Practical: Multiagent Simulations in Java 45
6.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 45
6.2 Setting up the environment . . . . . . . . . . . . . . . . . . . . . 45
6.3 Running two demos . . . . . . . . . . . . . . . . . . . . . . . . . 46
6.4 Simulating simple economic ecosystems . . . . . . . . . . . . . . 47
6.4.1 Brief description of the sscape simulation . . . . . . . . . 47
6.4.2 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . 47
7 Swarm intelligence: Ant Colony Optimisation 49
8 Practical: ACO simulation for TSP in Repast 59
8.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.2 Some code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59
8.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60
4
8.3.1 Delivering the assignment . . . . . . . . . . . . . . . . . . 60
9 Learning Agents & Games: Introduction 61
10 Practical: Survey of game AI competitions 73
10.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
10.2 What to do . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73
10.2.1 Presentation . . . . . . . . . . . . . . . . . . . . . . . . . 74
11 Evaluative feedback 77
12 Practical: Evaluating Evaluative Feedback 87
12.1 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.2 Some code . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 87
12.3 Exercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88
13 Learning Markov Decision Processes 89
14 Solving MDPs: basic methods 99
15 Temporal Difference Learning 109
16 Combining Dynamic programming and approximation archi-
tectures 117
17 Final Project 123
17.1 Background . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
17.2 Goals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123
17.3 Assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
17.4 Delivering the assignment . . . . . . . . . . . . . . . . . . . . . . 124
17.5 Possible choices . . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
17.5.1 Resources . . . . . . . . . . . . . . . . . . . . . . . . . . . 124
17.6 Questions? . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 125
1
Course Overview
What
Artificial Intelligence, Agents & Games:
•
– AI
(+DialogueSystemsand/orNaturalLanguageGeneration,TBD)
∗
– Distributed AI,
– Machine learning, specially reinforcement learning
A unifying framework:
•
– “Intelligent” agents
An agentisacomputersystemthatis situatedinsome environment
andthatiscapableof autonomousactioninitsenvironmentinorder
to meet its design objectives.
(Wooldridge, 2002)
Why
Theory: Agents provide a resonably well-defined framework for investi-
•
gating a number of questions in practical AI
– (unlike approaches that attempt to emulate human behaviour or
discover universal “laws of thought”)
Practice:
•
– Applications in the area of computer games,
– ComputergamesasadropsophilaforAI (borrowingfrom(McCarthy,
1998))
6 CHAPTER 1. COURSE OVERVIEW
Course structure
• Main topics:
– Agent architectures and platforms
– Reasoning (action selection)
– Learning
• Practicals:
– project-basedcourseworkinvolving,forexample,implementationofamulti-
agentsystem,amachinelearningandreasoningmodule,anaturallanguage
generation system, ...
• Knowledge of Java (and perhaps a little Prolog) is desirable.
Agent architectures and multi-agent platforms
Abstract agent architecture: formal foundations
•
Abstract architectures instantiated: case studies
•
– theory,
– standards, and
– agent development platforms (e.g.
Repast)
– and libraries (e.g. masona, swarmb)
ahttp://cs.gmu.edu/~eclab/projects/mason/
bhttps://savannah.nongnu.org/projects/
Agentsandsimulation: swarm
•
Reasoning
Probabilistic Reasoning:
•
– theoretical background
– knowledge representation and inference
– Survey of alternative approaches to reasoning under uncertainty
Chapter 1 7
Learning
Machine learning: theoretical background
•
Probabilistic learning: theory and applications
•
Reinforcement learning
•
Coursework and Project
• Coursework-based assessment:
• Weekly labs and mini-assignments (30%)
• Longer project, to be delivered in January (70%):
– Entering an AI Game competition
– Reinforcement Learning for Off-The-Shelf games: use of a game simula-
tor/server such as “Unreal Tournament 4” (UT2004) to create an agent
that learns.
– A Natural Language Generation system to enter the Give Challenge
– A project on communicating cooperative agents,
– ...
Multiagent systems course plan
An overview: (Wooldridge, 2002) (Weiss, 1999, prologue):
•
– What are agents?
– What properties characterise them?
Examples and issues
•
Human vs. artificial agents
•
Agents vs. objects
•
Agents vs. expert systems
•
Agents and User Interfaces
•
8 CHAPTER 1. COURSE OVERVIEW
A taxonomy of agent architectures
Formal definition of an agent architecture (Weiss, 1999, chapter 1) and
•
(Wooldridge, 2002)
“Reactive” agents:
•
– Brooks’ subsumption architecture (Brooks, 1986)
– Artificial life
– Individual-based simulation
“Reasoning” agents:
•
– Logic-based and BDI architectures
– Layered architectures
The term “reactive agents” is generally associated in the literature with
the work of Brooks and the use of the subsumption architecture in distributed
problem-solving. In this course, we will use the term to denote a larger class
of agents not necessarily circumscribed to problem-solving. These include do-
mainswhereoneisprimarilyinterestedinobserving,orsystematicallystudying
patterns emerging from interactions among many agents.
Reactive agents in detail
Reactive agents in problem solving
•
Individual-based simulation and modelling and computational modelling
•
of complex phenomena:
– Discussion of different approaches
Simulations as multi-agent interactions
•
Case studies:
•
– Agent-based modelling of social interaction
– Agent-based modelling in biology
– Ant-colony optimisation
Logic-based agents (Not covered this year.)
• Coordination issues
• Communication protocols
• Reasoning
• Agent Communication Languages (ACL):
– KQML
– FIPA-ACL
• Ontology and semantics:
– KIF
– FIPA-SL
Chapter 1 9
“Talking” agents
Embodied conversational Agents
•
Natural Language processing and
•
generation by machines
Some case studies
•
Learning
Learning and agent architectures (Mitchell, 1997)
•
Learning and uncertainty
•
Reinforcement learning:
•
– Foundations
– Problem definition: examples, evaluative feedback, formalisation,
– Elementary solution methods, temporal-difference learning,
– Case studies: various game-playing learners
Course organisation and resources
Course Web page:
•
http://www.cs.tcd.ie/~luzs/t/cs7032/
The page will contain:
•
– Slides and notes
– links to suggested reading and software to be used in the course.
It will also contain the latest announcements (“course news”), so please
•
check it regularly
Software resources
A game simulator (no AI methods incorporated):
•
– Robocode: http://robocode.sf.net/
Agent-based simulation software:
•
– MASON:http://http://cs.gmu.edu/~eclab/projects/mason/or
– SWARM: http://www.swarm.org/ or
10 CHAPTER 1. COURSE OVERVIEW
– REPAST http://repast.sourceforge.net/
TheGiveChallengewebsite: http://www.give-challenge.org/research/
•
See also:
•
– http://www.cs.tcd.ie/~luzs/t/cs7032/sw/
Reading for next week
“Partial Formalizations and the Lemmings Game” (McCarthy, 1998).
•
– available on-line1;
– Prepare a short presentation (3 mins; work in groups of 2) for next
Tuesday
Texts
• Useful references:
– Agents, probabilistic reasoning (Russell and Norvig, 2003)
– Agents, multiagent systems (Wooldridge, 2002)
– Reinforcementlearning(BertsekasandTsitsiklis,1996),(SuttonandBarto,
1998)
• and also:
– (some) probabilistic reasoning, machine learning (Mitchell, 1997)
– Artificial Intelligence for Games (Millington, 2006)
– Research papers TBA...
1http://www.scss.tcd.ie/~luzs/t/cs7032/bib/McCarthyLemmings.pdf