Joel's Assignment 13

Layered Learning in Multi-Agent Systems - Peter Stone, CMU 1998

In his thesis, Peter Stone tackles the problem of:
"Can agents learn to become individually skilled and to work together in the presence of both teammates and adversaries in a real-time, noisy environment with limited communication?"

Specifically, Stone uses machine learning techniques to improve an agents behavior in domains with the following characteristics:

Need for Real-Time decision making
Sensor and actuator noise with hidden state
- Sensors of the environment are noisy
- Noisy actuators that affect the environment
Several independent agents that have the same high-level goal (teammates)
Several independent agents with a conflicting goal (adversaries)

The agents can process sensory information (noisy) and make decisions based on it, as well as have low-level, unreliable communication.

So what is a suitable domain for testing this? Soccer, of course!

Stone's Thesis introduces several keys to success in robosoccer (as well as other areas)

Team member agent architecture - Combines "locker-room" agreements with real-time independence and low-level communication. A "locker-room" agreement can be set up before the game and at half time, thus information learned during the first half can be fully communicated with teammates at half-time. However, in game this would be a poor choice, given that if they are standing there taking the adversaries would have an easy time putting the ball in the goal.
Layered Learning - Abstracts from the Sensors to Behavior model, where actions are determined by several layers of abstractions and ML techniques are used for each one.
TPOT-RL - Used for pass selection. Learns from its and its teammates actions, as well as adversaries, based on long term effects to choose the next logical pass. Is the highest layer in the Layered Learning hierarchy.

All in all, I believe this is an excellent thesis. Stone's teams have done well at robot soccer (CMUnited, which he co-founded is a 4 time world champion, and his current team at UT is a 2 time world champion). Stone is also credited with being a major contributer to the early days of robot soccer, and is quite a pioneer in the area. Reading a thesis from a person who has pioneered something that currently has over 300 teams participating is always interesting, and I believe that his thesis contributes major work to Muti-Agent Systems and Machine Learning.

Last modified 3 December 2007 at 11:06 am by joelpfeiffer