First, it describes an architecture within which a flexible team structure allows member agents to decompose a task into flexible roles and to switch roles while acting. Second, it presents layered learning, a general-purpose machine-learning method for complex domains in which learning a mapping directly from agents' sensors to their actuators is intractable with existing machine-learning methods. Third, the book introduces a new multiagent reinforcement learning algorithm -- team-partitioned, opaque-transition reinforcement learning (TPOT-RL) -- designed for domains in which agents cannot necessarily observe the state-changes caused by other agents' actions. The final contribution is a fully functioning multi-agent system that incorporates learning in a realtime, noisy domain with teammates and adversaries -- a computer-simulated robotic soccer team.