A party that learns to cooperate without a commander
The target was not a toy Atari agent. It was combat AI that could eventually operate in multiplayer RPGs such as Diablo, AION, or League of Legends, where many characters, skills, effects, positions, and timing constraints interact in real time.
Irregular complexity
Characters, skills, effects, status changes, and continuous 2D/3D positions form a rule set with little clean regularity.
Multi-agent cooperation
Each agent must act locally while learning behavior that improves the whole team’s chance of winning.
Real-time constraints
Classical search exceeded practical compute budgets; handcrafted logic created an unsustainable maintenance burden.
An automated arena for learning systems
We removed graphics and built MiniLOL, a compact text-based game that preserved the important problem properties. It became a pluggable testbed where different agents and scenarios could compete continuously.
MiniLOL
Two classes, a simple map, and a text-only simulator with damage, buffs, debuffs, stun, heal, dash, and other effects.
Many simulators
About 30 simultaneous simulators running 1v1, 3v3, and 5v5 matches across classes and maps.
Black-box RL
Agents learned without explicit access to game rules, progressing from DQN toward a deeper LSTM agent.
Automated league
Behavior Tree, greedy, expectation-based greedy, reinforcement-learning, and deep-learning agents were compared by matches.
Continuous delivery
A GitLab push triggered automated Docker build, deployment, and execution on the compute machines.
Agents improved by competing against different ideas
The model was only one component of the system
STATEGlobal + self + friends + enemies
State design considered invariance across teammates and opponents, rather than treating an arbitrary list order as meaning.
ACTIONMovement + targeted skills
Movement and action commands were unrolled into targetable choices that the value network could evaluate.
REWARDTeam victory
A team reward allowed an individual agent to sacrifice itself when doing so improved the team outcome.
Stabilization and learning
In-house cluster stack
In hindsight, this was an early AI Factory loop
That phrase was not used in the original talk. But the system already connected the layers that now define an AI Factory: environments that produce experience, accelerated training, automated evaluation, resource orchestration, and a deployment loop.
Turn reality into a learnable world
MiniLOL compressed a complicated production domain into a controllable environment while preserving the structure of the hard problem.
Let agents expose each other’s limits
The automated arena converted competing methods into a continuously improving evaluation and selection process.
Optimize the full loop, not only the model
The cluster automated compute allocation, builds, experiments, measurement, and redeployment—the systems pattern that later continued into ML platforms.