Q-Learning Tic Tac Toe
An interactive reinforcement-learning demo: train a tabular Q-learning agent, play against it, then inspect the learned Q-table.
How Q-Learning Works
An agent interacts with an environment by taking actions and receiving rewards. Over time, it learns a policy: a mapping from states to actions that maximizes expected reward.
The Q-table
In this app, a state is the current nine-cell board and an action is the empty cell the agent chooses. The Q-table stores an estimated value for each state-action pair.
Rewards
| Outcome | Reward |
|---|---|
| Agent win | +1.0 |
| Agent loss | -1.0 |
| Draw | +0.3 |
| Non-terminal move | 0.0 |
Exploration vs exploitation
During training, epsilon controls how often the agent chooses a random move instead of the best move it currently knows. This helps it discover strategies it would miss by only exploiting early Q-values.
Why Tic Tac Toe
The state space is small enough for a readable table, games end quickly, and the optimal result is known: with perfect play from both sides, the game is a draw.