Reinforcement Learning Explorer

Interactive demonstrations of foundational reinforcement learning algorithms and concepts

Chapter 1: MDPs

Every learned behavior begins with a world to interact with. A Markov Decision Process (MDP) gives us the mathematical scaffolding to define states, the actions we can take, the probabilities of where we end up, and the rewards we gather along the way.

Tuple Definition

(S, A, P, R, γ)

S = { 0 States }
A = { 'Click' }
R = { 0.0 }

Click empty space to spawn states. Click two states to link them.

Chapter 2: Value Iteration

How does an agent know which path is best before walking it? By looking far into the future and propagating value backward via the Bellman Equation.

Value & Policy Extraction

V(s) = max_a Σ P(s'|s,a) [ R(s,a,s') + γ V(s') ]

Discount Factor (γ): 0.90

Chapter 3: Monte Carlo vs TD

Monte Carlo waits for the end of the episode to reflect. Temporal Difference (TD) updates its beliefs step-by-step. Let's race them.

Monte Carlo (Retroactive)

TD(0) (Immediate)

Cumulative Reward Tracker

Tick Rate:

Chapter 4: Q-Learning

To find the optimal path, an agent must risk stepping into the unknown. The ε-Greedy strategy controls this balance between exploration and exploitation.

Q(s,a) ← Q(s,a) + α[r + γ·max Q(s',a') - Q(s,a)]

Exploration Rate (ε)

0.0 (Exploit) 1.0 (Explore)

Total Reward: 0

Steps Taken: 0

Chapter 5: Actor-Critic

A continuous control system. The Actor decides *how* to move, the Critic learns *what* that move is worth.

Actor Panel (Policy Distribution)

Critic Panel (State Value)

Training Log

Initializing training sequence...