DQN v0.2: experience replay

ℒ(w) = 1BΣi ∈ batch[ ri + γ maxa Q̂(si+1, a; w) − Q̂(si, ai; w) ]2    w ← w − η ∇w ℒ(w)
DQN v0.2 ENV State (st) Action (at) Reward (rt) Next State (st+1) π at = ε-greedy argmaxa Q̂(st, a; w) action (at) Q̂(si, ai; w) ① maxa Q̂(si+1, a; w) ② update w ℒ mini- batch B = 4 two passes: si​, si+1​
ℒ = 1BΣi[ ri + γ max Q̂(si+1, a; w) ② − Q̂(si, ai; w) ① ]2
ReadyPress “Next ▶” to follow one transition through DQN with experience replay.
Stage – of 8

Keyboard: Space or → for next stage, ← for previous, S to play one full step, P to play or pause.

Settings