DQN v0.1: information flow

ℒi = [ ri + γ maxa Q̂(si+1, a; w) − Q̂(si, ai; w) ]2      w ← w − η ∇w ℒi
DQN v0.1 ENV State (st) Action (at) Reward (rt) Next State (st+1) two passes: si, si+1 π at = ε-greedy argmaxa Q̂(st, a; w) action (at) Q̂(si, ai; w) ① maxa Q̂(si+1, a; w) ② update w ℒ ℒi = [ ri + γ max Q̂(si+1, a; w) ② − Q̂(si, ai; w) ① ]²
ReadyPress “Next ▶” to follow one transition through DQN.
Stage – of 6

Keyboard: Space or → for next stage, ← for previous, S to play one full step, P to play or pause.

Settings