Published June 5, 2021
| Version v1
Book section
Full Gradient DQN Reinforcement Learning: A Provably Convergent Scheme
Contributors
Others:
- Network Engineering and Operations (NEO ) ; Inria Sophia Antipolis - Méditerranée (CRISAM) ; Institut National de Recherche en Informatique et en Automatique (Inria)-Institut National de Recherche en Informatique et en Automatique (Inria)
- Department of Electrical Engineering [IIT-Bombay] (EE-IIT) ; Indian Institute of Technology Kanpur (IIT Kanpur)
- Projet PIA - ANSWER - FSN2 (P159564-2661789\DOS0060094)
- Alexey Piunovskiy
- Yi Zhang
Description
We analyze the DQN reinforcement learning algorithm as a stochastic approximation scheme using the o.d.e. (for 'ordinary differential equation') approach and point out certain theoretical issues. We then propose a modified scheme called Full Gradient DQN (FG-DQN, for short) that has a sound theoretical basis and compare it with the original scheme on sample problems. We observe a better performance for FG-DQN.
Abstract
International audienceAdditional details
Identifiers
- URL
- https://hal.inria.fr/hal-03462350
- URN
- urn:oai:HAL:hal-03462350v1
Origin repository
- Origin repository
- UNICA