EventsMOL2NET'16, Conference on Molecular, Biomed., Comput. & Network Science and Engineering, 2nd ed.
Event submissions
Published
with-doi10.3390/mol2net-02-03871
This submission belongs to the session 03. USEDAT-02: USA-Europe Data Analysis Training Program Workshop, Cambridge, UK-Bilbao, Spain-Miami, USA, 2016 of the event MOL2NET'16, Conference on Molecular, Biomed., Comput. & Network Science and Engineering, 2nd ed.
Published date
18 Jan, 2017
Citation
wen Hu, Residual Value Iteration Algorithm based on Function Approximation, in Proceedings of MOL2NET'16, Conference on Molecular, Biomed., Comput. & Network Science and Engineering, 2nd ed., 15 October–20 October 2022, MDPI: Basel, Switzerland, doi: 10.3390/mol2net-02-03871
Share
Email
Facebook
Twitter
LinkedIn

Residual Value Iteration Algorithm based on Function Approximation

wen Hu 1,2,3
1. Institute of Electronics and Information Engineering, Suzhou University of Science and Technology, Suzhou，Jiangsu
2. Jiangsu Key Laboratory of Intelligent Building Energy Efficiency, Suzhou University of Science and Technology, Suzhou, Jiangsu
3. Suzhou Key Laboratory of Mobile Networking and Applied Technologies, Suzhou University of Science and Technology, Suzhou, Jiangsu
Abstract

With respect to the problem of unstable and slow convergence for traditional Value Iteration algorithm, we proposed an improved Residual Value Iteration Algorithm based on Function Approximation. The algorithm combines traditional Value Iteration algorithm and Value Iteration algorithm with Bellman residual, introduces weight factors and constructs new rules to update value function parameter vector. Theoretically, the new rule for updating value function parameter vector can guarantee the convergence of the algorithm and solve the unstable convergence problem of the traditional value iteration algorithm. In addition, the algorithm introduces a new factor, named forgotten factor, to speed up the convergence of the algorithm. Applying the proposed algorithm, Value Iteration algorithm and LSPI algorithm to the traditional Grid World problem, the experiment results show that the FARVI algorithm has a good performance and robustness to different scale problems.

Keywords
Reinforcement Learning
Value Iteration
Function Approximation
Gradient Descent
Bellman Residual
Poster
An Efficient Residual Q-learning algorithm with Function Approximation.pdf
An Efficient Residual Q-learning algorithm with Function Approximation.pdf
Chemometrical analysis of structure-structure and structure-activity trends of cycloartane-based saponins in Astragalus genus
Multiple Linear Regression Model of Thermolysin Inhibitors