TD algorithm based on double-layer fuzzy partitioning

Autor: Xiang MU, Quan LIU, Qi-ming FU, Hong-kun SUN, Xin ZHOU
Jazyk: čínština
Rok vydání: 2013
Předmět:
Zdroj: Tongxin xuebao, Vol 34, Pp 92-99 (2013)
Druh dokumentu: article
ISSN: 1000-436X
DOI: 10.3969/j.issn.1000-436x.2013.10.011
Popis: When dealing with the continuous space problems,the traditional Q-iteration algorithms based on lookup-table or function approximation converge slowly and are diff lt to get a continuous policy.To overcome the above weak-nesses,an on-policy TD algorithm named DFP-OPTD was proposed based on double-layer fuzzy partitioning and its convergence was proved.The first layer of fuzzy partitioning was applied for state space,the second layer of fuzzy parti-tioning was applied for action space,and Q-value functions were computed by the combination of the two layer fuzzy partitioning.Based on the Q-value function,the consequent parameters of fuzzy rules were updated by gradient descent method.Applying DFP-OPTD on two classical reinforcement learning problems,experimental results show that the algo-rithm not only can be used to get a continuous action policy,but also has a better convergence performance.
Databáze: Directory of Open Access Journals