Local integrated energy system operational optimization considering multi‐type uncertainties: A reinforcement learning approach based on improved TD3 algorithm

Autor: Yue Qiu, Suyang Zhou, Dong Xia, Wei Gu, Kaiyu Sun, Gaoyan Han, Kang Zhang, Hongkun Lv
Jazyk: angličtina
Rok vydání: 2023
Předmět:
Zdroj: IET Renewable Power Generation, Vol 17, Iss 9, Pp 2236-2256 (2023)
Druh dokumentu: article
ISSN: 1752-1424
1752-1416
DOI: 10.1049/rpg2.12725
Popis: Abstract Local integrated energy system (IES), usually a multi‐building heating and cooling system incorporating cogeneration systems and distributed energy resources (DERs), is becoming an efficient energy infrastructure for energy system decarbonization. However, for small and medium‐scale local IESs, the fluctuation of user loads seriously influences energy system operation, and the increasing DER penetration also enlarges the influence. Given the necessity of exploring an efficient way to handle the uncertainties in local IES, this paper proposes a reinforcement learning (RL) approach based on the improved TD3 algorithm. The mathematical model of local IES is first established considering supply‐ and load‐side flexible resources. The local IES dispatch problem is formulated as a Markov decision process (MDP), in which multi‐type uncertainties of renewable generation, electric load and heat load are considered. For solving the MDP, an improved twin delayed deep deterministic policy gradient (TD3) algorithm is proposed with a dynamic balance mechanism of exploration noise. Based on a local IES testbed in the Nantong Central Innovation District, China, a comparison analysis is conducted to verify the promoting effect of flexible resources on the operation economy and renewable energy consumption. The system operating cost reduces by 18.46%, and surplus renewable energy can all be accommodated considering flexible resources. The dispatch policies obtained by the deep deterministic policy gradient (DDPG), the improved TD3, the original TD3 and traditional optimization algorithms are also compared. The results reveal that the convergence stability and solving accuracy of the improved TD3 outperform the other two RL algorithms. Specifically, the system operating cost of the improved TD3 reduces by 2.76% compared with the DDPG, and the energy supply imbalance decreases by around 88%. Meanwhile, the improved TD3 exhibits better operation economy and adaptability to the uncertain environment than the deterministic optimization and intraday rolling algorithms.
Databáze: Directory of Open Access Journals
Externí odkaz: