Hyperbolic Tangent Polynomial Parity Cyclic Learning Rate for Deep Neural Network

Autor:	Hong Lin, Ruyan Xiong, Xiaodong Yang, Binyan Wu
Rok vydání:	2021
Předmět:	Polynomial Schedule Contextual image classification Artificial neural network Computer science Hyperbolic function Algorithm Field (computer science) Object detection Task (project management)
Zdroj:	PRICAI 2021: Trends in Artificial Intelligence ISBN: 9783030893620 PRICAI (2)
Popis:	With the development of artificial intelligence technology, optimizing the performance of deep neural network model has become a hot issue in the field of scientific research. Learning rate is one of the most important hyper-parameters for model optimization. In recent years, some learning rate algorithms with cycle mechanism have been proposed. Most of them adopt warm restart and cycle mechanism to make the learning rate value cyclically change between two boundary values and prove their effectiveness by practicing in image classification task. In order to further improve the performance of neural network model and prove the effectiveness in different training task, the paper proposes a novel learning rate schedule called hyperbolic tangent polynomial parity cyclic learning rate (HTPPC), which adopts cycle mechanism and combines the advantages of warm restart and polynomial decay. In addition, the performance of HTPPC is demonstrated on image classification and object detection tasks.
Databáze:	OpenAIRE
Externí odkaz:	https://explore.openaire.eu/search/publication?articleId=doi_________::cebbdbbb508bad3e2550218994105606 https://doi.org/10.1007/978-3-030-89363-7_34 Zobrazit plný text záznamu