Deep Reinforcement Learning Based MAC Protocol for Underwater Acoustic Networks
| Autor: | Xiaowen Ye, Liqun Fu, Yiding Yu |
|---|---|
| Rok vydání: | 2022 |
| Předmět: |
Artificial neural network
Computer Networks and Communications business.industry Computer science Throughput Energy consumption Propagation delay Transmission (telecommunications) Reinforcement learning State (computer science) Electrical and Electronic Engineering business Protocol (object-oriented programming) Software Computer network |
| Zdroj: | IEEE Transactions on Mobile Computing. 21:1625-1638 |
| ISSN: | 2161-9875 1536-1233 |
| Popis: | This paper develops a deep reinforcement learning (DRL) based MAC protocol for UWANs, referred to as delayed-reward deep-reinforcement learning multiple access (DR-DLMA), to maximize the network throughput by judiciously utilizing the available time slots resulted from propagation delays or not used by other nodes. In the DR-DLMA design, we first put forth a new DRL algorithm, termed as delayed-reward deep Q-network (DR-DQN). Then we formulate the multiple access problem in UWANs as a reinforcement learning (RL) problem by defining state, action, and reward in the parlance of RL, and thereby realizing the DR-DLMA protocol. The essence of DR-DQN is to incorporate the propagation delay into the DRL framework and modify the DRL algorithm accordingly. In addition, in order to reduce the cost of online training deep neural network (DNN), we provide an adaptive training mechanism for DR-DQN. Simulation results show that our DR-DLMA protocol with adaptive training mechanism can (i) find the optimal transmission strategy when coexisting with other protocols in a heterogeneous environment, (ii) outperform state-of-the-art MAC protocols (e.g., slotted FAMA and DOTS) in a homogeneous environment, (iii) greatly reduce energy consumption and run-time compared with DR-DLMA with traditional DNN training mechanism. |
| Databáze: | OpenAIRE |
| Externí odkaz: |
|