Deep-Reinforcement-Learning-Based Resource Management for Task Offloading in Integrated Terrestrial and Nonterrestrial Networks

Integrated terrestrial-nonterrestrial networks have recently gained much attention because they can bridge the gap between the conventional terrestrial infrastructure and nonterrestrial networks. In addition to seamless connectivity, such networks can offer edge computing services to the users with real-time data processing demand. In this article, an integrated terrestrial-nonterrestrial network with multiaccess edge computing (ITNT-MEC) system is considered in which the aerial users (AUEs) share the resources of terrestrial base stations (TBSs) with their existing terrestrial users (TUEs) and that of low-Earth orbit (LEO) satellites with their neighboring satellites. The goal is to minimize the total energy consumption of AUEs, TUEs, and LEO satellites by jointly optimizing the AUE-TBS/LEO satellite association, AUEs’ trajectories, task allocation, as well as network resource allocation. Due to the dynamic nature of network environment and nonconvex characteristics, it is significantly challenging to solve the formulated optimization problem. Therefore, a block coordinate descent (BCD)-based algorithm that integrates deep reinforcement learning (DRL) methods, such as double deep Q-learning network (DDQN), deep deterministic policy gradient (DDPG), and convex optimization methods, is proposed. Simulation results show that the total energy consumption in the proposed approach is reduced by 14%, 26.9%, 34%, 35.8%, 45.5%, and 55.4%, respectively, when compared to the baselines, such as DDPG-based task offloading (DDPG-TO), DDQN-based task offloading (DDQN-TO), DQN-based task offloading (DQN-TO), equal resource allocation (ERA), random association (RA), and fixed trajectory (FT).