Delay Optimal Random Access with Heterogeneous Device Capabilities in Energy Harvesting Networks Using Mean Field Game

In distributed random access (RA), each device should consider its own information as well as the influence from others, which is difficult to obtain with diverse capabilities of devices. In this paper, we study the RA problem for massive devices with heterogeneous device capabilities in large-scale energy harvesting IoT networks. To deal with the overload issue for massive devices with different capabilities, we propose an optimal RA policy by improving the conventional mean field games (MFG) via exchanging the mean field terms (MFT) among devices. Specifically, we formulate the delay optimal problem as a two-dimensional Markov Decision Process (MDP) problem involving both energy and data states. For distributed deployment of massive RA, we divide the MDP into multiple per-device subproblems, and propose the distributed RA scheme by solving the Hamilton-Jacobi-Bellman (HJB) equation using stochastic learning. Considering the deviation of MFT estimation induced by heterogeneous device capabilities, we design an MFT consensus scheme based on stochastic approximation by information exchange among neighbor devices. For reducing the state space and exchanging the MFT efficiently, we adopt the number of simultaneous access devices instead of the conventional MFT. Furthermore, we prove the convergence of the proposed scheme with coupling MFT and Q-factor. Finally, simulation results demonstrate that the proposed RA scheme outperforms baseline schemes on delay performance, especially in the heavy traffic load regime.