Smart D2D Resource Management in H-CRANs using Asynchronous Federated Reinforcement Learning

Abstract

Efficient mode selection and resource allocation remain critical challenges in Device-to-device (D2D) communication within heterogeneous cloud radio access networks (H-CRANs), especially in the context of 5G-Vehicle-to-everything (5G V2X) systems. Conventional centralized approaches suffer from high communication overhead, synchronization delays, sensitivity to independent and identically distributed (IID) data, and significant privacy concerns. To address these limitations, this paper proposes a novel framework that combines Asynchronous Federated Deep Reinforcement Learning (AF-DRL) with Federated Averaging. In the proposed approach, D2D agents learn optimal transmission policies locally without exchanging raw data and asynchronously transmit model updates to a central server. The server aggregates these updates to form a global model, which is redistributed to agents in an iterative manner. This decentralized learning paradigm ensures scalability, privacy preservation, and robustness against IID data. Extensive simulations validate the effectiveness of the framework, demonstrating a 20 % improvement in user satisfaction, 15 % improvement in resource utilization, 20 % reduction in latency, and 25 % improvement in throughput compared to state-of-the-art methods. These results highlight the potential of the proposed method for enhancing resource management in next-generation vehicular communication systems.