Segment Anything Model Aided Beam Prediction for the Millimeter Wave Communication

Millimeter wave (mmWave) and terahertz (THz) communication protocols employ extensive antenna arrays to ensure reliable signal reception. However, aligning these narrow beams incurs a significant cost in beam training, which increases proportionally with the number of antennas. While beam prediction methods based on semantics from the RGB images demonstrate initial feasibility, they are still low on accuracy. Motivated by the fact that the recently introduced Segment Anything Model (SAM) can generate very accurate masks, which can be considered semantic information in this domain, this study proposes a beam prediction solution based on the masks of SAM. The SAM can extract more accurate semantic data from visual sources. Then instead of using the RGB images directly, the semantics images have been used to predict the beamforming vector using a lightweight LeNet5 model. The experimental results show that the SAM-based proposed method can perform significantly better than the two state-of-the-art deep learning models. The proposed solution method can achieve near 100% in top-5 beam prediction accuracy in real-world communication scenarios.