Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication

Pyae Sone Aung; Loc X. Nguyen; Yan Kyaw Tun; Zhu Han; Choong Seon Hong

doi:10.1109/NOMS56928.2023.10154352

Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication

Pyae Sone Aung, Loc X. Nguyen, Yan Kyaw Tun, Zhu Han, Choong Seon Hong

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

6 Citations (Scopus)

Abstract

The significant growth in data consumption among mobile users necessitates the development of new architecture to meet the increasing demand. On the other hand, reconfigurable intelligent surface (RIS) has grown in popularity in 6G due to its improved spectral efficiency, simplicity of deployment, and low cost. However, with the constrained limitation of the coverage by conventional RIS, the research direction has turned towards simultaneously transmitting and reflecting RIS (STAR-RIS) to provide 360° coverage alongside the benefits of RIS. In this paper, a STAR-RIS-assisted downlink communication system for both indoor and outdoor users is investigated. Then, the optimization problem to maximize the spectral efficiency while jointly controlling the beamforming power for each user and phase shift values of the STAR-RIS is formulated. Since the formulated problem is NP-hard and challenging to solve in polynomial time, a policy gradient method for reinforcement learning named proximal policy optimization (PPO) is implemented to solve the problem. To demonstrate the effectiveness of our proposed algorithm, extensive simulation results are executed. Numerical results prove that our proposed algorithm outperforms several benchmark schemes in the literature.

Original language	English
Title of host publication	Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023
Editors	Kemal Akkaya, Olivier Festor, Carol Fung, Mohammad Ashiqur Rahman, Lisandro Zambenedetti Granville, Carlos Raniery Paula dos Santos
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781665477161
DOIs	https://doi.org/10.1109/NOMS56928.2023.10154352
Publication status	Published - 2023
Event	36th IEEE/IFIP Network Operations and Management Symposium, NOMS 2023 - Miami, United States Duration: 8 May 2023 → 12 May 2023

Publication series

Name	Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023

Conference

Conference	36th IEEE/IFIP Network Operations and Management Symposium, NOMS 2023
Country/Territory	United States
City	Miami
Period	8/05/23 → 12/05/23

Bibliographical note

Publisher Copyright:
© 2023 IEEE.

Keywords

Reconfigurable intelligent surface (RIS)
deep reinforcement learning
proximal policy optimization
simultaneously transmission and reflection

Access to Document

10.1109/NOMS56928.2023.10154352

Cite this

Aung, P. S., Nguyen, L. X., Tun, Y. K., Han, Z., & Hong, C. S. (2023). Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication. In K. Akkaya, O. Festor, C. Fung, M. A. Rahman, L. Z. Granville, & C. R. P. dos Santos (Eds.), Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023 (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/NOMS56928.2023.10154352

Aung, Pyae Sone ; Nguyen, Loc X. ; Tun, Yan Kyaw et al. / Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication. Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023. editor / Kemal Akkaya ; Olivier Festor ; Carol Fung ; Mohammad Ashiqur Rahman ; Lisandro Zambenedetti Granville ; Carlos Raniery Paula dos Santos. Institute of Electrical and Electronics Engineers Inc., 2023. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023).

@inproceedings{f407c91a65144524870f14975921ced6,

title = "Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication",

abstract = "The significant growth in data consumption among mobile users necessitates the development of new architecture to meet the increasing demand. On the other hand, reconfigurable intelligent surface (RIS) has grown in popularity in 6G due to its improved spectral efficiency, simplicity of deployment, and low cost. However, with the constrained limitation of the coverage by conventional RIS, the research direction has turned towards simultaneously transmitting and reflecting RIS (STAR-RIS) to provide 360° coverage alongside the benefits of RIS. In this paper, a STAR-RIS-assisted downlink communication system for both indoor and outdoor users is investigated. Then, the optimization problem to maximize the spectral efficiency while jointly controlling the beamforming power for each user and phase shift values of the STAR-RIS is formulated. Since the formulated problem is NP-hard and challenging to solve in polynomial time, a policy gradient method for reinforcement learning named proximal policy optimization (PPO) is implemented to solve the problem. To demonstrate the effectiveness of our proposed algorithm, extensive simulation results are executed. Numerical results prove that our proposed algorithm outperforms several benchmark schemes in the literature.",

keywords = "Reconfigurable intelligent surface (RIS), deep reinforcement learning, proximal policy optimization, simultaneously transmission and reflection",

author = "Aung, {Pyae Sone} and Nguyen, {Loc X.} and Tun, {Yan Kyaw} and Zhu Han and Hong, {Choong Seon}",

note = "Publisher Copyright: {\textcopyright} 2023 IEEE.; 36th IEEE/IFIP Network Operations and Management Symposium, NOMS 2023 ; Conference date: 08-05-2023 Through 12-05-2023",

year = "2023",

doi = "10.1109/NOMS56928.2023.10154352",

language = "English",

series = "Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

editor = "Kemal Akkaya and Olivier Festor and Carol Fung and Rahman, {Mohammad Ashiqur} and Granville, {Lisandro Zambenedetti} and {dos Santos}, {Carlos Raniery Paula}",

booktitle = "Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023",

address = "United States",

}

Aung, PS, Nguyen, LX, Tun, YK, Han, Z & Hong, CS 2023, Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication. in K Akkaya, O Festor, C Fung, MA Rahman, LZ Granville & CRP dos Santos (eds), Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023. Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023, Institute of Electrical and Electronics Engineers Inc., 36th IEEE/IFIP Network Operations and Management Symposium, NOMS 2023, Miami, United States, 8/05/23. https://doi.org/10.1109/NOMS56928.2023.10154352

Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication. / Aung, Pyae Sone; Nguyen, Loc X.; Tun, Yan Kyaw et al.
Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023. ed. / Kemal Akkaya; Olivier Festor; Carol Fung; Mohammad Ashiqur Rahman; Lisandro Zambenedetti Granville; Carlos Raniery Paula dos Santos. Institute of Electrical and Electronics Engineers Inc., 2023. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication

AU - Aung, Pyae Sone

AU - Nguyen, Loc X.

AU - Tun, Yan Kyaw

AU - Han, Zhu

AU - Hong, Choong Seon

PY - 2023

Y1 - 2023

N2 - The significant growth in data consumption among mobile users necessitates the development of new architecture to meet the increasing demand. On the other hand, reconfigurable intelligent surface (RIS) has grown in popularity in 6G due to its improved spectral efficiency, simplicity of deployment, and low cost. However, with the constrained limitation of the coverage by conventional RIS, the research direction has turned towards simultaneously transmitting and reflecting RIS (STAR-RIS) to provide 360° coverage alongside the benefits of RIS. In this paper, a STAR-RIS-assisted downlink communication system for both indoor and outdoor users is investigated. Then, the optimization problem to maximize the spectral efficiency while jointly controlling the beamforming power for each user and phase shift values of the STAR-RIS is formulated. Since the formulated problem is NP-hard and challenging to solve in polynomial time, a policy gradient method for reinforcement learning named proximal policy optimization (PPO) is implemented to solve the problem. To demonstrate the effectiveness of our proposed algorithm, extensive simulation results are executed. Numerical results prove that our proposed algorithm outperforms several benchmark schemes in the literature.

AB - The significant growth in data consumption among mobile users necessitates the development of new architecture to meet the increasing demand. On the other hand, reconfigurable intelligent surface (RIS) has grown in popularity in 6G due to its improved spectral efficiency, simplicity of deployment, and low cost. However, with the constrained limitation of the coverage by conventional RIS, the research direction has turned towards simultaneously transmitting and reflecting RIS (STAR-RIS) to provide 360° coverage alongside the benefits of RIS. In this paper, a STAR-RIS-assisted downlink communication system for both indoor and outdoor users is investigated. Then, the optimization problem to maximize the spectral efficiency while jointly controlling the beamforming power for each user and phase shift values of the STAR-RIS is formulated. Since the formulated problem is NP-hard and challenging to solve in polynomial time, a policy gradient method for reinforcement learning named proximal policy optimization (PPO) is implemented to solve the problem. To demonstrate the effectiveness of our proposed algorithm, extensive simulation results are executed. Numerical results prove that our proposed algorithm outperforms several benchmark schemes in the literature.

KW - Reconfigurable intelligent surface (RIS)

KW - deep reinforcement learning

KW - proximal policy optimization

KW - simultaneously transmission and reflection

UR - http://www.scopus.com/inward/record.url?scp=85164681616&partnerID=8YFLogxK

U2 - 10.1109/NOMS56928.2023.10154352

DO - 10.1109/NOMS56928.2023.10154352

M3 - Conference contribution

AN - SCOPUS:85164681616

T3 - Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023

BT - Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023

A2 - Akkaya, Kemal

A2 - Festor, Olivier

A2 - Fung, Carol

A2 - Rahman, Mohammad Ashiqur

A2 - Granville, Lisandro Zambenedetti

A2 - dos Santos, Carlos Raniery Paula

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 36th IEEE/IFIP Network Operations and Management Symposium, NOMS 2023

Y2 - 8 May 2023 through 12 May 2023

ER -

Aung PS, Nguyen LX, Tun YK, Han Z, Hong CS. Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication. In Akkaya K, Festor O, Fung C, Rahman MA, Granville LZ, dos Santos CRP, editors, Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023. Institute of Electrical and Electronics Engineers Inc. 2023. (Proceedings of IEEE/IFIP Network Operations and Management Symposium 2023, NOMS 2023). doi: 10.1109/NOMS56928.2023.10154352

Deep Reinforcement Learning based Spectral Efficiency Maximization in STAR-RIS-Assisted Indoor Outdoor Communication

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this