Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning

Jin Gyun Jeong; Ji Heon Oh; Hwanseok Jung; Jin Hyuk Lee; Ismael Nicolas Espinoza Jaramillo; Channabasava Chola; Won Hee Lee; Tae Seong Kim

doi:10.1145/3620679.3620693

Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning

Jin Gyun Jeong, Ji Heon Oh, Hwanseok Jung, Jin Hyuk Lee, Ismael Nicolas Espinoza Jaramillo, Channabasava Chola, Won Hee Lee, Tae Seong Kim

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

Abstract

In this work, we present an approach of long-horizon intelligence that utilizes Sub-goal network based hierarchical reinforcement learning (HRL) for long-horizon tasks by a single-arm robot. Long-horizon (LH) tasks are complicated due to their longer complex sequences and the large number of environmental variables. We attempt to solve the LH learning problem by the Sub-goal network based HRL. The proposed approach is tested in both simulation and hardware environments by a LH task of opening a drawer, grasping and relocating an object, and closing a drawer. Our Sub-goal network based HRL achieves a success rate of 90.3% in completing the LH tasks. Whereas the conventional deep reinforcement learning solution could not complete the LH task.

Original language	English
Title of host publication	ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology
Publisher	Association for Computing Machinery
Pages	88-92
Number of pages	5
ISBN (Electronic)	9798400707438
DOIs	https://doi.org/10.1145/3620679.3620693
Publication status	Published - 15 Jun 2023
Event	13th International Conference on Biomedical Engineering and Technology, ICBET 2023 - Tokyo, Japan Duration: 15 Jun 2023 → 18 Jun 2023

Publication series

Name	ACM International Conference Proceeding Series

Conference

Conference	13th International Conference on Biomedical Engineering and Technology, ICBET 2023
Country/Territory	Japan
City	Tokyo
Period	15/06/23 → 18/06/23

Bibliographical note

Publisher Copyright:
© 2023 ACM.

Keywords

Hierarchical Reinforcement Learning
Long-horizon
Robot Manipulation

Access to Document

10.1145/3620679.3620693

Cite this

Jeong, J. G., Oh, J. H., Jung, H., Lee, J. H., Espinoza Jaramillo, I. N., Chola, C., Lee, W. H., & Kim, T. S. (2023). Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning. In ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology (pp. 88-92). (ACM International Conference Proceeding Series). Association for Computing Machinery. https://doi.org/10.1145/3620679.3620693

Jeong, Jin Gyun ; Oh, Ji Heon ; Jung, Hwanseok et al. / Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning. ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology. Association for Computing Machinery, 2023. pp. 88-92 (ACM International Conference Proceeding Series).

@inproceedings{9b871c6915184b109bc12c28795b2f35,

title = "Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning",

abstract = "In this work, we present an approach of long-horizon intelligence that utilizes Sub-goal network based hierarchical reinforcement learning (HRL) for long-horizon tasks by a single-arm robot. Long-horizon (LH) tasks are complicated due to their longer complex sequences and the large number of environmental variables. We attempt to solve the LH learning problem by the Sub-goal network based HRL. The proposed approach is tested in both simulation and hardware environments by a LH task of opening a drawer, grasping and relocating an object, and closing a drawer. Our Sub-goal network based HRL achieves a success rate of 90.3% in completing the LH tasks. Whereas the conventional deep reinforcement learning solution could not complete the LH task.",

keywords = "Hierarchical Reinforcement Learning, Long-horizon, Robot Manipulation",

author = "Jeong, {Jin Gyun} and Oh, {Ji Heon} and Hwanseok Jung and Lee, {Jin Hyuk} and {Espinoza Jaramillo}, {Ismael Nicolas} and Channabasava Chola and Lee, {Won Hee} and Kim, {Tae Seong}",

note = "Publisher Copyright: {\textcopyright} 2023 ACM.; 13th International Conference on Biomedical Engineering and Technology, ICBET 2023 ; Conference date: 15-06-2023 Through 18-06-2023",

year = "2023",

month = jun,

day = "15",

doi = "10.1145/3620679.3620693",

language = "English",

series = "ACM International Conference Proceeding Series",

publisher = "Association for Computing Machinery",

pages = "88--92",

booktitle = "ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology",

}

Jeong, JG, Oh, JH, Jung, H, Lee, JH, Espinoza Jaramillo, IN, Chola, C, Lee, WH & Kim, TS 2023, Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning. in ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology. ACM International Conference Proceeding Series, Association for Computing Machinery, pp. 88-92, 13th International Conference on Biomedical Engineering and Technology, ICBET 2023, Tokyo, Japan, 15/06/23. https://doi.org/10.1145/3620679.3620693

Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning. / Jeong, Jin Gyun; Oh, Ji Heon; Jung, Hwanseok et al.
ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology. Association for Computing Machinery, 2023. p. 88-92 (ACM International Conference Proceeding Series).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning

AU - Jeong, Jin Gyun

AU - Oh, Ji Heon

AU - Jung, Hwanseok

AU - Lee, Jin Hyuk

AU - Espinoza Jaramillo, Ismael Nicolas

AU - Chola, Channabasava

AU - Lee, Won Hee

AU - Kim, Tae Seong

PY - 2023/6/15

Y1 - 2023/6/15

N2 - In this work, we present an approach of long-horizon intelligence that utilizes Sub-goal network based hierarchical reinforcement learning (HRL) for long-horizon tasks by a single-arm robot. Long-horizon (LH) tasks are complicated due to their longer complex sequences and the large number of environmental variables. We attempt to solve the LH learning problem by the Sub-goal network based HRL. The proposed approach is tested in both simulation and hardware environments by a LH task of opening a drawer, grasping and relocating an object, and closing a drawer. Our Sub-goal network based HRL achieves a success rate of 90.3% in completing the LH tasks. Whereas the conventional deep reinforcement learning solution could not complete the LH task.

AB - In this work, we present an approach of long-horizon intelligence that utilizes Sub-goal network based hierarchical reinforcement learning (HRL) for long-horizon tasks by a single-arm robot. Long-horizon (LH) tasks are complicated due to their longer complex sequences and the large number of environmental variables. We attempt to solve the LH learning problem by the Sub-goal network based HRL. The proposed approach is tested in both simulation and hardware environments by a LH task of opening a drawer, grasping and relocating an object, and closing a drawer. Our Sub-goal network based HRL achieves a success rate of 90.3% in completing the LH tasks. Whereas the conventional deep reinforcement learning solution could not complete the LH task.

KW - Hierarchical Reinforcement Learning

KW - Long-horizon

KW - Robot Manipulation

UR - http://www.scopus.com/inward/record.url?scp=85180539737&partnerID=8YFLogxK

U2 - 10.1145/3620679.3620693

DO - 10.1145/3620679.3620693

M3 - Conference contribution

AN - SCOPUS:85180539737

T3 - ACM International Conference Proceeding Series

SP - 88

EP - 92

BT - ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology

PB - Association for Computing Machinery

T2 - 13th International Conference on Biomedical Engineering and Technology, ICBET 2023

Y2 - 15 June 2023 through 18 June 2023

ER -

Jeong JG, Oh JH, Jung H, Lee JH, Espinoza Jaramillo IN, Chola C et al. Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning. In ICBET 2023 - Proceedings of 2023 13th International Conference on Biomedical Engineering and Technology. Association for Computing Machinery. 2023. p. 88-92. (ACM International Conference Proceeding Series). doi: 10.1145/3620679.3620693

Long-Horizon Manipulation by a Single-arm Robot via Sub-goal Network based Hierarchical Reinforcement Learning

Abstract

Publication series

Conference

Bibliographical note

Keywords

Access to Document

Other files and links

Fingerprint

Cite this