Counterfactual Multi-Agent Reinforcement Learning for Long- Horizon Medical Assistive Tasks with Dual-arm Robot

Dual-arm robots hold significant potential for performing medical assistive tasks in healthcare environments. However, executing such diverse and complex tasks requires advanced dual-arm robot intelligence, which faces substantial challenges due to multi-agent interactions in sequential long- horizon (LH) actions. This study introduces a novel multi-agent reinforcement learning approach, termed Counterfactual Multi-Agent Demo Augmented Policy Gradient (COMA-DAPG), to learn and perform LH medical assistive tasks for dual-arm robots. The proposed COMA-DAPG integrates a counterfactual critic network and demonstration-augmented policy gradient (DAPG) with three designed reward functions. Our experimental results demonstrate that COMA-DAPG outperforms each COMA and DAPG with over 25% improvement in average success rate across three LH tasks.Clinical Relevance - COMA-DAPG addresses key challenges in dual-arm robotics, such as credit assignment, gradient variance, and collision avoidance, to enable precise, cooperative execution of complex medical tasks, enhancing reliability and efficiency in clinical care settings.