Multi-dimensional digital twin for cable assembly by synthetic data and reinforcement learning

Due to the growing demand for mass customization which it is required for the capability of adaptive handling in tasks like plugging a cable, legacy robotic systems often struggle with these tasks, typically requiring human intervention. The authors introduce an advanced robotic manipulation system that utilizes a digital twin framework to autonomously update its programming for managing the complex handling of cable insertion tasks. By employing synthetic data derived from a virtual environment, this system enhances manufacturing efficiency and adaptability. Our approach minimizes the need for manual labor and frequent engineering adjustments, thereby reducing labor costs and accelerating development time. At the core of our proposed method is the integration of image-based object detection using synthetic data, reinforcement learning for optimal gripping, and a dynamic feedback mechanism designed for complex handling tasks in offline programming. Additionally, our system is capable of creating robot paths that effectively navigate around obstacles using the probabilistic road map algorithm. The proposed system is applied to connect a cable plug into a socket by using an industrial robot equipped with two 3D cameras. The connection task is completed without any human intervention from detecting the status of the cable to put the plug into the socket.