Translation as a Bridging Action: Transferring Manipulation Skills from Humans to Robots

Key points

• The paper argues that rotation information from human hands is noisy and that rotation-inclusive 6DoF learning is suboptimal because the human hand and parallel gripper have different contact patterns.
• It adopts relative wrist translation in the head-camera frame as a bridging action and builds a shared action space between humans and robots.
• An interleaved action token sequence, 3D-wrist to 6D-eef to gripper, together with attention masking, handles missing components in heterogeneous data.
• The training strategy has three stages: large-scale human-data pretraining, joint human-robot training, and a small amount of real-robot fine-tuning.
• Adding bridging-action supervision to robot data is essential, raising overall success from 12.5 percent to 38.3 percent.
• The upper-bound analysis shows that removing the embodiment gap lets the bridging objective achieve even stronger transfer performance, with overall progress rising from 59.75 percent to 73.54 percent.

Paper links

External research summaries. These are not HDATF publications or measured product results.