task · first seen 7 Oct 2026
3D hand pose estimation
Also: hand pose estimation, 3D hand reconstruction, hand kinematics estimation, monocular hand mesh estimation
Estimating three-dimensional hand joint positions and motion from images or video.
3D hand pose estimation recovers the 3D positions of hand joints, and often a full hand mesh such as MANO, from images, video or other signals like EMG. It has been a standard computer vision task for about a decade. The sources show it being reworked for robot learning from human video. Papers from KAIST, Microsoft Research Asia and others build hand trackers on top of video foundation models such as V-JEPA, recover metric hand and camera trajectories from egocentric footage, and fuse vision with forearm sEMG. A fine-tuned SAM 3D Body hand branch (FAH) already has an Apple-silicon Metal port.
The task itself is long-established. The recent activity reflects its new role as a data source for dexterous robot policies, so better metric, physically consistent hand tracking from ordinary video could lower the cost of collecting manipulation demonstrations.
Analyst note (established), 18 Nov 2026.
Mentions per week
PapersCodeDiscussion
Rank 88 on 15 Nov 2026 · 8 mentions in 28 days · 8 independent groups · papers → code · first seen 6 weeks ago · acceleration +0.12
Earliest sources
- RLHND: Video Foundation Models as Physically Grounded Hand Trackers for Robot LearningPapers · 7 Oct 2026 · Seungjun Moon, Subin Jeon, Sangwoo Kim, Hanbyul Joo, Jinwoo Shin
- Video-Conditioned Generative Joint 2D-3D Hand Motion RecoveryPapers · 7 Oct 2026 · Chen Xu, Yunqi Li, Binbin Huang, Brent Yi, Shenghua Gao, Yi Ma
- V-JEPA Hand: Simple, Accurate and Fast 3D Hand and Camera Trajectory Recovery from Egocentric VideosPapers · 19 Oct 2026 · Ruicheng Li, Qixiu Li, Yu Deng, Sicheng Xu, Jiaolong Yang, Baining Guo
- VERA-Hand: A Verified Multidimensional Benchmark for 2D Hand Pose Estimation in Third-Person Dynamic Gesture VideosPapers · 23 Oct 2026 · Gibran Benitez-Garcia, Hiroki Takahashi
- EMG Tokenization Preserves Movement Information at a Fraction of the BitratePapers · 23 Oct 2026 · Raul C. Sîmpetru, Amin Olamazadeh, Devon Rohlf, Vlad Cnejevici, Alessandro Del…
- noahcao/FAHCode · 3 Nov 2026 · noahcao · 92 stars
Latest sources
- WinDEX: Metric 3D Hand Motion Reconstruction in the Wild for Dexterous Teleoperation and Policy LearningPapers · 16 Nov 2026 · Hongyu Ma, Hairong Qu, Shiqi Zhao, Yongsong Yang, Kai Shi, Jiahao Ai, Xin You, P…
- DoF-Query Transformer for Adaptive Fusion of Egocentric Vision and Forearm HD-sEMG in Continuous Hand KinematicsPapers · 15 Nov 2026 · Jiahang Liu, Wenbo Zhang, Junsong Wang, Zhenzhi Ying, Siqi Hou, Naohiko Sugita,…
- LuckyIYI/fah-metalCode · 7 Nov 2026 · LuckyIYI · 42 stars
- S2HAND: Accuracy-Preservation Trade-Offs for Hand Specialization in Whole-Body Pose ModelsPapers · 7 Nov 2026 · Singyu Muk, Xuanren Song, Kalin Stefanov
- Fast and Accurate Hand Estimation in Pixels, Interaction and World SpacePapers · 5 Nov 2026 · Jinkun Cao
- noahcao/FAHCode · 3 Nov 2026 · noahcao · 92 stars
- EMG Tokenization Preserves Movement Information at a Fraction of the BitratePapers · 23 Oct 2026 · Raul C. Sîmpetru, Amin Olamazadeh, Devon Rohlf, Vlad Cnejevici, Alessandro Del…
- VERA-Hand: A Verified Multidimensional Benchmark for 2D Hand Pose Estimation in Third-Person Dynamic Gesture VideosPapers · 23 Oct 2026 · Gibran Benitez-Garcia, Hiroki Takahashi
- V-JEPA Hand: Simple, Accurate and Fast 3D Hand and Camera Trajectory Recovery from Egocentric VideosPapers · 19 Oct 2026 · Ruicheng Li, Qixiu Li, Yu Deng, Sicheng Xu, Jiaolong Yang, Baining Guo
- Video-Conditioned Generative Joint 2D-3D Hand Motion RecoveryPapers · 7 Oct 2026 · Chen Xu, Yunqi Li, Binbin Huang, Brent Yi, Shenghua Gao, Yi Ma
- RLHND: Video Foundation Models as Physically Grounded Hand Trackers for Robot LearningPapers · 7 Oct 2026 · Seungjun Moon, Subin Jeon, Sangwoo Kim, Hanbyul Joo, Jinwoo Shin