EgoH4RT: Bringing Egocentric Hands and Camera 4D Reconstruction Together

Self-captured Videos

No Ground Truth

Pipetting

1,802 frames · 30 FPS · 60.1 s

We use grayscale input for the hand detector in selected videos featuring colorful gloves.

Benchmark Demos

Long-range Motion

Xperience · 240 frames · 20 FPS · 5.30 m endpoint displacement

Long-range Head and Hand Motion

These continuous videos test whether hand reconstruction remains stable as the wearer moves through the scene. Xperience’s released metric SLAM is shown as a diagnostic camera reference.

Capture 3fd6d29e · ep1

2.00 m endpoint displacement

240 frames · 20 FPS · first-pose rigid alignment

Short-range Head and Hand Motion

HOT3D captures close-range manipulation with large head rotations and rapid hand motion. These clips provide qualitative diagnostics; the quantitative section below follows the final manuscript protocols and explicitly marks HandFlow's overlap with the evaluated HOT3D recordings.

P0014_8254f925_c23

Close-range Manipulation

240 frames · 30 FPS · first-pose rigid alignment

In-the-wild Generalization

These self-captured videos are evaluated without dataset-specific fine-tuning. They contain no 3D ground truth, so we show image-space hand predictions and native camera trajectories.

Long displacement

Long-range Camera Motion

1280 frames · 30 FPS · 42.7 s

Quantitative Evaluation

End-to-end Scaling with Video Length

Raw MP4 to native predictions on one H100 80GB GPU with node-local SSD.

Combined scatter plot of end-to-end runtime and peak GPU memory at 128, 512, 1024, and 1802 source frames — Every point is the arithmetic mean of two measured runs after warm-up. Along each connected trajectory, points proceed from 128 to 512, 1,024, and 1,802 source frames. EgoH4RT, HaWoR, HandFlow and MINT output bilateral hands and camera motion; ACE-Ego-Hand outputs bilateral hands only.

Camera-space Bimanual Hand Reconstruction

Bimanual errors from the final HOT3D evaluation.

millimetres ↓

Dataset	Method	TA-MPJPE	PA-MPJPE	Abs-MPJPE	Root Error
HOT3D	MINT	20.393	8.238	53.465	56.956
	WiLoR	20.949	7.051	50.574	44.051
	HaWoR	15.465	6.251	35.021	41.083
	HandFlow^†	12.722	5.642	21.911	20.004
	ACE-Ego-Hand	11.343	5.921	19.384	17.009
	EgoH4RT	9.398	4.524	16.059	14.704

^† HandFlow training includes 22/27 evaluated HOT3D recordings.

Long-range Camera Reconstruction on Xperience

Four held-out 235–297 s recordings with approximately 10 m trajectory diameter.

millimetres ↓

Method	ATE-SE(3)	ATE-Sim(3)
EgoH4RT	310.193	268.601
HaWoR	1593.024	1500.451
HandFlow	2330.169	2258.552
MINT	2608.152	2345.939
LingBot-Map	1421.826	724.688
ABot-Recon	2371.269	1873.781

Xperience's released metric SLAM supplies the camera reference for this evaluation.

Translation and rotation relative pose error over temporal horizons from one to 220 seconds on Xperience — Relative translation and rotation error over exact temporal gaps from 1 to 220 seconds; all four recordings are pooled by frame pair.

Abstract

Coming soon.

Overview

Coming soon.

BibTeX

Coming soon.

Acknowledgements

Coming soon.