Real human work, captured and audited frame by frame.
Egocentric video + IMU + hand-tracking of contact-rich trade work — the physics kitchen datasets don't have: heavy tools, deformable materials, real friction. Understood, not just recorded, and delivered as a LeRobotDataset with per-field confidence and truth state, per-clip provenance, and consent + license you can verify. Not synthetic. Not scraped.
Why labs pick this data — the full caseEvery worker is a synchronized multi-sensor rig.
Each stream carries an explicit presence state and a capture gate — a missing or degraded modality is named MISSING_SENSOR, never silently assumed present.
Egocentric video
Head-mounted action camera, configured up to 4K60 — the worker's exact point of view through real tool use, with consent-gated 48 kHz audio (suppressed and marked missing in two-party-consent states).
Camera-mount motion
200 Hz accelerometer + gyroscope on the helmet mount — ego-motion context for every frame, aligned to the video clock.
Dominant-wrist kinematics
100 Hz raw inertial + 50 Hz fused orientation on the working wrist, capture-gated (≥98% delivered samples required per stream) — measured motion that keeps flowing when the camera can't see the hand.
Body telemetry
Pocket-phone IMU, 1 Hz GPS, and barometer — coarse body motion, location, and elevation context, plus the worker's own task tags and error markers as first-class events.
One synchronized clock
All streams aligned onto one timebase with published per-stream offsets and drift budgets. Video↔IMU agreement is computed per moment and published as the confidence signal — disagreement widens uncertainty instead of hiding.
Hand-landmark trajectories
License-clean (Apache-2.0) 21-keypoint hand landmarks per frame, each marked direct, interpolated, or withheld — a confidence-graded sidecar, never passed off as metric ground truth.
During real tool use, the working hand is occluded. We say so.
Anyone who has trained on egocentric manipulation data knows the frames that matter most are the frames where the tool hides the hand. We don't claim to see through the tool — we engineer around it and publish the seams.
The wrist keeps measuring
The dominant-wrist IMU is a real sensor, not an inference — it keeps producing measured kinematics through the exact moments the tool hides the hand from the camera.
Per-frame landmark status, not bravado
Every hand-landmark frame is explicitly marked direct, interpolated, or withheld. You can filter to direct-only frames in one line — we don't launder interpolation into observation.
Agreement is the confidence signal
Per-moment video↔IMU agreement ships with the data. It dips honestly during occlusion — which is exactly the signal a training pipeline needs to weight those frames correctly.
Fail-closed, with named gates
A span that can't be evidenced is withheld with the gate that failed — a structured rejection, never a padded label. You get fewer labels than a vendor who guesses. That's the point.
Every field carries its evidence, confidence, and truth state.
We don't flatten uncertainty into a single score. Each labeled moment records what was observed, how confident the system is, where the evidence is in the footage, and whether a human confirmed it. Missing data renders as missing — never as a guess.
{
"action": "seat fastener with impact driver",
"evidence_span_ms": [161240, 162480],
"confidence": 0.83,
"truth_state": "WORKER_CONFIRMED",
"cross_stream_agreement": "video↔wrist_imu: AGREE",
"source": "egocentric_video + wrist_imu",
"code_citation": "contact_inference/contact.py:contact_active"
}MEASUREDRead directly from a sensor or pixel.INFERREDDerived; carries its confidence.WORKER_CONFIRMEDThe worker reviewed and confirmed it.DEGRADEDPresent but low quality; flagged.UNKNOWNNot determinable from the evidence.REJECTEDFailed a gate; never shipped as fact.
We publish what we don't know.
Missing shows as missing. Unconfirmed never ships as confirmed. A clip that fails its quality bar gets a structured rejection with a named gate — never a silent, padded result. Expert buyers trust a vendor that discloses its limits; so do we.
Provenance you can check, consent you can audit.
Consent-based capture
Footage is recorded by consenting workers on real job sites under signed agreements — not scraped from the web.
PII redaction
Faces, plates, badges, and names are blurred before any buyer-facing release — a clip that fails the blur pass is rejected, not shipped. Where two-party-consent law requires it, audio is suppressed.
License-clean & provenance-tracked
Acquisition method is documented per clip. Every published claim links to the decision that produced it, with a per-clip datasheet.
Human-in-the-loop ownership
Workers review and confirm labels and are paid for their expertise. Unconfirmed findings stay unconfirmed.
Evaluate a sample episode, then the datasheet.
We'll share a sample episode (RLDS / LeRobot-ready) and the per-clip datasheet so your team can judge usefulness before any commitment.
Request access →