Get a quote

On the job, on the clock, kollected.

World models learn to predict it. Robots learn to do it.

Real footage2704×2028 · 59.94 fps
  • Hand pose (estimated)
  • Bimanual: both hands in every frame
Robot hands, simulated

Each robot frame is retargeted from the worker's hand pose in the same video frame.

Carpentry
Frame from FV-C00456, 16.0–24.0 s: Carpentry, first-person footage. Measuring and marking out (task 9.1.01, first-pass label).
9.1.01first-passMeasuring and marking out
Metal
Frame from FV-B00023, 122.0–130.0 s: Metal, first-person footage. Grinding and deburring (task 15.5.01, first-pass label).
15.5.01first-passGrinding and deburring
HVAC services
Frame from FV-B00325, 270.0–278.0 s: HVAC services, first-person footage. HVAC equipment installation (task 6.2.01, first-pass label).
6.2.01first-passHVAC equipment installation

Fig. 1 Real trades, keeping time. You get the camera originals.

Fig. 2

A show of hands.

Hover to play. Click to open.Click one to open.Tap one to open.

Footage

Carpentry

Metal

Retail

HVAC services

Laundry

Kitchen

Also playing
Notes (5)
  • Picked by our scouts for visible hands, so not a random sample.
  • Some clips come from the camera's low-resolution proxy; the player says which.
  • Each clip's task is a reviewer's reading, not a label.
  • Faces are cropped or blurred before encoding.
  • Head motion (gyro p95) runs from 2.7 to 159 deg/s.

FootageFig. 2

Clip

Fig. 3

One hand, a billion teachers.

Estimated hand poses, retargeted frame by frame to simulated robot hands.

Render

Render: left, camera frames of a worker fitting a latch on a steel gate; middle and right, simulated Allegro and LEAP robot hands on grey arms following both hands.

k 0 / 239 · source frame 1200 · 20.020 s

Frames solved 1 / 1

  • Shallots: on Why Ego

Trial
Shallots: on Why Ego
0.00 / 8.00 s

Render of our pipeline output. Left: camera frames (re-encoded). Right: simulated robot hands, not camera footage.

Every trial, every frame counted
TrialClipHandsFrames with a robot poseNote
Spring onionsFV-B01149 · 958.1–966.1 sboth240/240 on all four armsLeft-hand grip followed loosely (aperture correlation r -0.25 on the Allegro).
LatchFV-B00476 · 20.0–28.0 sboth240/240 on all four armsThe two robot hands pass into each other in 54 Allegro frames (up to 11.5 mm) and 11 LEAP frames (up to 6.8 mm).
Green beansFV-Z00907 · 20.0–28.0 sright186/240 on both robotsRight hand only. k 184-237: no hand estimate, held and greyed on both robots.
Shallots: on Why Ego

Simulated arms, not speed-limited; no object is modelled, so nothing is held.

Notes (9)
  • Three clips of 8 s at 30 fps, both hands on two of them; robot frame k comes from video frame k.
  • The arms are not speed-limited: up to about 600 deg/s, against 75 to 135 deg/s for a real iiwa. The continuity limits were loosened per clip to 0.1 to 0.12 m and 30 to 35 degrees per frame, and the LEAP hand sits on a stand-in mount.
  • Hand pose is a model's estimate from one RGB camera. No object is modelled, so the robots hold nothing.
  • Across these renders the Allegro thumb misses its target by 31 to 44 mm on average: retargeting maps motion, it cannot give a robot a human thumb.
  • A frame with no robot pose keeps the last solved pose, greyed and counted; no gap is filled in.
  • The 3D replay is posed from the solved joint angles of each frame: a simulation, not camera footage.
  • The ORCA hand option shows the right hand only, retargeted to the ORCA v2 hand model with the wrist fixed. Drawn smoothed for display: each joint angle passes a 5-frame outlier filter and a centred 9-frame Gaussian (no lag), and is interpolated between frames on the video clock; a frame with no estimate stays held.
  • The headline is a figure of speech: the teachers are people at work, not a count of our footage.
  • Arm and hand models: KUKA iiwa 14 and Allegro meshes from cuRobo (BSD-2-Clause); LEAP Hand from LEAP_Hand_Sim (MIT); ORCA v2 hand from orcahand_description (MIT). ORCA is a trademark of ORCA Dexterity, Inc., which does not endorse this site. No maker of these robots or hands made, reviewed or endorses this figure.
Fig. 4

55,088 poses. A handful of patterns.

Two hours of footage, estimated. Eight of 16 directions hold 91.8% of the variance.

Mixed

The ORCA v2 hand model at −2σ, 0 and +2σ along PC1 and PC2 of the hand-posture prior (a rendered sheet, not camera footage).
Illustration, not data.ORCA v2 handORCA v2 hand at the chosen postureORCA v2 hand along one directionOur basis · ORCA v2 right handmean pose
Postures · PC1–PC3KitchenHVAC and electricalCarpentryWelding and fabrication
EigenDEXplore’s published basis · Sharpa left handmean pose

Footage · FV-B01149 · 958.1–966.1 s

k 0 / 239

Kitchen k90 7 · pooled top 8 92.6%
HVAC and electrical k90 8 · pooled top 8 91.9%
Carpentry k90 7 · pooled top 8 92.0%
Welding and fabrication k90 8 · pooled top 8 89.4%

The nearest real hands from each trade along PC1:

−2σ0+2σ

  1. 1 Sixteen dials. Turn each joint alone, and the hand stops looking like work.
  2. 2 One real hand. An estimated right hand, retargeted to the ORCA v2 hand model and smoothed over 9 frames for display.
  3. 3 Every frame, one dot. 6,000 of the 55,088 poses. Pick one to see the nearest real hands.
  4. 4 The main moves. Closing the whole hand: 42.9%. Knuckles against fingertips: 16.8%.
  5. 5 Shared across trades. The pooled top eight hold 89.4% to 92.6% of each trade's variance.
  6. 6 A second opinion. On an Allegro mapping, our top two directions overlap a published basis by 0.81.
Overlap of our top-k directions with EigenDEXplore’s published Allegro basis
0.000.250.500.751.0012345678910111213141516directions kk 1: ours 0.49k 2: ours 0.8103k 3: ours 0.5693k 4: ours 0.5372k 5: ours 0.5818k 6: ours 0.614k 7: ours 0.6276k 8: ours 0.6759k 9: ours 0.7459k 10: ours 0.7692k 11: ours 0.8266k 12: ours 0.8666k 13: ours 0.8879k 14: ours 0.9042k 15: ours 0.9349k 16: ours 1top-2

oursnaive close-all-fingers basisrandom subspaces, 95th percentile

At one direction, ours is no closer than a naive guess (0.490 against 0.488).

A finding about hands at work, not a training result.

Principal directions of 16 finger joints on the ORCA v2 hand model, from estimated poses in four trades.

Notes
Variance held by the first eight directions (pooled)
PC12345678
share42.9%16.8%9.4%7.0%5.1%4.5%3.1%2.9%
cumulative42.9%59.7%69.1%76.1%81.2%85.8%88.9%91.8%
Per trade
tradek90PC1 sharepooled top 8own top 8
Kitchen743.8%92.6%93.3%
HVAC and electrical846.5%91.9%92.2%
Carpentry747.9%92.0%92.5%
Welding and fabrication833.3%89.4%90.4%
  • Poses are monocular estimates; the hand detector and the wearer rule are not scored against labelled frames. PC3 is provisional: PC3 and PC4 change with the retargeter, while PC1, PC2 and the top-4 subspace do not.
  • The prior: PCA of 16 ORCA v2 finger joints in radians, from 40 three-minute spans of four trades and three kit clips. Each trade alone needs 7 or 8 directions to hold 90% of its variance.
  • Beat 1 is an illustration, not data: independent noise on each joint about the prior’s mean pose (a fixed seed), inside the joint limits.
  • Beat 2: footage of FV-B01149 @ 958.057 s, the wall’s loop; the keypoints and the hand model’s pose are HaMeR estimates from one RGB camera, retargeted by the prior’s own retargeter (k 210 is estimated on a box tracker’s box, not the hand detector’s). Drawn smoothed for display: each joint angle passes a 5-frame outlier filter and a centred 9-frame Gaussian (no lag), and is interpolated between frames on the video clock; a frame with no estimate stays held. The human hand is drawn as a keypoint skeleton.
  • Hand crops come from the 20 recordings of set H of the prior’s crops; nearest postures are computed within that set.
  • Beat 6 compares two different hands by eye; the 0.81 overlap and the k = 1 result are measured on an Allegro hand mapping (the kit’s comparison against EigenDEXplore’s published Allegro basis).
  • Hand model: ORCA v2 right hand (orcahand_description, MIT, © 2025 ORCA). ORCA is a trademark of ORCA Dexterity, Inc., which does not endorse this site.
  • Sharpa HA4 left hand: meshes from github.com/tylerlum/simtoolreal @ 313d5ae (MIT, © 2026 Tyler Lum; its package.xml says BSD). Not affiliated with or endorsed by Sharpa.
  • Beat 6 compares our ORCA basis with the Sharpa hand driven by EigenDEXplore’s published Sharpa basis, each sweeping its own first two directions.
  • Sharpa and Allegro bases: EigenDEXplore (hgupt3/eigendexplore @ 5d57180, MIT).
  • The tour runs only while the figure is on screen, stops on any input, and holds on beat 6.
Why real work

Not staged. Not scraped. Not synthetic.

Real shifts. Real tasks.

Camera files, motion included.

Gravity included.

Conditions
  • dust
  • arc glare
  • occupied shop floor
  • confined spaces
  • steam
Fig. 5

See it. Feel it. Name it.

Ten seconds of real work, layer by layer.

01 Capture

From the worker's head, about where a robot's head camera sits.

02 Sync

Every head move, felt 197 times a second.

03 Label

Every task named and coded.

Fig. 6

Now it's in your hands.

Move the playhead. Five streams follow, on one clock.

pitch —roll —heading —
FV-Z00908·10.1.06first-passMise en place setup2704×1520 · 50 fpst = 000.000000 s
x —y —z —

metres · —

x y z · m/s² · rad/s ·

Fig. 7

Body double.

Our re-implementation of Masquerade paints the arms out and draws a robot in.

Edited footage

Kitchen: on Why Ego 

Edited footage: the arms are painted out by a model and a simulated robot is drawn in. Not robot footage.

Edited footage: three square panels of a kneeling worker marking a panel with a try-square; in the middle and right panels the arms are replaced by simulated robot arms with grippers.
k 0

No gripper visibly holds anything.

Notes (8)
  • The arms are painted out by a model and simulated robot arms are drawn in, each posed from the hand estimated in the same frame. No gripper visibly holds anything: held objects are partly painted out with the hands.
  • Our re-implementations of published methods (Masquerade, arXiv 2508.09976; a contact-aware pass after arXiv 2610.06137, whose authors released no code). Not the authors' code or results.
  • Robot frame k comes only from the hand pose estimated in video frame k. Nothing is filled, held or interpolated.
  • Both arms are edited in every frame shown: 240 of 240 per arm in each clip (the HVAC tab shows k 46–239).
  • Carpentry: the square crop keeps another person's limb out of the frame.
  • HVAC: the robot bases were moved 8.2 cm to reach the contact targets.
  • HVAC: frames k 0 to 45 are left out (another person's foot).
  • Metal: the contact-aware right forearm sweeps across about a quarter of the frame at k 0 to 32.

Edited footage: the arms are painted out by a model and a simulated robot is drawn in. Not robot footage.

Edited footage

Edited footage: FV-C01442 frame k 104, square crop, sleeve fix off. A strip of the right sleeve stays human at the arm's outer edge.
Sleeve fix off. A strip of the right sleeve stays human at the arm's outer edge.
Edited footage: FV-C01442 frame k 104, square crop, sleeve fix on. The sleeve strip is gone; the fill still smears where no frame shows the floor.
Sleeve fix on. The sleeve strip is gone; the fill still smears where no frame shows the floor.
Every mask version
  • Edited footage: FV-C01442 frame k 104, square crop, one point per hand. The masks stop at the hands; most of each arm would stay human.One point per handThe masks stop at the hands; most of each arm would stay human.
  • Edited footage: FV-C01442 frame k 104, square crop, along the forearm. Hands and bare forearms are covered, up to the sleeve cuff.Along the forearmHands and bare forearms are covered, up to the sleeve cuff.
  • Edited footage: FV-C01442 frame k 104, square crop, hand to frame edge. Sleeves included, but the outer half of the right cuff is missed in many frames.Hand to frame edgeSleeves included, but the outer half of the right cuff is missed in many frames.
  • Edited footage: FV-C01442 frame k 104, square crop, sleeve regrown. Grown back inside a corridor from the wrist: the right mask no longer stops there.Sleeve regrownGrown back inside a corridor from the wrist: the right mask no longer stops there.
  • Edited footage: FV-C01442 frame k 104, square crop, outline closed. An arm hull closes the right arm's outline and its one-frame cuff misses.Outline closedAn arm hull closes the right arm's outline and its one-frame cuff misses.
  • Edited footage: FV-C01442 frame k 104, square crop, neighbours vote. A pixel both neighbouring frames mask is added; it only adds pixels, never moves the robot.Neighbours voteA pixel both neighbouring frames mask is added; it only adds pixels, never moves the robot.
  • Edited footage: FV-C01442 frame k 104, square crop, composite, fix off. A strip of the right sleeve stays human at the arm's outer edge.Composite, fix offA strip of the right sleeve stays human at the arm's outer edge.
  • Edited footage: FV-C01442 frame k 104, square crop, composite, fix on. The sleeve strip is gone; the fill still smears where no frame shows the floor.Composite, fix onThe sleeve strip is gone; the fill still smears where no frame shows the floor.

More fixes, on one kitchen clip: Why Ego

Mind the sleeve. A mask that stops at the cuff leaves a human sleeve behind.

Edited footage

Edited footage: FV-C01442 frame k 120, square crop, HACO contact estimates on the hands.
HACO contact estimates on the hands
Edited footage: FV-C01442 frame k 120, square crop, contact-aware pads against Masquerade.
Contact-aware pads against Masquerade

Aimed at the touch. Pads moved onto another model's contact estimates. Not measured contact.

Notes
  • Across these 5 clips the pads sit 1.4 to 6.5 mm from another model's contact estimates, against 12.5 to 44.9 mm for Masquerade; those estimates are themselves uncertain by centimetres.
  • The depth test hides at most 13.7% of a gripper on FV-C01442 and fails on four other clips.
Median distance from each gripper pad to the estimated finger contact (thumb pad / opposing pad, mm), right and left hand
ClipContact-aware, rightContact-aware, leftMasquerade, rightMasquerade, left
Carpentry FV-C014425.4 / 6.51.9 / 2.325.3 / 35.830.1 / 44.9
HVAC FV-C017203.9 / 4.62.9 / 3.314.4 / 24.419.1 / 38.9
Metal FV-B019192.0 / 2.91.4 / 1.712.5 / 23.820.0 / 15.7
Small parts FV-B011743.5 / 4.22.2 / 3.215.2 / 16.814.5 / 14.8
Kitchen (on Why Ego) FV-B017744.5 / 5.82.2 / 2.419.9 / 42.619.3 / 13.5
What a lab receives

What's in the box.

The camera original, its IMU on the video clock, labels and metadata.

  1. 1 Pick tasks
  2. 2 Test the free sample
  3. 3 A price in two working days
  4. 4 We film. You measure. We adjust.

Only footage an independent expert confirms meets your criteria is billed. Set your criteria

The proof is in the footage.

Free for teams evaluating a purchase. We reply within one working day.

Get free sample dataGet a quote

  • FV-Z00908.MP4

    2704×1520 · 50 fps704 s · hevc · GPMF

  • FV-Z00908_imu.csv

    timestamp_us,ax, ay, az,gx, gy, gz

    138,653 rows196.95 Hz measured

  • FV-Z00908_labels.json

    "taxonomy_version": "1.0.0","segments": [ …, { "code": "10.1.06", "name": "Mise en place setup", "start_us": 408000000, "end_us": 414000000, "provisional": true }, { "code": "10.1.01", "name": "Vegetable washing and chopping", "start_us": 414000000, "end_us": 418000000, "provisional": true }, … ]

    10.1.06 · 10.1.01provisional

  • FV-Z00908_meta.json

    "device": "HERO10 Black","mount": "head",…"duration_us": 704000000,"resolution": "2704x1520","fps": 50,…

    HERO10 Black · head

Ordering

354 tasks. Pick yours.

Search by task or code, or open a workplace.

  • Event venueStage and equipment setup and teardown
  • AgriculturePlanting and transplanting
  • LaundrySoiled-linen sorting and loading
  • CarpentryMeasuring and marking out
  • Food factoryProduction line operation
  • MetalPart layout and marking
  • ConstructionMaterial handling and stacking
  • HVAC servicesFixture installation (sink, toilet, faucet)
  • Tyre shopTyre dismount and mount
  • CleaningRestroom restock and disinfection
  • AssemblyInjection-moulding machine tending
  • RetailPallet breakdown and sorting
  • Personal careHair cutting and trimming
  • Auto serviceOil and fluid change
  • KitchenVegetable washing and chopping
  • WarehouseReceiving and unloading
  • Event venueLinen and tableware setting
  • AgricultureIrrigation setup and repair
  • LaundryIndustrial washer and dryer operation
  • CarpentryRouting and edge profiling
  • Food factoryIngredient weighing and batching
  • MetalSheet bending and press brake work
  • RetailBackroom organisation and zoning
  • Personal careManicure and pedicure
  • Auto serviceBrake pad and rotor service
  • KitchenLine setup and breakdown
  • WarehouseOrder picking
  • Event venueAV and cable management
  • AgricultureFeeding and watering rounds
  • LaundryIroning and folding
  • CarpentryClamping and glue-up
  • Food factoryQA and foreign-object inspection
  • MetalAssembly tack welding
  • ConstructionFraming and stud layout
  • HVAC servicesHVAC equipment installation
  • Tyre shopPre-soak and chemical application
  • CleaningSweeping and mopping
  • AssemblyLight manufacturing and assembly
  • RetailShelf restocking
  • Personal careTool sterilising and sanitising
  • Auto serviceVehicle lift and jack setup
  • KitchenBeverage and coffee station preparation
  • WarehouseCycle counting and inventory
  • Event venueStaging deck and riser assembly
  • ConstructionInsulation installation
  • HVAC servicesOutlet and switch wiring
  • Tyre shopInterior vacuuming
  • CleaningHigh-touch surface wiping
  • AssemblyTrimming and deflashing
  • RetailOnline-order picking
  • Personal careHair washing and conditioning
  • Auto serviceParts staging and matching
  • KitchenDishwashing and pot scrubbing
  • WarehousePut-away and replenishment
  • Event venueBar and beverage station setup
  • AgricultureGreenhouse and nursery tending
  • Auto serviceFilter replacement (oil, air, cabin)
  • CarpentrySanding and surface preparation
  • Food factoryMixing and blending
  • MetalClamping and fixture setup
  • ConstructionTile setting and grouting
  • HVAC servicesJob setup, drop cloths and area protection
  • Tyre shopHose and equipment reel handling
  • CleaningWaste collection and bin liner changes
  • AssemblyPress and jig operation
  • RetailCart and basket retrieval
See every task
All 16 workplacescollecting now
  1. 2Retail Store — Back of House28
  2. 3Active Construction Site28
  3. 4Agriculture, Farm and Landscaping32
  4. 6Trade Service — Plumbing, HVAC and Electrical29
  5. 8AAuto Service22
  6. 8BTyre Shop and Car Wash23
  7. 9Carpentry and Furniture Workshop29
  8. 10Restaurant and Commercial Kitchen29
  9. 11Commercial Cleaning and Janitorial25
  10. 15Metal Fabrication and Welding29
  11. 1Event Venue12
  12. 5Personal Care and Grooming13
  13. 7Hotel and Lodging15
  14. 12Food and Beverage Manufacturing13
  15. 13Warehouse and Distribution13
  16. 14Plastic and Light Assembly14

Acceptance gates. Both hands in frame for 90% of a clip, no idle stretch over 10 s, identifiable faces in at most 5% of frames. Every published gate

Consent. Every worker signs a licence for machine-learning training; minors are excluded; sites and workers are pseudonymised. The consent policy

Academic lab? Up to 100 hours, free for a citation →