Guide / First-person capture

Egocentric video data collection for robotics

First-person footage can show how a skilled person sees and performs a task. Useful robotics data collection starts by checking whether the required action, object and outcome remain visible — not by assuming a head-mounted camera solves every problem.

When is an egocentric view useful?

Egocentric, or first-person, video places the camera near the demonstrator's point of view. It can document tool choice, task sequence and hand–object interactions in assembly, inspection, packing or technical service. It may be especially relevant when the operator's field of view explains why an action happened.

Its limits matter equally: head motion can blur small components, hands or tools can leave the frame, and a camera does not necessarily see what the operator sees. A fixed or overhead reference view may provide clearer evidence for some tasks. Decide from the target model's requirements and a test recording, not the popularity of a particular camera perspective.

Pilot requirements

Write the capture specification before recording.

Use these questions to define an assessable pilot. Equipment suitability and availability are confirmed per project.

RequirementDefine in the briefCheck in sample footage
Task objectiveActions, objects, start/end states and intended learning or evaluation useCan a reviewer identify a successful and unsuccessful attempt?
Camera viewHead, chest or task-specific placement; field of view; optional fixed referenceAre hands, tools, objects and outcomes continuously observable?
Image qualityResolution, frame rate, lighting, focus and permitted movementAre small parts readable during normal task speed?
VariationsObject sizes, work surfaces, gloves, lighting, task order and approved retriesDoes the sample cover the variants you actually need?
Metadata and labelsTask IDs, clip boundaries, camera identifiers, timestamps and optional outcome labelsDoes the exported schema match the intended data loader?
Rights and safetyParticipant and workplace permissions, safe equipment use and excluded informationAre confidential screens, bystanders and restricted actions absent or handled?
What video does not provide

POV footage is not a robot action trace.

First-person RGB video does not inherently include calibrated 3D hand pose, force, tactile contact, robot joint commands or executable trajectories. These signals may require additional sensors, calibration, labeling or a separate robot-data source. A human demonstration dataset is only useful for training when your pipeline can use the information it contains.

Define any depth, synchronization or sensor requirement independently and test it in a small pilot before committing to collection volume.

Read the robot-learning data guide

A simple pilot acceptance checklist

These are scoping questions, not a promise that any particular rig, location or recording capacity is available. See the collection process and quality and rights framework.

Start with the task

Define a useful pilot.

Describe the capture requirements and intended use. Feasibility is assessed for every project.

Prepare a collection brief