Data collection

Two ways to record, one raw format

Both paths first produce the same LeRobot v2.1 raw dataset and can be replayed through the dataset transport. They differ only in what each frame contains. LeRobot v3.0, HDF5, and MCAP are post-QC export targets, not alternate live writers.

• Teleoperation collection — a leader device drives the robot; EVA records the robot's pose alongside the commanded action. Runs from the COLLECT, enabled by a collection schema in the config.
• Inference runs — a trained policy drives the robot; EVA records the robot's pose paired with the action the policy sent. This is what SFT consumes.

Before you record

• EVA installed — the eva command must resolve.
• Live robot connection — powered on and reachable over ros1, ros2, or zmq. Dataset-only configs cannot collect.
• Teleoperation source — either a transport-side Leader is publishing, or a configured client input such as the WebXR VR node is healthy.
• A collection config — defines a collection schema. Examples ship under configs/02_collection/.

Fill in the collection config

A config becomes a collection config once it lists the data columns to save (collection.schema.columns). EVA validates the schema on launch and refuses to start if anything required is missing.

Inside schema:

• Arms map each arm to a short prefix (e.g. left) used in per-joint column names like left.j0, left.gripper.
• Cameras map each camera to its dataset video name.
• Columns name four fields: qpos, eef, action_qpos, action_eef.
• min_episode_frames sets the shortest episode kept without a flag; shorter takes are flagged for review.

Start from the matching example under configs/02_collection/ and edit the marked values. Keep the column names unless your training pipeline expects different ones — they are the dataset's contract:

_base_ = ['../01_deploy/<your-robot>/_base.py']   # ← reuse your robot's deploy config

collection = dict(
    storage=dict(
        log_dir='<output folder>',    # ← where datasets are written; blank = work_dirs/<config name>
        fps=30,                    # target frame rate (sane default; lower for slow tasks)
        save_queue_max=15,          # how many finished episodes may wait to be saved
    ),
    schema=dict(
        robot_type='<your robot type>',   # ← e.g. agilex_piper
        min_episode_frames=10,           # shorter takes get an 'episode_too_short' flag
        arms=dict(left_arm='left', right_arm='right'),   # ← your arms → name prefixes
        cameras=dict(
            cam_high='observation.images.cam_high',   # ← your camera → its dataset name
        ),
        columns=dict(           # keep these names unless training needs others
            qpos='observations.state.qpos',
            eef='observations.state.eef',
            action_qpos='action.qpos',
            action_eef='action.eef',
        ),
    ),
    teleop=dict(control_source='transport'),  # hardware-side Leader; use 'client' for VR
    tasks=dict(
        pick_up_the_apple=[('pick up the apple', -1)],  # UI key → (prompt, repeat limit)
    ),
)

If log_dir is blank, EVA writes to work_dirs/<config name>/.

Run a collection

Launch EVA with the collection config; the COLLECT opens in the browser:

eva --config configs/02_collection/<your-config>.py

The full button workflow is on the COLLECT page. Each take runs on a background pipeline so live capture never stalls:

• Start — recording begins, frame buffers reset.
• Capture — frames are grabbed instantly and handed off for background processing. Sustained over-capture grows memory and logs a warning rather than dropping data.
• Skip before ready — frames before the teleop link is calibrated are skipped (no commanded action yet) rather than saved with empty actions.
• Stop — recording ends, every frame is checked, and the episode is handed to a background saver. Zero-frame takes save nothing.
• Cancel — discards the current take; nothing is written.

The console reports live status: collecting / saving / idle, frame count, save-queue depth, estimated time to flush. Each kept take is appended to the dataset for that task; recording never overwrites earlier episodes.

Confirm a recording worked

After STOP, wait for status to return to idle. A successful take adds:

• a new episode_NNNNNN.parquet under data/chunk-000/;
• one episode_NNNNNN.mp4 per camera under videos/chunk-000/;
• a new line in meta/episodes.jsonl with the episode's quality flag.

Each line records the episode's quality: green if every frame passed, red if any check failed. Red episodes are saved (bad values repaired, see below) but should be replayed from COLLECT or REPLAY before deciding to re-record.

What the dataset looks like

One folder per task. Episodes are six-digit numbered (episode_000000…) inside chunk-000. A chunk caps at 1000 episodes; EVA only fills the first.

<log_dir>/<task name>/
├── data/
│   └── chunk-000/
│       └── episode_000000.parquet      # main table, 1 row per frame
├── videos/
│   └── chunk-000/
│       └── <camera>/
│           └── episode_000000.mp4       # one mp4 per camera
└── meta/
    ├── info.json            # dataset metadata: format version, fps, columns
    ├── episodes.jsonl       # one line per episode: number, task, length, quality
    ├── episodes_stats.jsonl # per-episode statistics (collection only)
    ├── tasks.jsonl          # the task text for each episode
    └── stats.json           # whole-dataset statistics

A worked example ships at examples/agilex_dataset/ (dual-arm AGILEX, three cameras).

Per-frame quality checks

Every frame is checked on save. Any failure marks the episode red and the reason is stored in episodes.jsonl; otherwise the episode is green.

Missing or wrong-sized fields are replaced with zeros of the correct length so the table stays rectangular; the failure is flagged. Red episodes remain usable but are worth checking.

How timestamps are recorded

LeRobot requires perfectly even frame times. EVA synthesizes the official timestamp from the configured fps and stores the real capture time in a separate capture_time column for diagnostics.

• Stored fps equals the configured target, not a measured average.
• Measured average frame rate (jitter) is recorded per episode for diagnostics.
• Each camera video runs at the target fps, exactly one video frame per data row.

Dataset metadata and statistics

On finalization EVA writes info.json and stats.json:

• Column descriptions — for each video, size and codec; for each data field, type and per-element labels (joints as <arm>.j0…/<arm>.gripper, end-effector as <arm>.{x,y,z,qw,qx,qy,qz,gripper}).
• Statistics — per-feature min/max/mean/std, computed per episode and rolled up. Training tools use these to normalize.
• Tasks — instruction text per episode.

Inference runs as datasets

Inference runs write the same format with two differences:

• Each row pairs robot state with the action the policy sent — one row per executed step. End-effector pose is saved as a second state column if the deployment tracks it; joint-only runs omit it.
• Eval runs save immediately on STOP so the trial is scorable. The score lives in dataset metadata, never as a data column. Re-running the same prompt-and-trial combination overwrites that episode in place — teleop collection always appends.

For eval setup and scoring, see the Configuration page and the EVAL workflow.