Configuration

Prerequisites

• Installation complete (Python 3.10+, eva on PATH).
• Robot preset folder. Throughout this page <robot> stands for one of: agibot_g2, arx_r5, dual_agilex_AGILEX, dual_franka, r1lite, ur5e.
• For real-robot runs: robot reachable over ROS, and a policy server already listening on a known host/port (see Backends).

Open-loop replay

Open-loop replays a recorded episode through the full pipeline — no robot, no policy server:

eva --config configs/00_openloop/ur5e_openloop.py

Other open-loop files: arx_r5_openloop.py, dual_agilex_AGILEX_openloop.py, dual_franka_openloop.py, r1lite_openloop.py, all under configs/00_openloop/. EVA prints the console URL on success; failures stop with a Python traceback naming the cause (commonly a bad --config path or a missing dataset).

How configs work

• Inheritance — each config begins with _base_ = ['_base.py', ...], listing parents to build on. The child only writes the keys it changes.
• Single defaults file — all defaults live in configs/00_base/defaults.py. There are no hidden fallbacks: a config that omits _base_ is simply missing those settings.

File layout

Inheritance and overrides

EVA merges parent and child setting by setting:

• Re-listing a dict(...) block overrides only the keys you write; the rest are kept from the parent.
• A scalar or whole list replaces the parent's value. Lists are never concatenated.

Chains may be several levels deep, e.g. ur5e_eval.py → ur5e/openpi_qpos.py → 00_base/defaults.py. Fields marked ← below are the ones a user typically edits.

_base_ = ['_base.py']          # inherit this robot's shared deploy settings.

policy = dict(
    type='openpi',        # backend: openpi / openpi_rtc / starvla / gr00t / mock / replay.
    host='<policy-server-ip>',  # ← omit to keep 127.0.0.1.
    port=<policy-server-port>,    # ← policy server port.
)

inference_cfg = dict(
    debug_tasks=['<task prompt>'],  # ← task instruction.
    # Omit obs_space / action_space to keep joint-angle control.
)

Default control is joint-angle (JointState). For end-effector control (EEFPose), override both spaces — set n_arms to the robot's arm count:

inference_cfg = dict(
    obs_space=dict(type='EEFPose', n_arms=<arm count>, rotation='quat', include_gripper=True),    # ← 1 or 2.
    action_space=dict(type='EEFPose', n_arms=<arm count>, rotation='quat', include_gripper=True),  # ← same arm count.
)

Settings reference

robot

transport

Channel EVA uses to read sensors and publish actions. See Transport.

policy

Connection to the policy server. See Backends.

inference_cfg

obs_space and action_space switch between JointState and EEFPose; see Action spaces.

inference_strategies

Five named strategies are shipped and switchable live in the console. Reference the strategy by the name in the left column; see Strategies.

collection · eval_cfg · work_dir

collection configures teleop recording (columns, cameras, arms); presence of this section marks the config as a recording config — see Data collection. eval_cfg configures the checkpoint sweep (next section). work_dir (default "work_dirs") is the output root.

Eval configs

An eval config runs several checkpoints on the same tasks for a multi-checkpoint comparison. It inherits a deploy preset and adds an eval_cfg section listing checkpoints and tasks; EVA loads each checkpoint's deploy config and points it at the right server.

_base_ = ['../01_deploy/<robot>/openpi_qpos.py']   # ← deploy preset for this robot.

eval_cfg = dict(
    trials_per_prompt=5,           # trials per task per model.
    cli_mode='real',             # 'real' or 'sim'.
    inference_strategy='async',    # strategy name from the table above.
    reset_after_each_trial=False,  # True returns to start pose after each trial.
    checkpoints=[
        dict(
            name='<checkpoint name>',              # ← real checkpoint name, shown next to the slot letter.
            config='../01_deploy/<robot>/openpi_qpos.py',  # ← relative to THIS file.
            port=<policy-server-port>,            # ← this model's port.
        ),
        # One dict per model.
    ],
    tasks=[
        dict(prompt_en='<task in English>'),  # ← one entry per task.
    ],
)

• storage — fps stamped on saved frames; save_queue_max caps the write-queue depth.
• skip_warmup_after_first — when True, only trial 1 warms up.
• checkpoints[i]:
  • name — the checkpoint's real name; shown in EVAL as a slot letter (Model A / B / …) plus this name in the picker.
  • config — deploy config path, relative to the eval file. A bad path stops EVA at launch.
  • port — this checkpoint's policy server port.
  • host — optional, defaults to 127.0.0.1.
• shuffle_ckpts / shuffle_seed — randomize the trial order across checkpoints; seed defaults to 42.
• enable_ssh_forward — set True when a checkpoint's server is on a remote host; requires an ssh block:

enable_ssh_forward=True,
ssh=dict(
    host='<remote host>',
    user='<ssh user>',
    port=<remote port>,    # policy server port on the remote.
    # remote_sync_dir='<remote results path>',  # optional: copy results back.
),

• tasks[i] — required prompt_en; optional:
  • milestones — scored sub-goals as (short-key, description) pairs, e.g. (('grasp', 'grasp apple'),).
  • prompt_zh — Chinese prompt.
  • init_pose — per-task starting pose, same format as robot.initial_qpos.

Startup pipeline

What eva --config does, in order:

• Merge parents through the _base_ chain.
• Build control spaces from the joint / EEF choice.
• Fill paths — empty output paths derived from work_dir and the config name.
• Validate recording configs — must define state, EEF, and action columns plus at least one camera and one arm; otherwise startup aborts naming the missing field.
• Resolve eval checkpoints — each eval_cfg.checkpoints[i] config is loaded and pointed at its host/port.