RESULT

Overview

RESULT appears whenever the launched config contains an eval section. It replays what EVAL recorded; with nothing recorded the tree is empty.

eva --config configs/03_evaluation/ur5e_eval.py

The page splits into two panes:

• Tree (left) — three-level browser, model → task → trial. Each model and task row shows two ring gauges (score%, success%); expanding a task lays out its attempt cells.
• Detail / replay (right) — selected attempt's cameras, 3D motion, and charts.

Where the data comes from

There is no single results file. Each model's attempts are recorded into that model's own dataset folder under the run's output directory; rows load directly from those datasets.

Each recorded attempt carries the fields RESULT displays:

Re-testing the same attempt keeps only the latest recording.

Model · task · trial tree

Collapsible three-level tree: model, task, attempt. One model and one task expand at a time. Each model and task row shows two ring gauges:

• score% — total score earned divided by total possible score across that row's scored attempts.
• success% — fraction of scored attempts earning full marks. Tasks with no milestones (max score zero) are excluded.

A gauge reads — with nothing scored, and turns red on a zero. Expanding a task reveals its attempt grid, one cell per attempt slot.

Attempt grid

One cell per attempt:

• Scored cells fill in proportion to score and show score/max. Zero is flagged distinctly.
• Unscored attempts are dimmed and labelled trial N; not clickable.
• Click a scored cell to open its replay in the right pane.

If two recordings land on the same cell, the scored one wins, so a later unscored re-run cannot hide a score. Task labels show prompt_zh when set, otherwise prompt_en.

Selection

The right pane starts empty ("Expand a model and task, then select a trial to replay."). Clicking a scored cell loads its replay; the selected cell stays highlighted across tree redraws.

score% and success% roll up identically at every level: a task's gauges sum its scored cells; a model's gauges sum every scored cell across its tasks. A zero-scored cell still counts.

Replay

The replay shows all panels at once — no tabs. The header shows prompt, attempt number, score, and the task's read-only milestone steps. All panels share one timeline:

The scrub bar and play button drive one cursor across all panels, keeping cameras, 3D, and charts in lockstep. Playback paces by the recording's own frame timing.

Configuration

RESULT has no settings of its own — it shows whatever the eval run recorded. The same eval_cfg block that drives EVAL fills RESULT:

• checkpoints → model rows.
• tasks → per-model task rows and their attempt grids.
• trials_per_prompt → attempt cells per task.
• milestones per task → max score and the read-only step list above the replay panels.

An eval preset is a deploy config to inherit from plus a filled eval_cfg. Full field semantics: Configuration → Eval configs. Replace <…> values below.

# Eval preset: a deploy config to inherit + a filled `eval_cfg` section.
_base_ = ['<path to your deploy config, e.g. ../01_deploy/ur5e/openpi_qpos.py>']

eval_cfg = dict(
    storage=dict(fps=20, save_queue_max=15),  # recording frame rate; sets replay pacing.
    trials_per_prompt=5,                      # attempts per task = grid width.
    cli_mode='real',                          # real vs sim; see Eval configs.
    inference_strategy='async',               # action timing; see Inference strategies.
    checkpoints=[                              # one model row per entry.
        dict(
            name='<your model name>',       # names the on-disk folder and the tree row.
            config='<path to that model's deploy config>',
            port=9000,                          # policy server port; one per model.
        ),
    ],
    tasks=[                                    # one task row per entry.
        dict(
            prompt_en='<instruction>',
            milestones=(                       # (key, description) pairs; count = max score.
                ('<key>', '<what this step achieves>'),
                ('<key>', '<what this step achieves>'),
            ),
        ),
    ],
)

Full eval field reference — cli_mode, inference_strategy, reset_after_each_trial, shuffle_ckpts, the ssh block, optional prompt_zh and init_pose — see Eval configs.