Inference strategies

A strategy turns the policy's predicted action chunks into the per-step command stream the robot consumes. EVA ships five — sync / async / naive / act / rtc — selectable from a config file or live in the console.

Pipeline

End-to-end: observation → policy returns a chunk → strategy merges it into the action buffer → one command popped per tick → robot. Only the merge step differs across strategies. Live switching happens in DEBUG (see DEBUG Tab).

The five strategies

Key	Behavior	Choose when
`sync`	Blocking. Fetch chunk, execute its first few steps, fetch again. No background thread, no smoothing; the robot pauses on each prediction.	Simplest, most predictable behavior; brief pauses acceptable.
`async`	Background prediction; linear crossfade over the overlap between old and new chunks. Default.	Continuous smooth motion — the usual choice.
`naive`	Background prediction; each new chunk fully replaces the buffer. Trajectory jumps to the newest prediction.	Freshest prediction immediately; small jumps tolerable.
`act`	Background prediction; ACT-style temporal ensembling averages overlapping chunks per step.	Smoothest, most averaged motion; noisy model output.
`rtc`	Background prediction with Real-Time Chunking: each new chunk is aligned to the action already in flight for seamless handover at higher prediction rates.	Fast policy server; latency-aware handovers required.

act and naive are independent strategies, not sub-options of async. Switching swaps the whole merge behavior.

The background prediction loop

Only sync blocks on the policy. The other four run prediction continuously in the background: a worker requests a fresh chunk at the inference rate (default 3.0 Hz), trims it to the first exec steps, and merges it into the buffer. The control loop pops one ready command per tick and never blocks. Rounds without a new observation are skipped; the worker self-paces to the configured rate rather than running flat-out.

Why two rates? The policy produces a few chunks per second; the robot needs commands at tens of Hz. The buffer bridges that gap.

Merge behavior

Merge style	Used by	Effect
Crossfade (linear overlap)	`async`, `rtc`	Command blends linearly from old to new across the overlap. Smooth, no visible jump.
Replace	`naive`	Queue is discarded and refilled with the new chunk. Instant reaction; possible step at the handover.
Average (temporal ensembling)	`act`	Several recent overlapping chunks are kept; each upcoming step is a weighted average of their predictions for that step. Very smooth, less reactive.

All three first drop already-stale steps off the front of the incoming chunk: by the time a prediction returns, the robot has advanced. That front-trim is the latency k knob.

Diagram needed. Crossfade / replace / averaging would read more clearly with a trajectory-vs-time figure.

Tunable parameters

Each strategy exposes a few knobs as live inputs in the DEBUG tuning panel; type a value and press APPLY to take effect immediately. The same panel adjusts the global prediction rate.

Knob (console label)	Applies to	Meaning · range · default
`execute_horizon` (`exec steps`)	all	Leading steps of each chunk to keep before fetching the next. Integer ≥ 1. Unset ⇒ full chunk; `sync` preset ships `5`. Smaller ⇒ reacts sooner; larger ⇒ relies longer on each chunk.
`inference_rate`	async / naive / act / rtc	Background fetch rate, Hz. Default `3.0`. Raise only if the policy server keeps up.
`latency_k` (`latency k`)	async / naive / rtc	Max overdue steps trimmed off the front of a new chunk. Integer ≥ 0. `naive` preset ships `4`; base default is no trim. Larger ⇒ skips further ahead; too large ⇒ skips real motion.
`exp_weight_m`	act	Decay rate of older predictions in the average. Number ≥ 0, default `0.01`. Larger ⇒ oldest surviving prediction dominates; smaller ⇒ flatter average.

latency_k is shared across async, naive, and rtc. It is not tied to one merge style.

APPLY updates take effect on the next prediction round (sub-second at the default rate). Rejected values snap back or surface a panel error — check the range. If the robot stops moving, the policy server is the usual cause; check the DEBUG log.

Selecting and switching strategies

Live: the DEBUG strategy picker switches among the five keys at runtime. See DEBUG Tab.

Config: defaults live in shared config; your deploy config overrides only the knobs you change. Start from configs/01_deploy/<robot>/ (e.g. openpi_qpos.py) and add an inference_strategies block:

python

# Override only async/rtc latency_k; everything else stays at the shared default.
inference_strategies = {
    'async': dict(args=dict(
        latency_k=<steps>,   # integer >= 0
    )),
    'rtc': dict(args=dict(
        latency_k=<steps>,
    )),
}

Other knobs slot into the same args — execute_horizon=<steps> on any key, exp_weight_m=<number> on act. Full field list per key on the Configuration page.

Evaluation runs pin one strategy via eval.inference_strategy='async' (any of the five keys) in the eval config; see Configuration.

RTC alignment lives on the policy backend, not the strategy. The rtc strategy only schedules the background loop; the actual alignment runs in the openpi_rtc policy type. See Backends.

How to add a new inference strategy

Strategies live under src/strategy/. Subclass BackgroundLoopInferStrategy from strategy.base_strategy for any fixed-Hz async variant (the shared loop, chunk crop, and buffer plumbing are already there); subclass BaseInferStrategy instead for a blocking sync mode. The dataclass fields ARE the YAML args keys.

Add src/strategy/<your>_strategy.py. Subclass BackgroundLoopInferStrategy (or BaseInferStrategy), declare any extra params as dataclass fields, and override _make_buffer() if you need a buffer other than StreamActionBuffer.
Decorate the class with @STRATEGY_REGISTRY.register("YourInferStrategy") imported from core.registry. The registered name is what goes into type= in the config.
No registration boilerplate elsewhere — src/strategy/__init__.py auto-imports every submodule of the package on launch, so dropping the file in is enough.
Wire it into a deploy config under inference_strategies: add an entry like "your_key": dict(type="YourInferStrategy", args=dict(...)). The short key (your_key) is what the DEBUG strategy picker and eval.inference_strategy use.

python

# src/strategy/your_strategy.py
from __future__ import annotations
import dataclasses
from typing import ClassVar

from core.registry import STRATEGY_REGISTRY
from strategy.base_strategy import BackgroundLoopInferStrategy


@STRATEGY_REGISTRY.register("YourInferStrategy")   # ← name used in config type=
@dataclasses.dataclass
class YourInferStrategy(BackgroundLoopInferStrategy):
    """One-line description of your merge behavior."""

    your_knob: float = 0.5                       # ← becomes a YAML args key

    _thread_name: str = dataclasses.field(default="eva-your-loop", init=False)
    _log_label: str = dataclasses.field(default="your", init=False)

    # ← console tuning panel renders these as live inputs
    tune_fields: ClassVar[list[dict]] = [
        {"key": "execute_horizon", "label": "exec steps", "min": 1, "step": 1},
        {"key": "latency_k",       "label": "latency k",  "min": 0, "step": 1},
        {"key": "your_knob",       "label": "your knob",  "min": 0, "step": 0.01},
    ]

    def __post_init__(self) -> None:
        self.your_knob = max(0.0, float(self.your_knob))
        super().__post_init__()                  # ← clamps inference_rate / latency_k, builds buffer

    # Override _make_buffer() to swap StreamActionBuffer for a custom merge.
    # Override _inference_loop() only if the default fixed-Hz fetch isn't enough.

For a sync-only variant, subclass BaseInferStrategy and override take_or_fetch_chunk instead — leave start_loop/pop_next_action as the inherited no-ops. The full set of in-tree examples — async_strategy.py, naive_strategy.py, act_strategy.py, rtc_strategy.py — sits beside base_strategy.py; the merge semantics live in the buffer classes in strategy/action_buffer.py.