Representation Steeringfor Vision-Language-Action Policies

Anonymous supplementary material
OpenVLA-OFT · LIBERO-10

Redirect a robot by steering its internal representation. A probe reads what an action will do; the action head turns that insight into a direction.

Adapting to a change in intent

The policy is given the original instruction:

ℓ = “put the white mug on the left plate and put the yellow-and-white mug on the right plate”

Under this instruction, the policy starts with the white mug. Now the intent changes: move only the yellow-and-white mug to the right plate, leaving the white mug in place. We can (1) change the instruction, or (2) keep the original instruction ℓ and steer the policy's representation.

No additional fine-tuning, training, or data collection.

Using the already-fitted probe and the model's own action head, our method adapts to the new intent at inference time while keeping the policy weights and the original instruction unchanged.

Instruction changes and representation steering

The goal is to move only the yellow-and-white mug to the right plate, leaving the white mug in place. Compare the original instruction, two rewritten instructions, and steering with the original instruction.

Effect of steering strength

Same instruction and target, with α below 56. Rather than committing, the robot can switch between objects: it heads for one mug, turns to another, and sometimes knocks one over. Each example shows one failed run at each weaker strength, from different starts.

On the held-out starts, changing the instruction succeeds in –. Steering with the original instruction succeeds in –.

Success on held-out starts

Success: the yellow-and-white mug rests upright on the right plate, the white mug is never lifted and moves less than 3 cm, within 15 s. Fixed before the test run.

Success by steering strength

Outcome by steering strength (α = 16 moves the head input by 8% of its norm per re-plan). Weak pushes leave the learned habit in place and intermediate ones leave the policy undecided; at α = 56, steering redirects it reliably, and pushing further, to α = 80, degrades performance.

How this was set up, and its limits
  • What we choose: the target quantity (distance of the end effector to a point above the yellow-and-white mug, at the height the demonstrations reach before grasping it), the strength α and the release rule (stop steering within 5 cm of that point, once the mug is lifted, or after 4.8 s). The direction itself comes from the probe and the model's own action head, recomputed at every re-plan; no simulator query, no new data, no training.
  • Two runs. First: the height margin chosen on 12 development starts, several strengths tested on 13 other held-out starts. Confirmation: the strongest one (α = 56) fixed in advance, probes refit on the other half of the episodes, and 25 new held-out starts, each run once; the weaker strengths were run on the same starts afterwards. Everything on this page is from the confirmation run.
  • The strengths used here are larger than the range in which the paper validates local steering (see the paper).
  • After placing the yellow-and-white mug the policy stops: in its demonstrations, that scene is the end of the task. Moving the white mug next would need further steering.
  • One model, one task. A policy that follows language more closely may respond to the new instruction.
  • Baselines: three fixed steering vectors (CAA with an outcome contrast, CAA with a behaviour contrast, and the weight of a linear probe), built from the fitting episodes and applied with the same intervention site, strength unit, re-plan schedule and release rule. Each method, ours included, runs at six strengths and is shown at its best.

Method

A probe reads, from the representation, what any candidate action chunk would do to a physical quantity ψ, here the distance of the end effector to a point above the yellow-and-white mug. Pulling that read-out back through the action head gives a direction in the representation; pushing along it steers the policy. No simulator query, no new data, no training.

Read, pull back, and push: the probe predicts a candidate action chunk’s outcome from the representation and its signature. The probe-derived direction is pulled back through the action head and used to steer the representation, then released near the target. Only the small head is differentiated, never the language model or simulator.
Steering parameters

The quantity ψ, the strength α and the release rule. The direction itself comes from the probe and the model's own action head.

Validation scope

Small pushes bias the robot towards the chosen quantity. Changing which object it goes for needs the larger, released pushes of the use case, beyond that validated range.

Experimental setting

One model (OpenVLA-OFT), one LIBERO-10 task, the 32 leading principal directions of the action-head input.

Code

The experiments run as individual Colab cells. Outputs are saved in aclrh/ on Drive and read by subsequent steps. See README.md for the run order and PAPER_MAP.md for the scripts associated with each table and figure.

Running the code

Unzip code.zip in the Colab working directory. For example:

# once per runtime: unzip code.zip in the working directory
run = lambda p: exec(open(f"vla-lrh/{p}").read(),
                     {"__name__": "__main__"})

run("openvla_oft/02_collect_probe_data.py")  # GPU
run("probing/q1_probe_extended.py")          # CPU

GPU steps load the model or simulator; CPU steps analyse saved results. Use an A100 GPU, or an L4 for steps 16, 18 and 18b. Environment setup takes 15–25 minutes in the first GPU cell. Long runs save checkpoints to Drive and resume when the cell is rerun.

Experiment scripts and run order

OpenVLA-OFT

StepOutputRuntime
01 · 02Collect404 states × 32 candidate chunksGPU
03 · 05aProbeprobing tables (OFT)CPU
04 · 05b · 05cSteer one chunksteering table, 140 held-out statesGPU
06 → 07 · 08 · 09Closed loop, figuresresponse curves, closed-loop pathsGPU / CPU
11Steer with MLP probessame protocol, directions from nonlinear probesGPU

SmolVLA

StepOutputRuntime
s1Collect16 policy samples + 16 perturbations per stateGPU
q1Probe both modelsQ1 table (needs 02 too)CPU
q1cProbe capacityMLP widths 8–256 against the linear probeCPU or GPU
s2Steerfresh samples shared across strengthsGPU
s3 · s4Analysesteering tables, reversal diagnosisCPU
s5 → s6 · s7 · s8Diagnose failuresbimodal states, discontinuity and contact eventsGPU / CPU

Use case

StepOutputRuntime
16Instruction vs. steeringα chosen on 12 starts, swept on 13GPU
18Confirmα fixed, new probes, 25 new startsGPU
18bBaselinesCAA and probe-weight vectors and ours at six strengths, a third instruction, comparison videosGPU
19 = 17 + 12Figure 2 and this pagefig_usecase, siteCPU
20Add 18b to this pagebaseline card, comparison videosCPU
Repository structure

Paths are relative to vla-lrh/.

DirectoryStepsContents
openvla_oft/01–11Probing, steering, and closed-loop experiments
probing/q1, q1cQ1 probing table and probe-capacity sweep, both models
smolvla/s1–s8Stochastic policy experiments and failure analysis
use_case/16–18bFigure 2, the use case and its baselines
site/12, 19, 20Project page
colab/—One cell per step
extras/—Runs not reported in the paper
tools/—Anonymity check
Paper results and scripts
Q1 probing table, both modelsq1
Probe-capacity figure and appendix tableq1c
OpenVLA-OFT steering table04 · 05b · 05c
Response curves08
Closed-loop figure06 → 07 · 09
SmolVLA steering tabless2 → s3 · s4
SmolVLA failure analysiss5 → s6 · s7 · s8
Figure 2 and the use case above16 · 18 → 17
Steering baselines in the use case18b → 20

The complete mapping, using the labels in the paper, is in PAPER_MAP.md.

Experimental settings
  • OpenVLA-OFT fine-tuned on LIBERO-10, chunks of 8 actions; one LIBERO-10 task, all 50 layouts.
  • z(x): the 32 leading principal directions of the action-head input, fitted on training episodes only.
  • Held-out episodes everywhere: folds by episode (even / odd for steering), seed 0.
  • One unit of α moves the head input by 0.5% of its norm; finite-difference step 0.25 (standardized units).
  • Use-case success: the yellow-and-white mug upright on the right plate, the white mug never lifted and moved less than 3 cm, within 15 s; fixed before the runs.
  • Steering baselines: CAA and probe-weight vectors built from the fitting fold only, at the same site and strength unit; each run at α ∈ {16, 24, 32, 40, 56, 80} on the same 25 starts as ours.