Overview video
A short walkthrough of the control-centric objective, unified architecture, and updated evaluation evidence.
Overview
Instead of reconstructing an observation-complete video future, LaST1.0 supervises a compact latent representation of the evolving robot-object interaction from future wrist views.
A lightweight patch embedder projects multi-view images directly into the shared backbone, preserving fine-grained visual cues needed for contact-rich manipulation.
LaST-Attention applies 4D-RoPE across sequence, height, width, and relative time to enable structured spatiotemporal fusion of multimodal tokens.
Method
Multi-view image patches, language, and timestep information enter one decoder-only backbone. LaST-Attention organizes the multimodal sequence in four coordinates, and task-routed experts separate context, latent, and action computation.
A two-layer convolutional projector preserves local visual evidence without a deep vision encoder.
Frozen VGGT-Ω produces geometry-aware interaction-centric latent targets from future wrist-view observations.
Sequence, height, width, and relative-time subspaces preserve the structure of multimodal tokens.
Context, latent, and action tokens use dedicated experts while action-only deployment stays efficient.
Training
Two flow-matching branches shape a shared, physically informed context representation.
Deployment
The latent pathway and VGGT-Ω teacher are omitted for efficient closed-loop control.
Simulation results
All methods are trained in the multi-task setting, and we report average success rates (%).
| Models | Close box | Close laptop lid | Toilet seat down | Sweep to dustpan | Close fridge | Phone on base | Take umbrella | Take frame off hanger | Place wine at rack | Water plants | Average |
|---|---|---|---|---|---|---|---|---|---|---|---|
| OpenVLA-OFT | 96.0 | 70.0 | 80.0 | 50.0 | 84.0 | 38.0 | 44.0 | 20.0 | 38.0 | 26.0 | 54.6 |
| π0.5 | 90.0 | 96.0 | 94.0 | 84.0 | 100.0 | 18.0 | 28.0 | 86.0 | 76.0 | 38.0 | 71.0 |
| HybridVLA | 92.0 | 96.0 | 100.0 | 90.0 | 96.0 | 44.0 | 58.0 | 68.0 | 48.0 | 44.0 | 73.6 |
| Fast-WAM | 94.0 | 90.0 | 96.0 | 74.0 | 100.0 | 30.0 | 66.0 | 56.0 | 8.0 | 14.0 | 62.8 |
| Cosmos Policy | 94.0 | 94.0 | 100.0 | 60.0 | 76.0 | 78.0 | 80.0 | 70.0 | 54.0 | 58.0 | 76.4 |
| LaST0 | 96.0 | 92.0 | 100.0 | 82.0 | 86.0 | 74.0 | 78.0 | 66.0 | 80.0 | 64.0 | 81.8 |
| LaST1.0 (Ours) | 100.0 | 100.0 | 100.0 | 96.0 | 100.0 | 86.0 | 78.0 | 66.0 | 98.0 | 96.0 | 92.0 |
Success Rate (%) across four LIBERO suites and seven zero-shot LIBERO-Plus perturbations. Bold marks the best result in each column.
| Method | LIBERO | LIBERO-Plus · Zero-shot | |||||||||||
|---|---|---|---|---|---|---|---|---|---|---|---|---|---|
| Spatial | Object | Goal | Long | Average | Camera | Robot | Language | Light | Background | Noise | Layout | Average | |
| OpenVLA-OFT | 97.6 | 98.4 | 97.9 | 94.5 | 97.1 | 56.4 | 31.9 | 79.5 | 88.7 | 97.3 | 75.8 | 74.2 | 70.0 |
| π0.5 | 98.8 | 98.2 | 98.0 | 92.4 | 96.9 | 78.4 | 73.6 | 80.8 | 96.2 | 94.1 | 89.0 | 84.5 | 84.4 |
| StarVLA-α | 99.0 | 99.8 | 98.5 | 94.1 | 97.9 | 48.7 | 63.4 | 86.8 | 95.8 | 94.6 | 75.0 | 80.2 | 76.1 |
| Fast-WAM | 98.2 | 100.0 | 97.0 | 95.2 | 97.6 | 16.4 | 44.5 | 68.9 | 78.2 | 53.7 | 37.7 | 60.7 | 50.0 |
| Cosmos Policy | 98.1 | 100.0 | 98.2 | 97.6 | 98.5 | 75.8 | 63.3 | 81.7 | 96.5 | 88.9 | 92.7 | 82.2 | 82.2 |
| LingBot-VA | 98.5 | 99.6 | 97.2 | 98.5 | 98.5 | 40.9 | 83.0 | 86.4 | 82.3 | 53.1 | 64.4 | 76.2 | 69.5 |
| VLA-JEPA | 96.2 | 99.6 | 97.2 | 95.8 | 97.2 | 63.3 | 67.1 | 85.4 | 95.6 | 93.6 | 66.3 | 85.1 | 78.0 |
| LaST0 | 99.2 | 99.6 | 98.0 | 95.6 | 98.1 | 81.4 | 36.5 | 80.2 | 86.4 | 85.7 | 79.8 | 68.9 | 73.2 |
| LaST1.0 (Ours) | 98.5 | 99.8 | 99.1 | 98.5 | 99.0 | 77.9 | 68.9 | 89.4 | 94.7 | 93.1 | 97.3 | 87.3 | 86.3 |
LIBERO results are averaged across three evaluation runs. LIBERO-Plus directly evaluates weights trained on LIBERO without additional fine-tuning.
LaST1.0 reaches a 99.0% LIBERO average and maintains 98.5% on the Long suite, matching the best long-horizon result in the comparison.
Without LIBERO-Plus fine-tuning, LaST1.0 obtains the best overall average of 86.3%, including 97.3% under sensor noise.
The latent branch serves as training-time supervision and is omitted during deployment. The manuscript reports 54.7 ms action inference on an RTX 4090.
Visual representation
On RLBench, LaST1.0's encoder-free visual interface reaches 92.0% average success. A SigLIP2 encoder reaches 84.8%, while a reconstruction-oriented VAE reaches 87.4%.
Real-world manipulation
69.3%average stage success
57.0%average task success
54.7 msaction inference on RTX 4090
Videos
All five demonstrations are shown side by side for direct comparison of the long-horizon tasks.
Dual-arm Franka · Grippers
Dual-arm Franka · Grippers
Dual-arm Tianji · Grippers
Dual-arm Tianji · Grippers
Dual-arm Tianji · Sharpa Hands
Videos
Train with a facial cleanser in the original setup; test without fine-tuning under object, background, and target-position shifts.
Object: replace the facial cleanser with a banana of different shape and texture. Background: scatter distractors across the workspace. Position: move the target outside the training distribution.
bag_object
bag_background
bag_position
Abstract
World-Action Models couple action learning with future prediction, but reconstruction-oriented video modeling allocates capacity to appearance details incidental to control. LaST1.0 instead predicts compact, geometry-aware future latents from short-horizon future wrist views, concentrating supervision on evolving robot-object interactions.
An encoder-free visual interface preserves patch-level evidence, while LaST-Attention with 4D-RoPE structures multimodal fusion across sequence, height, width, and relative time. A task-routed mixture of experts separates context, latent, and action computation. The latent branch provides training-time regularization and is omitted for efficient action-only deployment.