StarVLA/Qwen3-VL-OFT-RoboTwin2-All

Qwen3-VL OFT baseline for RoboTwin 2.0

This repository provides a baseline policy for RoboTwin 2.0 trained using a minimal visual-to-action configuration.

The goal of this release is to provide a simple and reproducible baseline for RoboTwin, using:

• RGB observations only
• direct absolute joint prediction
• no robot state input
• no action transformer
• no pretraining

Despite its simplicity, the model achieves strong performance across RoboTwin 2.0 tasks.

---

Model and action contract

Component	Setting
VLM	Qwen3-VL-4B
Action head	Two-block residual MLP, 2,560 input / 5,120 hidden / 14 output; direct L1 regression

The policy directly maps RGB observations (head + two wrist cameras) to 14D absolute joint positions. The saved config uses a 50-step action horizon and does not enable robot-state input.

Item	Value
StarVLA framework	QwenOFT
Base VLM	Qwen3-VL-4B-Instruct
Action/state dimensions	14 / no state input recorded
Action mode	Absolute joint position
Action horizon	50
Input resolution	224 × 224
Released checkpoint	checkpoints/steps_140000_pytorch_model.pt
Checkpoint format	Complete StarVLA framework state dict (.pt)

Training recipe

Setting	Value
Dataset mixture	robotwin_all_50
Configured maximum steps	150,000
Released step	140,000
Per-device batch size	4
Gradient accumulation	1
Warm-up	5,000 steps
Base / VLM-interface / action LR	1e-5 / 1e-5 / 1e-4
freeze_modules	Packaged boolean true; the public trainer expects module paths as a string, so this value names/selects no modules
Seed	42

---

Training dataset

The model is trained using the official RoboTwin 2.0 dataset.

The training data consists of demonstrations from 50 manipulation tasks, combining clean demonstrations and randomized demonstrations.

Clean Demonstrations

• 50 tasks × 50 trajectories per task

Randomized Demonstrations

• 50 tasks × 500 trajectories per task

The randomized demonstrations include environment perturbations to improve robustness.

---

Evaluation protocol

The model is evaluated on RoboTwin 2.0 using 50 manipulation tasks.

Each task is evaluated under two evaluation settings:

• Easy setting
• Hard setting

For each task and setting:

• 100 evaluation trials are performed
• The success rate is reported

Final results are computed as the average success rate across tasks.

---

Benchmark results

Task	Easy	Hard
stack_blocks_two	1.0000	1.0000
place_cans_plasticbox	1.0000	0.9900
grab_roller	1.0000	1.0000
place_empty_cup	1.0000	1.0000
shake_bottle_horizontally	1.0000	1.0000
lift_pot	1.0000	1.0000
place_burger_fries	1.0000	1.0000
move_playingcard_away	1.0000	0.9800
adjust_bottle	1.0000	0.9900
shake_bottle	1.0000	1.0000
blocks_ranking_rgb	0.9900	0.9800
stack_bowls_two	0.9900	1.0000
place_container_plate	0.9900	0.9900
press_stapler	0.9900	0.9600
place_object_stand	0.9900	0.9800
open_laptop	0.9800	1.0000
handover_mic	0.9800	0.9600
move_pillbottle_pad	0.9800	1.0000
handover_block	0.9700	0.9300
place_shoe	0.9600	1.0000
stack_bowls_three	0.9500	0.9100
place_fan	0.9400	0.9500
scan_object	0.9400	0.9100
stack_blocks_three	0.9400	0.8600
place_object_basket	0.9300	0.9400
beat_block_hammer	0.9300	0.9200
place_object_scale	0.9300	0.9300
place_dual_shoes	0.9100	0.8900
pick_dual_bottles	0.9100	0.9300
place_bread_basket	0.9100	0.7800
dump_bin_bigbin	0.9100	0.9400
move_can_pot	0.9100	0.9000
put_bottles_dustbin	0.9000	0.8500
place_a2b_left	0.9000	0.9500
place_bread_skillet	0.8900	0.8000
put_object_cabinet	0.8900	0.9100
place_a2b_right	0.8800	0.9500
rotate_qrcode	0.8800	0.9000
pick_diverse_bottles	0.8700	0.8600
place_mouse_pad	0.8700	0.9400
stamp_seal	0.8600	0.9000
place_phone_stand	0.8600	0.9500
blocks_ranking_size	0.7900	0.8000
place_can_basket	0.7500	0.7500
move_stapler_pad	0.7400	0.9000
turn_switch	0.6500	0.6200
click_alarmclock	0.5800	0.5100
hanging_mug	0.3400	0.2900
open_microwave	0.2800	0.3900
click_bell	0.2300	0.2700
Average	0.8818	0.8832

---

---

Download and evaluation

huggingface-cli download StarVLA/Qwen3-VL-OFT-RoboTwin2-All \
  --local-dir Qwen3-VL-OFT-RoboTwin2-All

CKPT="$PWD/Qwen3-VL-OFT-RoboTwin2-All/checkpoints/steps_140000_pytorch_model.pt"
CUDA_VISIBLE_DEVICES=0 python deployment/model_server/server_policy.py \
  --ckpt_path "$CKPT" --port 5694 --use_bf16 \
  --config_override framework.qwenvl.base_vlm=Qwen/Qwen3-VL-4B-Instruct

The RoboTwin evaluator must use the saved absolute-action convention, normalization statistics, camera order, and 50-step prediction contract. See the StarVLA RoboTwin guide.

Evidence and limitations

• Architecture and training settings were checked against config.yaml; the released 140k step was checked against the Hub file tree.
• The benchmark table and 100-trial-per-task protocol are retained from the original Card. This repository does not include raw rollout logs, so the aggregate cannot be independently reconstructed from the Hub files alone.
• Only the 140k weight is available; the 150k configured endpoint is not released here.
• The policy is tied to the saved three-camera, absolute-joint and normalization contract. Different camera ordering, action mapping, chunk execution horizon, or normalization can materially change results.
• Results cover RoboTwin 2.0 simulation only. Real-robot performance and safety are not established.

Citation

If you use this model or benchmark results in your research, please cite RoboTwin and this repository.

@article{community2026starvla,
  title         = {StarVLA: A Lego-like Codebase for Vision-Language-Action Model Developing},
  author        = {Community, StarVLA},
  journal       = {arXiv preprint arXiv:2604.05014},
  year          = {2026},
  eprint        = {2604.05014},
  archivePrefix = {arXiv},
  primaryClass  = {cs.RO}
}