Instructions to use sii-rhos-ai/ViFailback-8B with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Libraries

How to use sii-rhos-ai/ViFailback-8B with Transformers:

# Use a pipeline as a high-level helper
from transformers import pipeline

pipe = pipeline("image-text-to-text", model="sii-rhos-ai/ViFailback-8B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
pipe(text=messages)

# Load model directly
from transformers import AutoProcessor, AutoModelForImageTextToText

processor = AutoProcessor.from_pretrained("sii-rhos-ai/ViFailback-8B")
model = AutoModelForImageTextToText.from_pretrained("sii-rhos-ai/ViFailback-8B")
messages = [
    {
        "role": "user",
        "content": [
            {"type": "image", "url": "https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/p-blog/candy.JPG"},
            {"type": "text", "text": "What animal is on the candy?"}
        ]
    },
]
inputs = processor.apply_chat_template(
	messages,
	add_generation_prompt=True,
	tokenize=True,
	return_dict=True,
	return_tensors="pt",
).to(model.device)

outputs = model.generate(**inputs, max_new_tokens=40)
print(processor.decode(outputs[0][inputs["input_ids"].shape[-1]:]))

Notebooks
Google Colab
Kaggle
Local Apps

vLLM

How to use sii-rhos-ai/ViFailback-8B with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "sii-rhos-ai/ViFailback-8B"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sii-rhos-ai/ViFailback-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/sii-rhos-ai/ViFailback-8B

SGLang

How to use sii-rhos-ai/ViFailback-8B with SGLang:

Install from pip and serve model

# Install SGLang from pip:
pip install sglang
# Start the SGLang server:
python3 -m sglang.launch_server \
    --model-path "sii-rhos-ai/ViFailback-8B" \
    --host 0.0.0.0 \
    --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sii-rhos-ai/ViFailback-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker images

docker run --gpus all \
    --shm-size 32g \
    -p 30000:30000 \
    -v ~/.cache/huggingface:/root/.cache/huggingface \
    --env "HF_TOKEN=<secret>" \
    --ipc=host \
    lmsysorg/sglang:latest \
    python3 -m sglang.launch_server \
        --model-path "sii-rhos-ai/ViFailback-8B" \
        --host 0.0.0.0 \
        --port 30000
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:30000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "sii-rhos-ai/ViFailback-8B",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Docker Model Runner
How to use sii-rhos-ai/ViFailback-8B with Docker Model Runner:
```
docker model run hf.co/sii-rhos-ai/ViFailback-8B
```

ViFailback-8B

ViFailback-8B is a Vision-Language Model (VLM) designed to diagnose robotic manipulation failures and provide both textual and visual correction guidance. It is fine-tuned from Qwen3-VL-8B-Instruct as part of the ViFailback framework.

The model utilizes explicit visual symbols (arrows, crosshairs, state icons) to bridge the gap between failure diagnosis and policy correction, allowing robotic systems to learn from and recover from real-world failures.

Paper: Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols
Project Page: https://x1nyuzhou.github.io/vifailback.github.io/
Repository: https://github.com/x1nyuzhou/ViFailback

Usage

To run inference and render the corrective visual symbols, use the vifailback_infer.py script provided in the official GitHub repository:

python vifailback_infer.py \
    --model_path sii-rhos-ai/ViFailback-8B \
    --json_path ./examples/example_vifailback_infer.json \
    --dataset_root /path/to/ViFailback-Dataset \
    --output_dir ./inference_visualizations

Citation

@article{zeng2025diagnose,
  title={Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols},
  author={Zeng, Xianchao and Zhou, Xinyu and Li, Youcheng and Shi, Jiayou and Li, Tianle and Chen, Liangming and Ren, Lei and Li, Yong-Lu},
  journal={arXiv preprint arXiv:2512.02787},
  year={2025}
}

Downloads last month: 6

Safetensors

Model size

9B params

Tensor type

BF16

Collection including sii-rhos-ai/ViFailback-8B

ViFailback

Collection

Official collection of the CVPR 2026 paper "Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols". • 2 items • Updated Mar 17 • 1

Paper for sii-rhos-ai/ViFailback-8B

Diagnose, Correct, and Learn from Manipulation Failures via Visual Symbols

Paper • 2512.02787 • Published Dec 2, 2025