FFY0 / VisionWeave

### Weaving Elastic Visual Representations as a Native Capability of MLLMs
**Contents:** Overview · Results · Usage · Agent Setup · Citation
## Overview
VisionWeave learns **where to preserve visual detail and where to compress**, weaving fine- and coarse-grained representations according to visual content. 
Watch a quick walkthrough video.
https://drive.google.com/file/d/1AFFi4HKOdg-0gPdd8-i2s1lzJJh23Uyp/view 
 Fine detail where it matters, compact representations elsewhere. Figure 1 from the paper. 
## Performance
### Content-adaptive token savings
 43.0% visual token savings · 98.9% native performance retained · Qwen3.8-27B averages across eight benchmarks 
### Across tasks and input resolutions
 Consistently favorable efficiency–quality trade-offs across tasks and input resolutions. 
### SGLang serving efficiency
Qwen3.8-27B on LongVideoBench, 2 × A100-80GB (BF16, TP=2).
| Metric | Native | VisionWeave | Improvement |
| --- | ---: | ---: | ---: |
| Throughput (requests/min) ↑ | 0.90 | **2.07** | **2.30×** |
| Mean TTFT (s) ↓ | 169.90 | **77.54** | **54.4% lower** |
| P95 TTFT (s) ↓ | 281.41 | **120.59** | **57.1% lower** |
| Mean TPOT (ms) ↓ | 355.64 | **140.29** | **60.6% lower** |
| P95 TPOT (ms) ↓ | 715.58 | **289.19** | **59.6% lower** |
## Usage
This release provides image/video inference for **dense Qwen3.5**. Trained checkpoints are supplied separately.
 [code] 
### Agent-assisted setup (Strong Recommendation)
Open this checkout in your coding agent and use the bundled VisionWeave skill:
 [code] 
For end-to-end inference, also provide your local checkpoint path and image/video inputs and ask the agent to verify both responses.
### Manual quick start
Install the project environment using the reference configuration:
 [code] 
If starting from **native Qwen3.5 weights**, initialize the VisionWeave router/pooler first. Skip this step if your checkpoint already contains those modules:
 [code] 
The added modules are untrained and default to retaining all visual tokens. The initializer writes the VisionWeave model configuration, so you can serve its output directly. 
Launch on two GPUs. The default attention backend is **FlashAttention-3 (`fa3`)**; select a supported alternative with `--attention-backend` if needed.
 [code] 
After `LANGUAGE SERVICE READY`, check the service from another terminal:
 [code] 
Run the image/video request examples. Detailed setup and troubleshooting and development/baseline instructions live alongside the skill and can also be followed manually.
## Model Weights:
Due to internal policies, model weights are currently unavailable but will be released pending future compliance approval. In the meantime, we are also collaborating with the open-source community toward an independently reproducible version of VisionWeave and will share updates as this effort progresses. Stay tuned.
## Citation
If VisionWeave is useful for your research, please cite our paper.
 BibTeX 
 [code]