MCG-NJU / VideoJev

System One Model for Video Understanding and Interaction 
## Overview
**VideoJev** performs temporal grounding and multiple-choice video QA through direct candidate scoring, without text generation.
Try VideoJev-4B on ModelScope Studio ↗
https://github.com/user-attachments/assets/b8acb023-26d5-4940-bde1-2d623bc019fe
 Architecture 
## Results
 Rankings as of 2026-10-17. 
### Temporal grounding
### Video question answering
## Models
| Model | Base model | Weight format | Usage |
|---|---|---|---|
| **VideoJev-4B** | Qwen3.5-4B | BF16 · Safetensors | Checkpoint & examples ↗ · ModelScope ↗ |
| **VideoJev-35B-A3B** | Qwen3.6-35B-A3B | BF16 · Safetensors | Checkpoint & examples ↗ · ModelScope ↗ |
## Quick start
Set up the environment, then run the 10-second demo:
 [code] 
Use `ZhuYuhan/VideoJev-35B-A3B` for the larger model.
## Inference output
Recorded 4B output:
 [code] 
`interval` is in seconds; `answer_index` is zero-based. Probabilities use sigmoid for grounding and softmax for QA.
Grounding defaults to `threshold=0.25`. Pass `threshold=...` to `predict_video` or `predict_features` to override it.
## Environment
Python 3.12, PyTorch 2.8 / CUDA 12.8, Triton 3.4, FlashAttention 2 and causal-conv1d. Install the matching CUDA packages first:
 [code] 
Training requires XTuner; 35B also requires its compiled grouped-GEMM/token-permutation extension. Set `XTUNER_ROOT` before sourcing the environment, or install XTuner in it.
Put the base models in `models/Qwen3.5-4B` and `models/Qwen3.6-35B-A3B`, or set `VIDEOJEV_BASE_4B` and `VIDEOJEV_BASE_35B`.
## Data
Prepare your JSONL data; the command generates ready-to-train configs for either model.
## Training
| Stage | Trainable parameters | Grounding loss | QA loss |
|---|---|---|---|
| Head warmup | Head | BCE | CE |
| Joint | Language backbone + head | BCE + 2 × duration-weighted SoftIoU | CE |
Vision stays frozen. Each stage runs for one epoch with a global batch of 64 packs.
### Launch
Use the configs generated by data preparation:
 [code] 
For 35B-A3B, use `scripts/train_qwen36.py` with its generated configs and `scripts/env_qwen36.sh`. See Data & training for preparation and GPU settings.
### Checkpoints
4B exports **`joint/hf_checkpoint/`** in Hugging Face format, preserving the base model's dtype (BF16). See Checkpoint export for recovery files and standalone 35B conversion.
## Evaluation
 [code] 
The same command supports `ZhuYuhan/VideoJev-35B-A3B`. QA uses a fixed 256K context. See Evaluation for benchmarks and distributed runs.
## Verification
 [code] 
Set `VIDEOJEV_TEST_MODEL=/path/to/Qwen3.5-4B` for tokenizer checks. Tests skip when required local inputs are absent.
## Documentation
Paper · Data · Evaluation · Checkpoint export
## Citation
 [code] 
## Acknowledgements
Thanks to the teams behind **Qwen** for the open model checkpoints, **XTuner** for the training framework, and **TimeLens-100K** and **TimeLens2-93K** for the open training data.
 Apache 2.0 · NOTICE