PSRben / VisionHOPE

VisionHOPE 
 Visual Backbones as Self-Modifying Learning Systems 
 Siran Peng * · Tianshuo Zhang * · Tianyu Fu · Weisong Zhao · Haoyuan Zhang 
 Jiankuo Zhao · Minghui Wu · Ping Jiang · Xiangyu Zhu · Chenxu Zhao † · Zhen Lei † 
 * Equal contribution.   † Corresponding authors. 
 📄 Paper ·
 🤗 Hugging Face Paper ·
 Updates ·
 Overview ·
 Results & checkpoints ·
 Citation 
 Installation ·
 Models ·
 SRNL & operator ·
 Fast inference ·
 Datasets ·
 Training ·
 Evaluation ·
 Efficiency 
PyTorch implementation of **VisionHOPE**, a visual backbone based on self-referential nested learning (SRNL).
This repository provides ImageNet classification, COCO detection and instance segmentation, and ADE20K semantic segmentation, along with standalone SRNL and VisionHOPE operator modules.
**Pretrained weights:** GitHub Releases · Hugging Face · Baidu Netdisk (`visionhope_weights`, access code: `8866`).
 From static local aggregation to self-modifying learning within an image. 
## Updates
- **2026-09-29** · Pretrained checkpoints for ImageNet-1K, COCO, and ADE20K are available from GitHub Releases, Hugging Face, and Baidu Netdisk (access code: `8866`).
- **2026-09-27** · The VisionHOPE preprint is available on arXiv.
## Overview
VisionHOPE builds on the self-referential construction in Nested Learning: The Illusion of Deep Learning Architectures. It lets **what the model remembers and how it learns evolve together** while processing an image. The VisionHOPE operator combines:
1. **Coupled memories.** Five memories store content, generate key and value representations, and govern learning rate and retention. They co-evolve as visual context accumulates.
2. **Stable updates.** A soft injection cap and a spectral clamp provide non-expansion guarantees for both token-wise and chunk-wise memory recurrences.
3. **Spatially aligned scans.** Four directional scans use row- and column-aligned chunks, followed by spatial restoration and learned channel-wise fusion.
 The VisionHOPE operator (left) and residual block (right). Click the figure to inspect the full resolution. 
The hierarchical **Tiny / Small / Base** backbones support classification and dense prediction. For use in other architectures, see the standalone SRNL, operator, and block interfaces.
## Installation
Use Linux with an NVIDIA GPU and the CUDA toolkit, including `nvcc` and a C++ compiler.
Python 3.10 is recommended for all tasks; classification also supports Python 3.11.
Run the following from the repository root, with CUDA 12.1 installed:
 [code] 
CUDA extensions compile on first use. Keep model parameters in FP32 and use `torch.autocast` for mixed precision.
 Additional dependencies for COCO and ADE20K 
Use a separate Python 3.10 environment for each task, with the PyTorch and CUDA versions above. Install MMCV, then the task dependencies:
 [code] 
For COCO:
 [code] 
For ADE20K:
 [code] 
## Models
Use these model names and checkpoint filenames for ImageNet-1K classification at 224 × 224. Accuracy and compl