GENTEL-lab / Fold2Reason

# Fold2Reason
 📄 Paper ·
 🤗 Model checkpoints ·
 🤗 FoldingCorpus 
Protein-structure post-training for language models. **Fold2Reason** combines **FoldingCorpus** supervision with a shared spatial workspace and a frozen geometry decoder. General reasoning is evaluated with the adapted language model; structural readout uses the workspace and decoder.
This repository contains the Qwen3.5 training and evaluation pipeline, a FoldingCorpus-only Pure-LoRA baseline, deterministic data preparation, and General-10 score aggregation.
The figure shows LM-Folding performance and per-benchmark gains over the base model.
Start here: install → connect `dataset.zip` → download the base model and external benchmarks → evaluate released checkpoints or retrain.
## Install
Use Python 3.12 and a CUDA-compatible PyTorch installation. The tested environment uses PyTorch 2.10.0. Transformers and PEFT are pinned to the source revisions recorded in this repository; see installation.
 [code] 
Install the matching PyTorch/torchvision CUDA wheels for your system before the editable install. Run all commands below from the repository root in this environment. The source checkout contains code and recipes; project data and trained weights are distributed separately in `dataset.zip`.
## Project bundle: `dataset.zip`
Download dataset.zip from Google Drive and save it as `data/dataset.zip`. For command-line download, use gdown:
 [code] 
`--continue` resumes an interrupted download. If Google Drive temporarily limits automated downloads, use the browser link and save the ZIP at the same path. After the ZIP integrity check passes, follow Extract and connect the bundle.
The archive contains the main experiment's data, frozen decoder, and three trained LoRA/workspace checkpoints:
 [code] 
The 34 files cover 1,000 training / 100 development / 100 held-out proteins, their FoldingCorpus supervision, all four FTB-Core splits, the processed 334-protein FoldBench snapshot, and the main experiment's fixed epoch-3 / step-375 checkpoints. Keep each adapter paired with its same-seed workspace. The frozen decoder is shared across seeds.
The official Qwen3.5-9B weights and nine external General-10 datasets are downloaded separately below. Tokenized training/FoldBench caches, visual-feature caches, and the 50–4,000-protein scaling collections are **not included** in this ZIP. Cache-building instructions follow; the bundle supports the 1,000-protein main experiment.
### Extract and connect the bundle
Run this once after downloading. It accepts archives with or without an outer `dataset/` directory, preserves the uploaded layout, and creates directory symlinks at the paths used by the recipes. Re-running checks existing links; conflicting destinations raise an error instead of being replaced.
 [code] bash
fold2reason data geometry \
 --model models/Qwen3.5-9B \
 --data-root data/protein_training \
 --train-file data/protein_training/train.jsonl \
 --validation-file data/protein_training/dev.jsonl \