alienflm / MacTok

MacTok: Robust Continuous Tokenization for Image Generation 
CVPR 2026 · **Highlight**
Hengyu Zeng\* · Xin Gao\* · Guanghao Li · Yuxiang Yan · Jiaoyang Ruan · Junpeng Ma · Haoyu Albert Wang · Jian Pu† 
 Fudan University   |   \*Equal contribution   |   †Corresponding author 
 ImageNet samples at 256×256 and 512×512 from generators trained on MacTok latents (64 and 128 tokens). 
---
Official PyTorch implementation of **MacTok**, a **M**asked **A**ugmenting 1D **C**ontinuous **Tok**enizer.
Continuous tokenizers compress an image into a small set of latent vectors that a diffusion model can then
generate. A KL term keeps that latent space smooth, but under strong compression it also causes **posterior
collapse**: the encoder stops writing useful information into the latents, and the decoder reconstructs from
its prior instead.
MacTok prevents collapse by making the latents indispensable. During training the encoder sees only part of
the image, so the latent tokens must carry whatever the decoder cannot infer on its own. Aligning those
tokens with pretrained DINOv2 features keeps the compressed space semantically organized. With 64 or 128
tokens, MacTok reaches a gFID of 1.44 on ImageNet 256×256 and 1.52 on 512×512.
## Highlights
- **64 or 128 tokens per image**, up to 64× fewer than conventional 2D tokenizers, at 256×256 and 512×512.
- **State-of-the-art generation quality**: gFID 1.44 (256×256) and 1.52 (512×512) with SiT-XL.
- **No posterior collapse**, because masked reconstruction forces real information into the latents.
- **Semantically structured latents** from joint global and local DINOv2 alignment, which makes the
 downstream diffusion model easier to train.
## Method
A ViT encoder takes an image split into patch tokens, concatenates a learnable set of `L` latent tokens,
and keeps the encoder outputs at the latent positions as the continuous code. That code is treated as a
Gaussian and regularized toward an isotropic prior. A ViT decoder reconstructs the image from it. Three
ingredients keep the code informative under this constraint.
**Masked reconstruction.** A fraction of patches is replaced by mask tokens before encoding, so the model
must reconstruct the full image from partial evidence. The mask ratio is drawn from `max(0, U(-0.1, M))`
with `M = 0.7`; the negative lower bound means the model occasionally sees an unmasked image, which stops
persistent masking from degrading reconstruction quality.
**Semantic masking.** Random masking ignores image content, so MacTok adds a second strategy that scores
each patch by its cosine similarity to the DINOv2 class token and masks the highest-scoring ones. Hiding
the most informative regions makes the task harder and pushes object-level structure into the latents. The
two strategies are applied with equal probability during training.
**Representation alignment.** Each latent token is repeated to patch granularity and aligned with DINOv2
features both locally and globally. Local alignment preser