microsoft / unilm

Large-scale Self-supervised Pre-training Across Tasks, Languages, and Modalities

datajuicer / data-juicer

Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷

datawhalechina / diy-llm

Covers pre-training data, Tokenizer, Transformer, MoE,distributed training, Scaling Laws, inference & alignment .6 progressive code assignments for full-stack LLM learning | 涵盖预训练数据、分词器、Transformer、MoE、分布式训练、缩放定律、推理与对齐，6 项渐进代码作业，掌握 LLM 全栈知识

LMResiliency / lm-resiliency

Closed-loop fault localization and checkpoint recovery for LLM Pre-training

starVLA / VLAct

[NeurIPS 2026] Beyond Data Scaling: Representation-Centric Continued Pre-training for Vision-Language-Action Models

datamllab / OpenEBFM

Open-source code for training and scaling up energy-based foundation models.

ifm-ai / xllm

Light-weight infrastructure for express LLM pre-training

WB-WaM / WB-WAM-Official

WB-WAM: Heterogeneous Body–Hand Pre-training for Humanoid Loco-Manipulation.