SakanaAI / AI-Scientist

The AI Scientist: Towards Fully Automated 
 Open-Ended Scientific Discovery 🧑‍🔬 
 📚 [Paper] |
 📝 [Blog Post] |
 📂 [Drive Folder] 
One of the grand challenges of artificial intelligence is developing agents capable of conducting scientific research and discovering new knowledge. While frontier models have already been used to aid human scientists—for example, for brainstorming ideas or writing code—they still require extensive manual supervision or are heavily constrained to specific tasks.
We're excited to introduce **The AI Scientist**, the first comprehensive system for fully automatic scientific discovery, enabling Foundation Models such as Large Language Models (LLMs) to perform research independently.
We provide all runs and data from our paper here, where we run each base model on each template for approximately 50 ideas. We *highly* recommend reading through some of the Claude papers to get a sense of the system's strengths and weaknesses. Here are some example papers generated by **The AI Scientist** 📝:
1. DualScale Diffusion: Adaptive Feature Balancing for Low-Dimensional Generative Models
2. Multi-scale Grid Noise Adaptation: Enhancing Diffusion Models For Low-dimensional Data
3. GAN-Enhanced Diffusion: Boosting Sample Quality and Diversity
4. DualDiff: Enhancing Mode Capture in Low-dimensional Diffusion Models via Dual-expert Denoising 
5. StyleFusion: Adaptive Multi-style Generation in Character-Level Language Models
6. Adaptive Learning Rates for Transformers via Q-Learning
7. Unlocking Grokking: A Comparative Study of Weight Initialization Strategies in Transformer Models
8. Grokking Accelerated: Layer-wise Learning Rates for Transformer Generalization
9. Grokking Through Compression: Unveiling Sudden Generalization via Minimal Description Length
10. Accelerating Mathematical Insight: Boosting Grokking Through Strategic Data Augmentation
> **Note:** 
> **Caution!** This codebase will execute LLM-written code. There are various risks and challenges associated with this autonomy, including the use of potentially dangerous packages, web access, and potential spawning of processes. Use at your own discretion. Please make sure to containerize and restrict web access appropriately.
## Table of Contents
1. Introduction
2. Requirements
 - Installation
 - Supported Models and API Keys
3. Setting Up the Templates
 - NanoGPT Template
 - 2D Diffusion Template
 - Grokking Template
4. Run AI Scientist Paper Generation Experiments
5. Getting an LLM-Generated Paper Review
6. Making Your Own Template
 - Community-Contributed Templates
7. Template Resources
8. Citing The AI Scientist
9. Frequently Asked Questions
10. Containerization
## Introduction
We provide three templates, which were used in our paper, covering the following domains: **NanoGPT**, **2D Diffusion**, and **Grokking**. These templates enable The AI Scientist to generate ideas and conduct experiments in these areas. We accept contributions of new templates from the community, but plea