adam-maj / tiny-gpu

# tiny-gpu
A minimal GPU implementation in Verilog optimized for learning about how GPUs work from the ground up.
Built with [!IMPORTANT]
>
> **tiny-gpu** is a minimal GPU implementation optimized for learning about how GPUs work from the ground up.
>
> Specifically, with the trend toward general-purpose GPUs (GPGPUs) and ML-accelerators like Google's TPU, tiny-gpu focuses on highlighting the general principles of all of these architectures, rather than on the details of graphics-specific hardware.
With this motivation in mind, we can simplify GPUs by cutting out the majority of complexity involved with building a production-grade graphics card, and focus on the core elements that are critical to all of these modern hardware accelerators.
This project is primarily focused on exploring:
1. **Architecture** - What does the architecture of a GPU look like? What are the most important elements?
2. **Parallelization** - How is the SIMD progamming model implemented in hardware?
3. **Memory** - How does a GPU work around the constraints of limited memory bandwidth?
After understanding the fundamentals laid out in this project, you can checkout the advanced functionality section to understand some of the most important optimizations made in production grade GPUs (that are more challenging to implement) which improve performance.
# Architecture
## GPU
tiny-gpu is built to execute a single kernel at a time.
In order to launch a kernel, we need to do the following:
1. Load global program memory with the kernel code
2. Load data memory with the necessary data
3. Specify the number of threads to launch in the device control register
4. Launch the kernel by setting the start signal to high.
The GPU itself consists of the following units:
1. Device control register
2. Dispatcher
3. Variable number of compute cores
4. Memory controllers for data memory & program memory
5. Cache
### Device Control Register
The device control register usually stores metadata specifying how kernels should be executed on the GPU.
In this case, the device control register just stores the `thread_count` - the total number of threads to launch for the active kernel.
### Dispatcher
Once a kernel is launched, the dispatcher is the unit that actually manages the distribution of threads to different compute cores.
The dispatcher organizes threads into groups that can be executed in parallel on a single core called **blocks** and sends these blocks off to be processed by available cores.
Once all blocks have been processed, the dispatcher reports back that the kernel execution is done.
## Memory
The GPU is built to interface with an external global memory. Here, data memory and program memory are separated out for simplicity.
### Global Memory
tiny-gpu data memory has the following specifications:
- 8 bit addressability (256 total rows of data memory)
- 8 bit data (stores values of <256 for each row)
tiny-gpu program memory has the following specifications:
- 8 bit addressability (256 rows