chenglou / handwired

# Handwired
Turn your regular JavaScript code into a neural net; no training, exact, yet also fully differentiable!
## How It Works
A program's made of if-else, loops, and state. Handwired transforms them into matrix multiplication + reglu over floats. For example:
 [code] 
The above can be expressed as basic arithmetic:
 [code] 
You can calculate that by hand and see that both arrive at the same result.
We then collect the + and * and relu into a giant matmul, and either run it on JS or WebGPU, for fun.
Because we basically only use these 2 primitives:
- the full program stays differentiable
- the weights don't need training since they're written manually
- existing gpu techniques, such as fusion, sparsity, reordering, have a great time optimizing such minimal primitives
This unlocks lots of funny use-cases, like:
- unfreezing the weights and train them (what does that even do? Let's find out!)
- attaching a specific frozen program to a generic neural net and train end-to-end (now a nanoGPT can do both Shakespeare and your particular UI?)
- achieving GPU-level of parallelism with JS code that was written in a traditional single-threaded way (see the JSON search demo that runs in milliseconds over 100k documents)
## Play With It
Try the demos live: https://chenglou.me/handwired/
`git clone && bun install`, then `bun start` to run them locally. Check `docs/`, `src/` and `template/` to get started! (`lab/` is a bunch of old experiments you can check or ignore)