andrisgauracs / whistle-esp32

# Whistle speech recognition on an ESP32-S3
Hold a button and say *"flash the light blue"*. The LED flashes blue.
The speech recognition runs entirely on a $10 microcontroller: no network, no
cloud, no phone. It transcribes whatever you say rather than matching a fixed
list of phrases, and it understands commands in seven languages.
The model is Whistle by
Cactus Compute (Apache 2.0). Cactus ships no ESP32 build and no source for
Whistle's engine. Everything here is an independent C99 implementation with no
dependencies, checked tensor by tensor against the official engine.
## What you need
| Part | Notes |
|---|---|
| ESP32-S3 board, **N16R8** | 16 MB flash and 8 MB **octal** PSRAM. The model alone is 14.5 MB. ESP32-S3-DevKitC-1 N16R8 or a clone. |
| INMP441 I²S microphone module | the common round breakout board |
| Momentary push button | any tactile switch |
| Jumper wires, breadboard | 8 wires |
| USB cable | to the board's native **USB** port |
The RGB LED is the WS2812 that is already on the board.
## Wiring
| From | To | |
|---|---|---|
| INMP441 VDD | 3V3 | 3.3 V, not 5 V |
| INMP441 GND | GND | |
| INMP441 L/R | GND | picks the left channel, which the firmware reads |
| INMP441 SCK | GPIO4 | I²S bit clock |
| INMP441 WS | GPIO5 | I²S word select |
| INMP441 SD | GPIO6 | I²S data |
| Button | GPIO7 and GND | the internal pull-up is used, so no resistor |
| WS2812 LED | GPIO48 | on the board already |
Clone boards place header pins differently, so go by the GPIO numbers printed
on your board. If your LED sits on another pin (DevKitC-1 v1.1 uses GPIO38),
change `ONBOARD_GPIO` at the top of `esp32/whistle_demo/main/main.c`. A
PNG version of the diagram is there for printing.
**Check the microphone before anything else.** Loose jumper wires are the
most common fault. `esp32/mic_test` records while the button is held, and
`tools/mic_capture.py` saves each clip as a WAV you can listen to. Speech
should sit well above the noise floor (about -45 vs -68 dBFS) with `jumps=0`.
Stretches of an exact bit pattern such as `0xAAAA` mean a bad SCK, WS or power
contact.
## Setup
You need ESP-IDF
v5.5 or newer, Python 3 and CMake.
 [code] 
Find the port with `ls /dev/cu.usb*` (macOS) or `ls /dev/ttyACM*` (Linux).
The model only needs flashing once. After a code change, repeat step 3.
**Why cut the model down?** The full model is bigger than the board's entire
flash. Whistle's decoder is laddered: every depth from 2 layers up was trained
to work on its own. `whistle_strip.py` keeps only the layers a depth uses.
Depth 4 fits once the model partition runs to the end of flash
(`esp32/whistle_demo/partitions.csv`).
### Watching it work
 [code] 
The console shows each transcript, the detected language, the LED action and
the time each phase took. The board works without it, too.
## Using it
The LED glows dim blue for about a second while the model loads, then goes
off. That means it is ready.
1. **Hold** the button. The LED turns **white** while it records.
2. *