raullenchai / esp32-thinks

# esp32-thinks
**A multi-model Muse — a personal AI assistant on ESP32, talking out of the box.**
Think of it as a Muse that thinks with **your** models: say **"Hi, ESP"** (or press the knob), ask anything, and it answers **out loud** — streaming sentence-by-sentence while the model is still writing, in English, Chinese or Korean, on a small gadget with a mic, a speaker and a round display.
No vendor lock on the brain: **speech, transcription and chat are three independent endpoints**, each pointed at any OpenAI-compatible server — one provider for all three, or a different one per task. Run it bare against QuickSilver, or put the in-repo **openmuse** agent in front to give it location, weather, time and memory.
- ESP32-S3 · built on Meta's open-source muse-gadget-sdk · license notices
- Models: **OpenAI**, **QuickSilver**, a local agent, or any mix — presets for the common ones, fully custom for the rest
## Why
Cloud assistants rent you a brain: one vendor's models, one vendor's pricing, one vendor's uptime, and your voice on their servers. The gadget hardware is fine — the lock is in the firmware.
esp32-thinks flips that. The device runs **your** stack: pick the cheapest transcription, the smartest chat model, the voice you like — per task, changeable in one config line and a rebuild. The implementation is a reviewable patch against a pinned open-source SDK, so you can read it, change it, or point it at a local server. The optional openmuse agent keeps the extra context (location, memory, tools) on **your** machine too.
## What you get
- **Wake word + button** — say "Hi, ESP" (esp-sr WakeNet10, runs on-chip) or hold the knob; both start a turn. Wake turns end on a voice-activity timer; the button works exactly like before.
- **Bring your own brain** — three independent OpenAI-compatible endpoints (STT / chat / TTS) in one config file. Mix providers freely; presets: `openai`, `quicksilver`, `openmuse`, `custom`.
- **Optional agent layer** — the in-repo `openmuse/` adds location, clock, weather, persistent memory and optional web search, owns the system prompt, and proxies STT so STT + chat share one TLS session. TTS still goes straight to your provider.
- **Streaming voice replies** — the chat stream is parsed at sentence boundaries (ASCII + CJK); each sentence is synthesized the moment it lands and appended to the audio already playing. First word arrives while the model is still generating.
- **Long-lived brain connection** — TLS is prepared before you ask, probed every 180 s while idle, and resumed with session tickets after an idle close. A failed probe triggers background preparation; a disconnect between probes may still need turn-time reconnection.
- **Barge-in** — press the knob while Muse is speaking: the reply stops mid-sentence, in-flight requests are cancelled, and your next words start a fresh turn immediately.
- **Silence guard** — a quiet recording (interrupted turn, pocket press) is discarded instead of becoming a hallucinated an