Whisper

OpenToys transcribes your toy’s microphone audio with Whisper large-v3-turbo, loaded through mlx-audio.


Model	`mlx-community/whisper-large-v3-turbo-asr-fp16`
Runs on	Apple Silicon via MLX
Languages	Multilingual (English, Chinese, Spanish, French, Japanese, Korean, Portuguese, German, Italian and more)

How it works

The ESP32-S3 captures audio from an I2S microphone (INMP441) and streams it to your Mac over a websocket.
Voice activity detection finds the end of your sentence.
Whisper transcribes the utterance and hands the text to the LLM.

The model downloads on first setup into the app data directory. After that it works offline.