> ## Documentation Index
> Fetch the complete documentation index at: https://elatoai.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Realtime Pipeline

> How OpenToys streams audio between an ESP32-S3 and a local STT, LLM and TTS pipeline over a websocket.

Cloud providers like OpenAI Realtime and Gemini Live ship one speech-to-speech model. OpenToys gets the same experience locally by chaining three models on your Mac.

```
ESP32-S3 mic ──ws──▶ VAD ─▶ Whisper (STT) ─▶ LLM (MLX) ─▶ TTS ──ws──▶ ESP32-S3 speaker
```

## Connection flow

**1. Toy boots**

The ESP32 starts an open access point called `ELATO` at `192.168.4.1`. The LED glows pink while it waits.

**2. Mac joins and handshake**

The toy probes ports `49320` and `8000` on your Mac until the OpenToys server answers, then opens a websocket. The LED turns white.

**3. Conversation**

Mic audio streams up, synthesized speech streams back. Press **End** in the app to close the session and view the transcript.

## Troubleshooting

* **`ELATO` isn't listed:** the toy isn't powered or the firmware didn't flash. Reflash and watch for the pink LED.
* **Joined but nothing happens:** make sure OpenToys is running on the Mac.
* **Play is greyed out:** wait for the status row to read `Ready on device`.

Want a true speech-to-speech model? Use [Cloud AI](https://elatoai.com/docs/models/openai.md).


This documentation is built and hosted on [Mintlify](https://mintlify.com), a developer documentation platform.