How to Set Up Local Voice Control in Home Assistant with Assist

Saying "turn off the lounge lights" and having it happen, without a single word leaving your house, is the payoff of a local voice pipeline. Assist is Home Assistant's built-in voice assistant, and paired with the Whisper and Piper add-ons it handles speech-to-text and text-to-speech entirely on your own hardware. Your commands never touch a cloud server, which is both a privacy win and a reliability one.

Quick Answer

Set up local voice control by installing the Whisper add-on for speech-to-text and the Piper add-on for text-to-speech, then create an Assist pipeline in Home Assistant that uses both. With those local components, every spoken command is processed on your own hardware and nothing is sent to the cloud.

What runs the pipeline

A voice command passes through three stages, and a fully local setup handles each on your own machine. First, wake-word detection notices you are talking to it. Second, speech-to-text turns your words into a command, which is Whisper's job. Third, text-to-speech speaks the reply back, which is Piper's job. Whisper suits more capable hardware, while Piper is light enough to run well even on modest devices like a Raspberry Pi 4.

The result is a pipeline where your audio is captured, understood, acted on and answered without an internet round trip. That keeps response times consistent and means your voice data stays in your home.

Step-by-step: set up local voice with Assist

1. Install the Whisper add-on for speech-to-text

In Home Assistant, go to Settings, Add-ons, open the add-on store and install Whisper. Start it and choose a model size that suits your hardware: smaller models are faster and lighter, larger ones are more accurate but want more processing power. Whisper is the component that converts your spoken words into text Home Assistant can act on.

2. Install the Piper add-on for text-to-speech

From the same add-on store, install Piper and start it. Piper is a fast local neural text-to-speech engine, and it is efficient enough to run smoothly even on lower-powered hardware. Pick a voice you like in its configuration. This is what speaks responses back to you.

3. Build an Assist pipeline

Go to Settings, Voice assistants, and create or edit an Assist pipeline. Under speech-to-text select your Whisper engine, and under text-to-speech select Piper. This ties the local components together into one pipeline so Assist uses your own hardware end to end rather than any cloud service.

4. Set up wake-word detection

Choose a wake word so the assistant only listens after it hears the trigger, such as the built-in "Okay Nabu". Wake-word detection runs in Home Assistant: a voice satellite samples audio in the room, and when it detects the wake word it sends the following audio to Home Assistant to process. Configure your wake word in the pipeline or on the satellite device.

5. Add a voice satellite or input device

You need something to listen and speak in the room: a dedicated voice satellite device, or a small device running the satellite software with a microphone and speaker. Point it at your Home Assistant instance so its captured audio flows into the pipeline you built. For the mic, speaker and small-form hardware that make a good satellite, the smart home and accessories range at Evetech is a useful place to assemble the parts.

6. Test and refine

Say the wake word, then a command like turning a light on or off, and confirm Assist acts and replies through Piper. If recognition is shaky, try a larger Whisper model or move the satellite closer to where you speak. If responses are slow, a more capable host helps Whisper run faster, since speech-to-text is the heaviest stage of the pipeline. You can browse mini PCs and other capable hosts through the popular hardware on the Evetech bestsellers shelf if your current device is struggling to keep up.

Why local is worth the effort

A cloud voice assistant stops working when your internet does, and it routes your voice to someone else's servers. A local Assist pipeline keeps working through an outage and keeps your audio in your home, which is the whole appeal for a privacy-minded smart home. The trade-off is that speech-to-text is demanding, so a slightly more capable host pays off in responsiveness, especially if you run a larger, more accurate Whisper model.

Frequently Asked Questions

Does local voice control really keep my audio private?

Yes. With Whisper for speech-to-text and Piper for text-to-speech, every stage runs on your own hardware, so your spoken commands are never sent to a cloud server. The audio stays within your local network from capture to response.

What hardware do I need for a local Assist pipeline?

Piper runs well even on modest hardware, but Whisper is heavier and benefits from a more capable host. A mini PC or similar device handles speech-to-text far more responsively than a low-powered board, especially with a larger, more accurate model.

Which speech-to-text engine should I use?

Whisper is the common choice for accurate local speech-to-text, with model sizes you can match to your hardware. Smaller models are faster and lighter; larger ones are more accurate but want more processing power, so balance speed against accuracy for your device.

How does the wake word work?

A voice satellite continuously samples room audio, and when it detects the wake word it switches from passive listening to active and sends the following audio to Home Assistant. The detection happens locally, and you can use a built-in wake word such as Okay Nabu.

Will local voice work when my internet is down?

Yes, that is a key advantage. Because the whole pipeline runs on your own hardware, a fully local Assist setup keeps responding to commands during an internet outage, unlike a cloud assistant that goes silent the moment the connection drops.