Matrix Voice ESP32 MQTT Audio Streamer

#1

Hi all,

I wanted to share my little project, which can act as a remote microphone running standalone, without a Raspberry Pi.
The project started a while ago, when I received my Matrix Voice and wanted to use it with Snips. In my case I had the Snips Addon running on Hassio.

Since I ordered the esp32 version, I wanted to be able to have it running standalone.
The first version only was able to respond the hotword detection by changing the everloop colors.
I had to find out how the Audio Server from Snips was sending audio over MQTT and found that it published a large amount of wave files, so in my next version I was able to mimic this behaviour.

There were quite some stability issues, but I realized this was due to the fact that my code was trying to read and write to the SPI at the same time. I therefore implemented a semaphore system, after which stablility issues were no more occuring.
In the same process I switched to Arduino IDE, I was used to it and the libraries I used worked well.

For version 3, it was time to add over-the-air updates because I now had to attach the Voice to a Pi every time I wanted to update.
I also added in some other dynamic stuff.

The last big step is in version 3.4, which implements audio out through the headphone jack.
It now listens to a topic playBytes, which Snips uses to publish its output as 1 big wave message.
There were several issues and there are still issues, but this works for me at the moment.
The biggest issue for me right now is that the audio output is sometimes garbage, must be something with the ring buffer I use but I need put more effort in it to investigate.
It currently only outputs 441000 16bit stereo correctly, which is a problem with Snips

The Audio Streamer supports Snips, but is also able to be used with the Rhasspy Addon. I am actually using that because I can create voice commands in my native language (Dutch)

If someone is intrested, please check:

There are probably a lot of code standard issues and other issues, so if anyone has improvements please contribute to the project.

Enjoy!

4 Likes