(Solved) MATRIX Voice - Documentation clarification & PyAudio test

Firstly, thanks for a promising product for making speech recognition an accessible development.
So, after a much anxious wait, I received my MATRIX Voice 2 days back and it looks flashy and exciting. However, my excitement slowly waned once I started looking into the documentation and tried setting it up. There are a couple of clarifications I am looking for in terms of the documentation and voice recognition of Matrix Voice.

  1. I assume that the installation instructions for matrix creator are the same one for matrix voice. The basic installation was alright but some of the terms are confusing. I understand that Matrix OS is only available in Javascript and it is mainly purposed for building applications for Matrix App store. Being a Python developer, Matrix Core seems to be the best option for me to develop on Matrix Voice. However, it is not clear from the documentation, and from the various community questions about how we can access microphone data using Python. Also, is MALOS same as matrix core or is it different?

  2. After reading various answers in the community I have come to the understanding that it’s not easy (or, not out-of-the-box setup) to integrate a custom wakeword or speech recognition module(like wit.ai, google speech api etc.) in Matrix Voice using python as there is no support for PyAudio. Am I right about this?

I understand it takes time to develop the libraries but it would be useful to explicitly state the current limitations and maybe, give an approx. timeline about upcoming features so that we don’t spend/waste time on something that is bound to not work, yet.

  1. MALOS is the same as MATRIX Core. Matrix OS will support Python someday soon.
  2. There is currently support for PyAudio, we don’t have that in docs yet, here is an example. https://github.com/matrix-io/matrixio-kernel-modules/blob/master/misc/pyaudio_test.py

Thanks @CreatorSean for pointing me in the right direction.

Finally, I got the Snowboy detector to work with Matrix Voice using the pyaudio code in kernel-modules. Now, I am trying to recreate Alexa example, wherein, the everloop glows when the hotword is detected.
From this post (Kitt_ai wakeword engine still not working on Matrix VOICE), I understand that kernel-modules and Matrix-HAL cannot work together. And since Matrix-HAL controls the everloop function, that doesn’t work.

I followed the instructions from that post by uninstalling MALOS and HAL packages. Built the kernel-modules and then built the hal package from source. I, then checked out the ac/kernel_space branch from the matrix-hal repo.
The microphone works fine but when running the everloop demo, the demo does not work. Earlier “can’t open any device” used to show up. But now, the demo gets stuck and does not give any output (on terminal and on Matrix voice).

Would be great if you can help me with that!

no that’s wrong… the new matrixio-hal now supports also kernel modules. it’s using the new regmap device. you get an info message saying which device was used (spi or regmap). you could also use the matrixio_everloop character device to control the everloop even without any hal drivers, or if you want you can use my python hal drivers to do this in python, if you don’t want to use malos.

here are both devices you can have look at.

pi@matrix2:~ $ ls -l /dev/matrixio_*
crw-rw-rw- 1 root root 241, 0 Apr 19 21:37 /dev/matrixio_everloop
crw-rw-rw- 1 root root 240, 0 Apr 19 21:37 /dev/matrixio_regmap

for example to control the the first led you could echo this to /dev/matrixio_everloop:

 echo -n -e '\x00\x10\x00\x00' > /dev/matrixio_everloop

this is setting the red led to a value of 16 (hex 10). but the matrixio_everloop is still having a wired byte order. it is not RGBW it is GRWB. each led is using 4 bytes so if you add another 4 bytes you address the second led and so on. i’ll will talk (submitting a patch) to the devs if the want to change the order to RGBW so it is more intuitive.

update
i also created a pull request to change the order to RGBW maybe they will merge it:slight_smile:

1 Like

@loom
Nicee !

So, I have re-installed raspbian Stretch. Made sure to build kernel-modules first and HAL and MALOS. The MALOS and HAL demos are working without error. However, now my RPi is not detecting any Matrix Voice’s microphone. The mic_energy demo works but not the recorder (obviously).

I updated the .asoundrc file as given here (https://github.com/matrix-io/matrixio-kernel-modules/blob/master/misc/asound.conf) but still no effect.

Any idea why this is happening? Sorry, I am just trying to figure out all the installations and configurations and the fact that they are all over the place isn’t making this easier.

Also, can you point to your python libraries?

You do not need to build the modules manually. Just install them with ‘sudo apt install matrixio-kernel-modules’ . This will also set all required configuration settings. E.g. The Spi buffer size and so on.

1 Like

@loom Thanks a lot! Its kinda working now.

Couple of things to note:

  1. After kernel-modules are loaded, the pyaudio_test.py works and records sound albeit with white noise.
  2. Running a HAL-demo (eg everloop_demo) works and opens at regmap like you mentioned. But once this is opened, the previous pyaudio_test.py does not work.
    15%20AM

Could be a bug or something but for now I am happy that I got the recording and everloop working.

Is there a way that both can work simultaneously?

Hmm you mean, if you have the everloop_demo running, then the recording doesn’t work anymore? But if you finish the everloop_demo it will work again? If so, can you describe exactly what you call where and when, I will test it myself tonight.

As far as I know the internal SPI device is locked now, as soon as traffic runs over it, maybe something goes wrong, or the everloop_demo sends too much data too fast.

running both everloop_demo and pyaudio_test.py at the same time is working on my rpi. eventually you have your on .asoundrc in you home folder, or something other strange thing. do you have malos installed? if yes try stopping it with ‘sudo systemctl stop matrixio-malos.service’ and try again.

1 Like

Strangely, it works now without any errors. Thanks a lot for your help.

I have managed to get the Snowboy detector and Speech Recognition working with Matrix Voice. The MALOS’s python script to set everloop color works great too with the system. The audio quality is bad and I need to be close to the MVoice for the word to be detected but I guess that could be fixed if I use arecord or set a specific mic_channel.

For reference/example: https://github.com/shashankpr/HeyDobby/tree/matrix-voice

On a side note, can you point me to your python drivers for HAL or a way using MALOS to create a arc/spinning animation on everloop (like the C++ demos in HAL library)?
Certainly not a priority. Asking out of curiosity!

here it is: https://github.com/cmetz/python-matrixio-hal/blob/master/examples/led_rotate.py

1 Like

Awesome! Thank you once again, @loom