[solved] Matrix voice not working

#1

Hi,

I got a matrix voice few weeks back and we made it work somehow before. The everloop was working and was working on the microphone part to make it work. I was working on the snowboy integration. On generating the model, i tested the model and it worked fine but on the matrix it did not work. I double checked the mic, it worked fine. Hence, i reinstalled the raspberry pi debian os and since then i don’t remember seeing the white everloop at startup as before. I installed matrix core couple of times, reinstalled the debian os couple of times. I tried the matrix core demo and hal demo but in any of it the starter everloop demo did not work. It does not give any error or output. I tried console.log and it was printing. It successfully goes into the set interval method and keeps printing on the console but the ever loop does not work. I don’t know whats the problem. Is the device broken? Should i reset the matrix voice? if yes, how do i do it? Can someone please help me out as soon as possible? thanks a lot

KV

[solved] ]Matrix voice Software issues
#2

Hi @koushikv,
First, if you have any doubt about a hardware problme you can always using this test ,MATRIX Creator/Voice Hardware Test

Few quesions:

You said you tried the snowboy integration. The Snowboy repo comes with pretrained models for alexa, snowboy and jarvis (here).

  1. Did you tried with these models?
  2. Were you able to build the Sampleapp using the snowboy wakeword?

Take a look to this conversation about the snowboy wakeword Kitt_ai wakeword engine still not working on Matrix VOICE

-Yoel

#3

Hi,

thanks for the info. I did a clean install of os and hal, the everloop demo works but before i proceed to the snowboy, i have a couple of doubts

  1. Does matrix voice work with matrix OS and core ? since i was trying before it didn’t work and also the matrix voice is MENTIONED only in requirements of matrix hal.

  2. If no, could you let me know how to execute python scripts in matrix hal? since it only has c++ language examples and the js and python links in the documentation does not work.

  3. in the kitti audio not working post , it is mentioned to uninstall hal/malos. could you let me know how to do it? and can i execute snowboy only in c++ or is python possible?

thanks

#4

Hi @koushikv!

To work with Matrix Voice using Python, you need to install/build MALOS. MALOS has JS and Python support. There are python examples in the MALOS repo, if you want to check how it works : https://github.com/matrix-io/matrix-creator-malos/tree/master/src/python_test

Alternatively, there is a great python bindings based on HAL, maintained by @loom. I use it as I find it easier to understand and it gives more ways to play with everloop and sensors. The repo is : https://github.com/cmetz/python-matrixio-hal

Regarding snowboy, I got it working with Matrix Voice using python. If you can record using Matrix Voice then you just need to specify Matrix Voice’s device index in snowboydecoder.py file.
For reference this is how I got it to work: https://github.com/shashankpr/HeyDobby/tree/matrix-voice

2 Likes
#5

thanks for the info. The everloop and mic recorder direct demo in creator hal worked fine.Hence, i was about to start with snowboy. I started with recording but i haven’t crossed the recording audio step yet. The reason is the microphone is not available in arecord list . I copied the soundrc file from below link:

I remember doing this before and it worked but not now. The arecord -l output is

**** List of CAPTURE Hardware Devices ****
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
Subdevices: 8/8
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7

and arecord -L output is :

null
Discard all samples (playback) or generate zero samples (capture)
default
mic
speaker
array
sysdefault:CARD=Dummy
Dummy, Dummy PCM
Default Audio Device
dmix:CARD=Dummy,DEV=0
Dummy, Dummy PCM
Direct sample mixing device
dsnoop:CARD=Dummy,DEV=0
Dummy, Dummy PCM
Direct sample snooping device
hw:CARD=Dummy,DEV=0
Dummy, Dummy PCM
Direct hardware device without any conversions
plughw:CARD=Dummy,DEV=0
Dummy, Dummy PCM
Hardware device with all software conversions

I tried various steps mentioned in below ink but in vain

Can somebody please help me to make the mic record? Once this works i can move on to snow boy :slight_smile:

thank you

#6

thanks a lot guys. Recording works. I made only one change.
in /boot/config.txt , i added the dtoverlay=matrixio line at the end but with a space above. I removed the space and it worked.
Now, the snowboy sample python script with snowboy umdl is not recognising my voice. The arecod -l goes like this

**** List of CAPTURE Hardware Devices ****
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
Subdevices: 8/8
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7
card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0

Am i suppose to reorder to make it work ? or is there any other reason it doesn’t work?
thanks :slight_smile:

#7

Hello @koushikv,

1- Did you install the kernel module via apt or you compile it from source?

Check the kernel modules documentation: https://github.com/matrix-io/matrixio-kernel-modules.

2- Could you share with us the error message from the snowboy sample python? As I understood you are able to record using arecord?

Regards,

Kevin

#8

Hi Kevin,

I installed the kernel modules from source

I am not getting any errors. It is listening but not detecting the command. I tried both snowboy and alexa models but not working . Below is the output i get on execution of python script

ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘defaults.bl uealsa.device’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned err or: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or di rectory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘defaults.bl uealsa.device’
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned err or: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or di rectory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unloc k
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unloc k
Listening… Press Ctrl+C to exit

that is the reason i asked in the previous reply to reorder the input device list. Should i do it ? and how ?
thanks

#9

Hi,

I modified snowboydetector.py as mentioned by @shashankpr. I appended index to the code as 2 as can be seen from arecord -l in the previous post but still only listening and no response.

I wanted to install matrix malos and look for python script to access microphones but before that, i wanted to check the everloop demo as i did in the beginning and it wasn’t working. In the beginning during testing of the device, i installed hal and tested the everloop demo but now it doesn’t. It throws error “can’t open device”

Can someone please help me with this issue? and also i don’t find a sample python script for mic but avaialble in js, can someone pls share or let me know where to find a sample python script for mic?
thank you

#10

Hi @koushikv, I faced similar issues initially.

Let’s tackle them one-by-one:

  1. Check if matrix voice is able to record your voice using PyAudio.
    Check this link first: https://github.com/matrix-io/matrixio-kernel-modules/blob/master/misc/pyaudio_test.py

This is the pyaudio test that Matrix Voice is tested to work after installing/updating to the latest Kernel modules. If this pyaudio code runs for you, you should see a output.wav file. Use aplay output.wav to check if you are able to hear what you recorded.

  1. If the above step works for you, then Snowboy should work too - with some tweaks to the original snowboydecoder’s code.

Change the following code in snowboydecoder.py :

Default code:
self.stream_in = self.audio.open(
input=True, output=False,
format=self.audio.get_format_from_width(
self.detector.BitsPerSample() / 8),
channels=self.detector.NumChannels(),
rate=self.detector.SampleRate(),
frames_per_buffer=2048,
stream_callback=audio_callback)

Modified for Matrix Voice
self.stream_in = self.audio.open(
input=True, output=False,
format=self.audio.get_format_from_width(
self.detector.BitsPerSample() / 8),
channels=MV_CHANNELS,
rate=MV_SAMPLERATE,
frames_per_buffer=2048,
stream_callback=audio_callback,
input_device_index = MV_INDEX)

where, MV_CHANNELS = 1, MV_SAMPLERATE=16000, MV_INDEX=2 (in my case)

Try these two steps and let me know if it works.

  1. Regarding the

can’t open device error

I got the same error when I tried to first record voice using Matrix voice’s mic and then running the HAL examples. It seems like they cannot run simultaneously. However, I fixed the issue after uninstalling all the matrix modules (kernel modules, HAL and MALOS) and reinstalling them.

I would suggest you to first try the first two steps and check if the microphone works with Snowboy. If the error in 3) still persists then I will point you the right sequence of installations.

1 Like
#11

The way I resolved the error in 3) mentioned above is:

Uninstall MALOS/HAL

  1. sudo apt-get purge matrixio-malos matrix-creator-init matrix-creator-malos
  2. sudo apt-get purge matrixio-creator-init libmatrixio-creator-hal libmatrixio-creator-hal-dev
  3. sudo apt autoremove

Install the New Kernel Modules
Follow the steps given in this post : MATRIX Hal - Important Update

Check if the error still persists.

#12

Hi @shashankpr,

I purged and reinstalled everything. I modified the snowboydecoder.py too but it is only listening and no response. I tried the everloop demo as well and it still throws the same error. The pyaudio still works as before . The script executed successfully and output was good but snowboy’s not working. Am i missing any step for snowboy to work with the matrix microphone? should i try any python script from malos? but i don’t see a sample python script for mic. Do you know where i can find a script to input the microphone output directly to snowboy? thanks

#13

Just mention here that there is an update guide for Alexa (with support for Snowboy in C++).
Read here: Kitt_ai wakeword engine still not working on Matrix VOICE

Not sure if this would help.

#14

@koushikv,

I was reading all this thread and was wandering if you have any problem with the MATRIX Voice hardware itself. Were you be able to record audio from the mics and verify the LEDs are good?

If there are not problems, I’ll suggest to change the title of the post or just mark it as [Solved] and maybe open a new one for your current questions.

Thank you,

#15

Hi @yoelrc88,

The mics work with arecord and once i tried with hal mic_direct_demo,it worked. This was intially but now hal examples doesn’t work and i tried with malos as well, test everloop throws error “Connected to error publisher with port 20023”. Should i continue with these problems in this thread ?or a new one?. These errors keeps coming back.

#16

From what you said it looks like the LEDs and the mics don’t have any hardware problem. The issues are probably related to software installation problems. Sorry, I suggested marking the post title as Solved because of this. We can continue different issues in new posts, this way the content is more organize for other users can get help here.

Also regarding this issues:

Can you provide more info related to this test. Outputs?

Also would need mor information about this one.

Better to open a new post for this different tests/issues.

Thank you for the feedbacks on this!

-Yoel

#17

Hi,

thanks @yoelrc88 i am continuing the software issues on new thread

#18

Hi @yoelrc88,

I only followed the steps mentioned in hal. Initially, it worked. This i am talking after installation but i tried later it throws error. Do you think it might be to do with snowboy installation? and also i would like to inform that this is not the first time.I’ve reinstalled OS due to this issue before. Is there any setup i need to do before i try every time to run the hal examples?

The same with malos as well but this time it didn’t work even once. I tried driver script it worked hence moved on to everloop but it throws error as mentioned before. Also i would like to add it has been long time since i am able to execute any malos scripts. Earlier, i don’t get any output for the everloop demo. It used to be blank but i added console log and it printed the output continuously. So, it means it calls the interval method but there was no light on the matrix voice. This time i am getting an error output.

#19

Ok I understad that you still have some issues with the setup. HAL and MALOS are currently in stable mode and there should be no problem with them, so we will help you out to solved the issues.

I see that you added new posts with new questions … see you there

Thanks

-Yoel