[Solved] Matrix Creator and Google Assistant

Unable to get the microphones to work with Google Assistant SDK.

The error I am getting:
shell$ google-assistant-demo --device_model_id XXXX
ALSA lib pcm.c:8382:(snd_pcm_set_params) Sample format not available for CAPTURE: Invalid argument
device_model_id: XXXX
device_id: XXXXXX

ON_MUTED_CHANGED:
  {'is_muted': False}
[4352:4374:ERROR:audio_input_processor.cc(748)] Input error
ON_START_FINISHED
[4352:4382:ERROR:audio_input_processor.cc(748)] Input error

my ~/.asoundrc file looks like this:
pcm.!default {
type asym
capture.pcm “mic”
playback.pcm “speaker”
}
pcm.mic {
type file
file “/tmp/matrix_micarray_channel_0”
infile “/tmp/matrix_micarray_channel_0”
format “raw”
slave {
pcm “hw:1,0”
}
}
pcm.speaker {
type plug
slave {
pcm “hw:0,0”
channels 2
}
}

the RPi has been configured with the matrix creator snd_dummy as index 1, snd_usb - simple c-media usb speaker as index 0 and the built-in bcmXXXX sound as index 2. hence the pcm values.

speaker-test, arecord and aplay all work fine.

Google Assitant SDK uses portaudio to access the mics. You can use instructions here https://github.com/matrix-io/matrixio-kernel-modules to setup the mics.
Let us know how it goes

the installation went without a hitch, but when i try and run the pyaudio_test.py i get a lot of errors:

$ python pyaudio_test.py
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1514
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2818
Traceback (most recent call last):
  File "pyaudio_test.py", line 17, in <module>
    frames_per_buffer=CHUNK)
  File "/home/pi/env/lib/python3.5/site-packages/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "/home/pi/env/lib/python3.5/site-packages/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
OSError: [Errno -9998] Invalid number of channels

none of these pcm definitions are in my /etc/asound.conf or ~/.asoundrc.

google assistant demo still showing:

$ google-assistant-demo --device_model_id XXXX
ALSA lib pcm.c:8382:(snd_pcm_set_params) Sample format not available for CAPTURE: Invalid argument
device_model_id: XXXX
device_id: XXXX

[5273:5295:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[5273:5301:ERROR:audio_input_processor.cc(748)] Input error

I have modified my ~/.asoundrc to:

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
  type hw
  card 1
  device 0
}
pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
    channels 2
  }
}

and aplay -l:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

finally, arecord -l:

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7

Can you use this asoundrc file instead?

some progress…

$ google-assistant-demo --device_model_id XXXX
device_model_id: XXXX
device_id: XXXX

[5458:5480:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[5458:5487:ERROR:audio_input_processor.cc(748)] Input error

Hi,

did you add “dtoverlay=matrixio” to the /boot/config.txt and rebooted?

you can check if the modules get loaded with:

lsmod | grep matrix

# should return something like this

matrixio_env            2772  0
industrialio           62929  1 matrixio_env
matrixio_pcm          267079  1
matrixio_codec          5030  1
matrixio_uart           5939  0
snd_soc_core          180471  2 matrixio_pcm,matrixio_codec
matrixio_core           4415  2 matrixio_pcm,matrixio_env
snd_pcm                98501  5 matrixio_pcm,snd_pcm_dmaengine,snd_bcm2835,snd_soc_core,snd_dummy

also:

arecord -l

# should also return

card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

i had appended matrixio to a previous dtoverlay=… which the matrix software added. fixed that problem.

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 3: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

the device shows up as card 3, and trying both card 2 and card 3 in capture.pcm { … } doesn’t work.

i’m not sure, but can you unplug the usb headphone device and test it again, after another reboot. This may help to narrow down the error.

if it is still showing up as card3, switching the card in pcm.sc { … to 3 could also help.

undid all the stuff I did for changing order and indexes of pcm devices. unplugged usb.

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7



$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0



$ google-assistant-demo --device_model_id XXXX
device_model_id: XXXX
device_id: XXXX

[1038:1060:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[1038:1066:ERROR:audio_input_processor.cc(748)] Input error


$ cat .asoundrc
pcm.sc {
    type hw
    card 2
}

pcm.!default
{
  type asym
  playback.pcm {
    type hw
    card 0
    device 0
  }
  capture.pcm {
    type hw
    card 2
    device 0
  }
}

ok i will test this on my device, if i’ll get the same error. But your settings look good now.

Hello @loom and @vizir

Could you remove the /etc/asound.conf . It is the ALSA configuration for HAL support. So if you will work with the kernel modules you don’t need this file.

Regards,

Kevin Patino

removed. same thing.

the “googlesamples-assistant-audiotest” and " googlesamples-assistant-pushtotalk" is working the hotword not, i’ll look into the python code.

googlesamples-assistant-audiotest runs, but i cannot verify if it’s actually recording from the creator mic’s since I don’t have any other audio out except the usb speaker which is unplugged.

but you can run the googlesamples-assistant-pushtotalk it’s outputting text after pressed enter. mine needs permission in some permission in google activity controls, to function.

this is the output

$ googlesamples-assistant-audiotest test
INFO:root:Starting audio test.
INFO:root:Recording samples.
INFO:root:Finished recording.
INFO:root:Playing back samples.
INFO:root:Finished playback.
INFO:root:audio test completed.

hmm not audiotest, the pushtotalk demo.

I just got problems with my SD card, got some kernel crashes. I’ll try again tomorrow, as soon as I have a new one. :slight_smile:

deleted previous reply since I was trying something else.

here’s what push to talk demo says:

$ googlesamples-assistant-pushtotalk --device-model-id trident
INFO:root:Connecting to embeddedassistant.googleapis.com
WARNING:root:Device config not found: [Errno 2] No such file or directory: '/home/pi/.config/googlesamples-assistant/device_config.json'
INFO:root:Registering device
ERROR:root:Option --project-id required when registering a device instance.

looks like I need to register some more info about this device before I can test this.

Can you try this asoundrc file:

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
   type softvol
   slave {
     pcm "array"
   }
   control {
     name "MicArray Master"
        card 0
   }
}
pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
  }
}
pcm.array {
  type plug
  slave {
    pcm "hw:2,0"
  }
}

See it working

If can’t play the video use this link.

Thanks

1 Like

Yeah it works. I also looked at the softvol option yesterday, but the alsa stuff is partly really weird, at least for me.

Thank you very much.