[Solved] Matrix Creator and Google Assistant

vizir · February 21, 2018, 2:31am

Unable to get the microphones to work with Google Assistant SDK.

The error I am getting:
shell$ google-assistant-demo --device_model_id XXXX
ALSA lib pcm.c:8382:(snd_pcm_set_params) Sample format not available for CAPTURE: Invalid argument
device_model_id: XXXX
device_id: XXXXXX

ON_MUTED_CHANGED:
  {'is_muted': False}
[4352:4374:ERROR:audio_input_processor.cc(748)] Input error
ON_START_FINISHED
[4352:4382:ERROR:audio_input_processor.cc(748)] Input error

my ~/.asoundrc file looks like this:
pcm.!default {
type asym
capture.pcm “mic”
playback.pcm “speaker”
}
pcm.mic {
type file
file “/tmp/matrix_micarray_channel_0”
infile “/tmp/matrix_micarray_channel_0”
format “raw”
slave {
pcm “hw:1,0”
}
}
pcm.speaker {
type plug
slave {
pcm “hw:0,0”
channels 2
}
}

the RPi has been configured with the matrix creator snd_dummy as index 1, snd_usb - simple c-media usb speaker as index 0 and the built-in bcmXXXX sound as index 2. hence the pcm values.

speaker-test, arecord and aplay all work fine.

yoelrc88 · February 21, 2018, 4:01pm

Google Assitant SDK uses portaudio to access the mics. You can use instructions here https://github.com/matrix-io/matrixio-kernel-modules to setup the mics.
Let us know how it goes

vizir · February 21, 2018, 7:14pm

the installation went without a hitch, but when i try and run the pyaudio_test.py i get a lot of errors:

$ python pyaudio_test.py
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition 'defaults.bluealsa.device'
ALSA lib conf.c:4528:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:4996:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
ALSA lib pcm_params.c:2162:(snd1_pcm_hw_refine_slave) Slave PCM not usable
Expression 'parameters->channelCount <= maxChans' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 1514
Expression 'ValidateParameters( inputParameters, hostApi, StreamDirection_In )' failed in 'src/hostapi/alsa/pa_linux_alsa.c', line: 2818
Traceback (most recent call last):
  File "pyaudio_test.py", line 17, in <module>
    frames_per_buffer=CHUNK)
  File "/home/pi/env/lib/python3.5/site-packages/pyaudio.py", line 750, in open
    stream = Stream(self, *args, **kwargs)
  File "/home/pi/env/lib/python3.5/site-packages/pyaudio.py", line 441, in __init__
    self._stream = pa.open(**arguments)
OSError: [Errno -9998] Invalid number of channels

none of these pcm definitions are in my /etc/asound.conf or ~/.asoundrc.

google assistant demo still showing:

$ google-assistant-demo --device_model_id XXXX
ALSA lib pcm.c:8382:(snd_pcm_set_params) Sample format not available for CAPTURE: Invalid argument
device_model_id: XXXX
device_id: XXXX

[5273:5295:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[5273:5301:ERROR:audio_input_processor.cc(748)] Input error

I have modified my ~/.asoundrc to:

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
  type hw
  card 1
  device 0
}
pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
    channels 2
  }
}

and aplay -l:

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0

finally, arecord -l:

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7

yoelrc88 · February 21, 2018, 7:19pm

Can you use this asoundrc file instead?

github.com

matrix-io/matrixio-kernel-modules/blob/master/misc/asoundrc


pcm.sc {
    type hw
    card 2
}

pcm.!default
{
  type asym
  playback.pcm {
    type hw
    card 0
    device 0
  }
  capture.pcm {
    type hw
    card 2
    device 0
  }
}

vizir · February 21, 2018, 7:36pm

some progress…

$ google-assistant-demo --device_model_id XXXX
device_model_id: XXXX
device_id: XXXX

[5458:5480:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[5458:5487:ERROR:audio_input_processor.cc(748)] Input error

loom · February 21, 2018, 7:49pm

Hi,

did you add “dtoverlay=matrixio” to the /boot/config.txt and rebooted?

you can check if the modules get loaded with:

lsmod | grep matrix

# should return something like this

matrixio_env            2772  0
industrialio           62929  1 matrixio_env
matrixio_pcm          267079  1
matrixio_codec          5030  1
matrixio_uart           5939  0
snd_soc_core          180471  2 matrixio_pcm,matrixio_codec
matrixio_core           4415  2 matrixio_pcm,matrixio_env
snd_pcm                98501  5 matrixio_pcm,snd_pcm_dmaengine,snd_bcm2835,snd_soc_core,snd_dummy

also:

arecord -l

# should also return

card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

vizir · February 21, 2018, 8:20pm

i had appended matrixio to a previous dtoverlay=… which the matrix software added. fixed that problem.

$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 0: Set [C-Media USB Headphone Set], device 0: USB Audio [USB Audio]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 3: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0

the device shows up as card 3, and trying both card 2 and card 3 in capture.pcm { … } doesn’t work.

loom · February 21, 2018, 8:30pm

i’m not sure, but can you unplug the usb headphone device and test it again, after another reboot. This may help to narrow down the error.

if it is still showing up as card3, switching the card in pcm.sc { … to 3 could also help.

vizir · February 21, 2018, 8:40pm

undid all the stuff I did for changing order and indexes of pcm devices. unplugged usb.

$ aplay -l
**** List of PLAYBACK Hardware Devices ****
card 0: ALSA [bcm2835 ALSA], device 0: bcm2835 ALSA [bcm2835 ALSA]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 0: ALSA [bcm2835 ALSA], device 1: bcm2835 ALSA [bcm2835 IEC958/HDMI]
  Subdevices: 1/1
  Subdevice #0: subdevice #0
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7



$ arecord -l
**** List of CAPTURE Hardware Devices ****
card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
  Subdevices: 8/8
  Subdevice #0: subdevice #0
  Subdevice #1: subdevice #1
  Subdevice #2: subdevice #2
  Subdevice #3: subdevice #3
  Subdevice #4: subdevice #4
  Subdevice #5: subdevice #5
  Subdevice #6: subdevice #6
  Subdevice #7: subdevice #7
card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
  Subdevices: 1/1
  Subdevice #0: subdevice #0



$ google-assistant-demo --device_model_id XXXX
device_model_id: XXXX
device_id: XXXX

[1038:1060:ERROR:audio_input_processor.cc(748)] Input error
ON_MUTED_CHANGED:
  {'is_muted': False}
ON_START_FINISHED
[1038:1066:ERROR:audio_input_processor.cc(748)] Input error


$ cat .asoundrc
pcm.sc {
    type hw
    card 2
}

pcm.!default
{
  type asym
  playback.pcm {
    type hw
    card 0
    device 0
  }
  capture.pcm {
    type hw
    card 2
    device 0
  }
}

loom · February 21, 2018, 8:51pm

ok i will test this on my device, if i’ll get the same error. But your settings look good now.

kevindpatino · February 21, 2018, 8:58pm

Hello @loom and @vizir

Could you remove the /etc/asound.conf . It is the ALSA configuration for HAL support. So if you will work with the kernel modules you don’t need this file.

Regards,

Kevin Patino

vizir · February 21, 2018, 9:23pm

removed. same thing.

loom · February 21, 2018, 9:25pm

the “googlesamples-assistant-audiotest” and " googlesamples-assistant-pushtotalk" is working the hotword not, i’ll look into the python code.

vizir · February 21, 2018, 9:42pm

googlesamples-assistant-audiotest runs, but i cannot verify if it’s actually recording from the creator mic’s since I don’t have any other audio out except the usb speaker which is unplugged.

loom · February 21, 2018, 9:47pm

but you can run the googlesamples-assistant-pushtotalk it’s outputting text after pressed enter. mine needs permission in some permission in google activity controls, to function.

vizir · February 21, 2018, 10:11pm

this is the output

$ googlesamples-assistant-audiotest test
INFO:root:Starting audio test.
INFO:root:Recording samples.
INFO:root:Finished recording.
INFO:root:Playing back samples.
INFO:root:Finished playback.
INFO:root:audio test completed.

loom · February 21, 2018, 10:28pm

hmm not audiotest, the pushtotalk demo.

I just got problems with my SD card, got some kernel crashes. I’ll try again tomorrow, as soon as I have a new one.

vizir · February 21, 2018, 10:32pm

deleted previous reply since I was trying something else.

here’s what push to talk demo says:

$ googlesamples-assistant-pushtotalk --device-model-id trident
INFO:root:Connecting to embeddedassistant.googleapis.com
WARNING:root:Device config not found: [Errno 2] No such file or directory: '/home/pi/.config/googlesamples-assistant/device_config.json'
INFO:root:Registering device
ERROR:root:Option --project-id required when registering a device instance.

looks like I need to register some more info about this device before I can test this.

yoelrc88 · February 22, 2018, 3:29pm

Can you try this asoundrc file:

pcm.!default {
  type asym
  capture.pcm "mic"
  playback.pcm "speaker"
}
pcm.mic {
   type softvol
   slave {
     pcm "array"
   }
   control {
     name "MicArray Master"
        card 0
   }
}
pcm.speaker {
  type plug
  slave {
    pcm "hw:0,0"
  }
}
pcm.array {
  type plug
  slave {
    pcm "hw:2,0"
  }
}

See it working

If can’t play the video use this link.

Thanks

loom · February 22, 2018, 4:14pm

Yeah it works. I also looked at the softvol option yesterday, but the alsa stuff is partly really weird, at least for me.

Thank you very much.