Voice microphone unexpected output

luca · June 5, 2020, 9:17am

hi, I am a newcomer getting started with my brand new Voice board.
tring to get up at speed with microphone usage i was following the Microphone Array Record to File example at https://matrix-io.github.io/matrix-documentation/matrix-hal/examples/microphone/.

i run the script while singing at the microphone and i got the 8 audio tracks for all the phisical channels plus the beamformed ch 8.

however playng the traces and looking at them visually i see unexpected behaviour: channels from 1 to 7 are muted apparently and channel 0 and 8 give me the same weird bursty output that you can see below.

any idea on what is going on?

thank you in advance for any support!
Luca

Figure_1

Samreen · June 8, 2020, 2:13pm

Hi @luca,

Welcome to the community!

Could you try recording using the MATRIX kernel modules as shown here?

Remember to use the -c 8 flag to record all 8 channels.

arecord recording.wav -f S16_LE -r 16000 -d 5 -c 8
aplay recording.wav

This should result in one .wav file with 8 channels if you open it with some audio analyzing software - one for each mic. Let me know the behavior of the waveforms in this case as well so I can gauge better whether it could be a code issue or if there could be a different cause.

Best,
Samreen

luca · June 10, 2020, 9:25pm

Hi Samreen,
thank you for the feedback, unfortunatelly following the example you provided me, and after running the arecord command i got this result:

ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: main:828: audio open error: No such file or directory

I’m trying to solve it but if you have any suggestion would be great! (i forgot to stress that i am new also to raspberry, my background is more circuit level).

ciao
Luca

Samreen · June 12, 2020, 11:40pm

Luca,

Hmm interesting. Can you send me the output of the following command in terminal?

arecord -l

Best,
Samreen

luca · June 14, 2020, 9:14pm

hi Samreen, here it is:
pi@raspberrypi:~ $ arecord -l
**** List of CAPTURE Hardware Devices ****
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (0): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (1): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (2): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (3): No such file or directory

I’m using a raspberry pi4 model B.
hope that could help, I am still failing to have the example running
thank you for your support!

Luca

little-stone · June 18, 2020, 8:31am

Hi, I am wondering if you have found any solutions to your problems?

I was playing with my brand new Matrix Voice, and went into a similar problem myself.

I tried this command:

arecord recording.wav -f S16_LE -r 16000 -d 5 -c 8
aplay recording.wav

and visualized the recording.wav using Audacity. The result looks fine: all 8 channels have decent data.

But when I tried the example of recording from all channels in C++ (https://matrix-io.github.io/matrix-documentation/matrix-hal/examples/microphone/, the same one that the poster tested), the result was similar: only channel 0 and channel 8 (beamformed) have results, but they were distorted, and all other channels were empty.

So I guess there is something wrong with the code?

little-stone · June 18, 2020, 8:51am

One more thing, once the code starts running, even if the program finishes recording (say for 5 seconds, as configured by default in the example’s program) and the program stops running, the audio devices would still be occupied. If I run:

arecord recording.wav -f S16_LE -r 16000 -d 5 -c 8

after the program runs, the terminal would complain:

arecord: main:828: audio open error: Device or resource busy

Do you have any idea what would be the cause of these problems? @Samreen

luca · June 18, 2020, 1:36pm

@little-stone
Hi, I am still struggling to fix it!

i will update the post if i could see any improvement

little-stone · June 19, 2020, 10:41am

@luca
Hi, thank you for letting me know.

A little update from my side:
while I am still unable to make the C++ example work, the recording example using pyaudio in python 3 actually worked in my test.

https://matrix-io.github.io/matrix-documentation/matrix-lite/py-reference/alsa-mics/

If python is also OK for your project, maybe you can consider using this example to get some quick results.

Hope this hopes!

Samreen · June 19, 2020, 5:57pm

@little-stone,

Welcome to the community!

It looks like ALSA is working for you which is great. As for the C++, let me test this and get back to you. The code hasn’t changed so I’m not sure what could be causing this but I will check.

As for the device or resource busy issue, are you still having it? That should not happen if the recording has ended, unless some other program is using the mics.

If you provided me with more information as to what your environment consists of, I could try to replicate the error. What is your Pi make/model, power supply voltage and amperage, what RPi OS version, linux version (check with uname -a in terminal), and what packages have you installed since first boot (you can check MATRIX packages with dpkg -l | grep matrix in terminal)?

@luca,

Apologies for the delayed response.

I have not seen ALSA errors like yours before, it almost seems like ALSA is corrupted on your device because arecord -l should just list the RPi mics regardless of MATRIX device configurations.

Could you also provide me with more information about your dev environment so I can try to replicate?

You said you are using a Pi 4 so what is your power supply voltage and amperage, what RPi OS version, linux version (check with uname -a in terminal), and what packages have you installed since first boot (you can check MATRIX packages with dpkg -l | grep matrix in terminal)?

Additionally, could you show me the output of your sound configuration file? The command is

sudo cat /etc/asound.conf

If you are able, I would recommend a fresh installation of Raspbian OS on your microSD for the Pi 4 with installations from here (the kernel module link I had sent before) to see if your mics work.

Best,
Samreen

luca · June 22, 2020, 5:40pm

@Samreen, thank you for the support! here some feedback:

I am using a pi4 Model B kit from labists with 5.1V 3A PSU.
Linux raspberrypi 4.19.118-v7l+ #1311 SMP Mon Apr 27 14:26:42 BST 2020 armv7l GNU/Linux
about the packages:
ii libmatrixio-creator-hal:armhf 0.3.8 armhf Hardware Abstraction Layer for MATRIX.
ii libmatrixio-creator-hal-dev 0.3.8 armhf Hardware Abstraction Layer for MATRIX (headers)
ii libswresample3:armhf 7:4.1.4-1+rpt7~deb10u1 armhf FFmpeg library for audio resampling, rematrixing etc. - runtime files
ii matrixio-creator-init 0.4.17 armhf Install scripts that can program the MATRIX Creator FPGA and SAM3 IMU.
ii matrixio-kernel-modules 0.2.3 all Source DKMS for MATRIXIO kernel modules.
ii matrixio-openocd 1.0.1.002 armhf Open on-chip JTAG debug solution for ARM and MIPS systems
ii matrixio-xc3sprog 1.1.1.003 armhf Spartan3, XCF and CPLD JTAG programmer and other utilities.

luca · June 22, 2020, 9:06pm

Now i tried following your instructions:

fresh installation of Raspbian
full installation of Matrix Voice and HALL
following basic example Compiling Programs

and the result is:

INFO: [/dev/spidev0.0] was opened
MATRIX device has not been detected

then I started tring the troubleshooting procedure but got this unexpected message

pi@raspberrypi:~ $ echo 26 > /sys/class/gpio/export 2>/dev/null
pi@raspberrypi:~ $ echo out > /sys/class/gpio/gpio26/direction  
pi@raspberrypi:~ $ echo 1 > /sys/class/gpio/gpio26/value  
pi@raspberrypi:~ $ echo 0 > /sys/class/gpio/gpio26/value  
pi@raspberrypi:~ $ echo 1 > /sys/class/gpio/gpio26/value
pi@raspberrypi:~ $ xc3sprog -c matrix_voice blob/bscan_spi_s6lx9_ftg256.bit
XC3SPROG (c) 2004-2011 xc3sprog project $Rev: 774 $ OS: Linux
Free software: If you contribute nothing, expect nothing!
Feedback on success/failure/enhancement requests:
	http://sourceforge.net/mail/?group_id=170565 
Check Sourceforge for updates:
	http://sourceforge.net/projects/xc3sprog/develop

No JTAG Chain found

little-stone · June 23, 2020, 10:51am

Hi @Samreen
Today I tried to replicate my result the other day using ALSA in Python, and unfortunately I went to a similar problem to Luca’s.

This time, when I run the python code, it tells me:

Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘defaults.bluealsa.device’
ALSA lib conf.c:4568:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5036:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib confmisc.c:1281:(snd_func_refer) Unable to find definition ‘defaults.bluealsa.device’
ALSA lib conf.c:4568:(_snd_config_evaluate) function snd_func_refer returned error: No such file or directory
ALSA lib conf.c:5036:(snd_config_expand) Args evaluate error: No such file or directory
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM bluealsa
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
ALSA lib pcm_dsnoop.c:575:(snd_pcm_dsnoop_open) The dsnoop plugin supports only capture stream
Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM input
Expression ‘alsa_snd_pcm_hw_params_set_period_size_near( pcm, hwParams, &alsaPeriodFrames, &dir )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 924
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
Expression ‘parameters->channelCount <= maxChans’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 1514
Expression ‘ValidateParameters( inputParameters, hostApi, StreamDirection_In )’ failed in ‘src/hostapi/alsa/pa_linux_alsa.c’, line: 2818
Traceback (most recent call last):
File “pyaudio_recording_test.py”, line 18, in
frames_per_buffer=CHUNK)
File “/usr/local/lib/python3.7/dist-packages/pyaudio.py”, line 750, in open
stream = Stream(self, *args, **kwargs)
File “/usr/local/lib/python3.7/dist-packages/pyaudio.py”, line 441, in init
self._stream = pa.open(**arguments)
OSError: [Errno -9998] Invalid number of channels

“arecord -f S16_LE -r 16000 -c 8 out.wav” also failed with the following messages:

ALSA lib pcm.c:2565:(snd_pcm_open_noupdate) Unknown PCM input
arecord: main:828: audio open error: No such file or directory

And following are the outputs from

arecord -l
**** List of CAPTURE Hardware Devices ****
card 2: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
Subdevices: 8/8
Subdevice #0: subdevice #0
Subdevice #1: subdevice #1
Subdevice #2: subdevice #2
Subdevice #3: subdevice #3
Subdevice #4: subdevice #4
Subdevice #5: subdevice #5
Subdevice #6: subdevice #6
Subdevice #7: subdevice #7
card 3: MATRIXIOSOUND [MATRIXIO-SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
Subdevices: 1/1
Subdevice #0: subdevice #0

I wonder if there is something wrong that could cause the device to be temporarily unavailable to be read by the OS? Do you have any idea how to deal with this?

Samreen · June 23, 2020, 10:16pm

@little-stone,

It seems your problems are arising because your MATRIX device is 3,0 (ALSA card 3, device 0) as shown in your arecord -l output, rather than 2,0 as noted in the /etc/asound.conf file.

Try the following command to record and let me know if it works.

arecord -f S16_LE -D hw:3,0 -r 16000 -c 8 out.wav

If so, it may help to replace all instances of 2,0 to 3,0 in /etc/asound.conf.

Best,
Samreen

Samreen · June 23, 2020, 10:28pm

@luca,

That is odd.

Could you send me the output of the following command to see if the FPGA can be detected at all?

cd /usr/share/matrixlabs
sudo xc3sprog -j -c sysfsgpio_voice

Additionally, can you send me pictures of your setup? I just want to make sure everything is connected properly and that there are no visual defects.

Best,
Samreen

little-stone · June 24, 2020, 10:27am

Hi @Samreen

I tried setting the /etc/asound.conf file as you mentioned today, and the recording command worked.
The python script also worked once I set the stream in pyaudio to open the correct device.

Thank you for your help.

By the way, do you know how can I get the beamformed output from python (instead of implementing the “delay and sum” by myself)? Is there an example code like the one in the C++ recording example that provides this function?

Best,
little-stone

luca · June 25, 2020, 8:01pm

hi @Samreen,

i do not have such a directory as /usr/share/matrixio
but I do have a /usr/share/matrixlabs, then running this command i have this result:

pi@raspberrypi:~ $ cd /usr/share/matrixlabs/
pi@raspberrypi:/usr/share/matrixlabs $ sudo xc3sprog -j -c sysfsgpio_voice
XC3SPROG (c) 2004-2011 xc3sprog project $Rev: 774 $ OS: Linux
Free software: If you contribute nothing, expect nothing!
Feedback on success/failure/enhancement requests:
	http://sourceforge.net/mail/?group_id=170565 
Check Sourceforge for updates:
	http://sourceforge.net/projects/xc3sprog/develop

JTAG loc.:   0  IDCODE: 0x24001093  Desc:                        XC6SLX9 Rev: C  IR length:  6

for the photo here they are, but my setup is defenetivelly nothing special…

the only sign i can get from the Voice board is the blue led on the back of the Voice…

any suggestion is highly appreciated!!!

Samreen · June 25, 2020, 8:03pm

@little-stone,

Unfortunately, we only have beamforming in C++ and not yet in the kernel implementation. I will test the C++ and get back to you ASAP.

Best,
Samreen

Samreen · June 26, 2020, 10:24pm

@luca,

Your FPGA can be detected as shown by your output so this is good news!

I would recommend installing the MATRIX kernel modules

sudo apt install matrixio-kernel-modules

A reboot

sudo reboot

Then checking if the MATRIX mics show up

arecord -l

Let me know how it goes!

Best,
Samreen

luca · June 29, 2020, 7:52pm

@Samreen,
i am a bit confused, i think that i already had the modules installed, infact this is the result of the command you suggest:

pi@raspberrypi:~ $ sudo apt install matrixio-kernel-modules
Reading package lists... Done
Building dependency tree       
Reading state information... Done
matrixio-kernel-modules is already the newest version (0.2.3).
The following packages were automatically installed and are no longer required:
  libmicrodns0 rpi-eeprom-images
Use 'sudo apt autoremove' to remove them.
0 upgraded, 0 newly installed, 0 to remove and 3 not upgraded.

while this is the output of the arecord command:

pi@raspberrypi:~ $ arecord -l
**** List of CAPTURE Hardware Devices ****
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (0): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (1): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (2): No such file or directory
ALSA lib conf.c:823:(get_char_skip_comments) Cannot access file !DOCTYPE html
ALSA lib conf.c:1887:(_snd_config_load_with_include) _toplevel_:6:15:No such file or directory
ALSA lib conf.c:3650:(config_file_open) /etc/asound.conf may be old or corrupted: consider to remove or fix it
ALSA lib conf.c:3572:(snd_config_hooks_call) function snd_config_hook_load returned error: No such file or directory
ALSA lib conf.c:4026:(snd_config_update_r) hooks failed, removing configuration
arecord: device_list:281: control open (3): No such file or directory

and still the device is undetected!

pi@raspberrypi:~/matrix-hal-examples/build/everloop $ ./everloop_green 
INFO: [/dev/matrixio_regmap] was opened
MATRIX device has not been detected

i fear we are not making any progress…