Custom Speech Assistant with Python

Help and Troubleshooting
#1

Hello,

I have a Matrix Voice and would like to build a custom offline Speech assistant with Python.

Until now I was able to use the the python Snowboy (https://github.com/Kitt-AI/snowboy) Wrapper after doing these installation steps: Kitt_ai wakeword engine still not working on Matrix VOICE

Or I was able to run the everloop example using these installation steps: https://matrix-io.github.io/matrix-documentation/matrix-hal/getting-started/installation/

Is it possible at the moment to do both, use snowboy and control the leds in Python? How can I do that?

Any help would be nice, Thanks.

#2

Hi, can you please explain in detail how you use the wrapper?

#3

Here is my code for using the wrapper:

from snowboy import snowboydecoder_arecord as snowboydecoder
import requests
import time

lastWake = 0

def isAwake():
    return time.time() - lastWake < 4

def detectedWake():
    global lastWake
    lastWake = time.time()
    print("Awake!")

def lightOn():
    if not isAwake(): return
    requests.post("http://192.168.178.23:8080/api/power/4", json={'turnedOn': 1})
    print("Light on!")

def lightOff():
    if not isAwake(): return
    requests.post("http://192.168.178.23:8080/api/power/4", json={'turnedOn': 0})
    print("Light off!")

def ledsOff():
    if not isAwake(): return
    print("Leds off!")

models = [
    "./snowboy/models/computer.pmdl",
    "./snowboy/models/licht_an.pmdl",
    "./snowboy/models/licht_aus.pmdl"
]
callbacks = [detectedWake, lightOn, lightOff]

detector = snowboydecoder.HotwordDetector(models, sensitivity=0.4)
print('Listening...')

# main loop
detector.start(detected_callback=callbacks, sleep_time=0.01)

detector.terminate()
#4

Thx!! Unfortunately it doesnt solve my issue, But once I have it working, then this example will be a great help, Thanks again,

#5

this is using the acrecord example it should also work without kernel modules, by using the alsa pipes.

#6

You mean this should work after only doing these installation steps: https://matrix-io.github.io/matrix-documentation/matrix-hal/getting-started/installation/ ?

When I try that, I get this error:

Listening...
Exception in thread Thread-1:
Traceback (most recent call last):
  File "/usr/lib/python3.5/threading.py", line 914, in _bootstrap_inner
    self.run()
  File "/usr/lib/python3.5/threading.py", line 862, in run
    self._target(*self._args, **self._kwargs)
  File "/home/pi/VoiceRecognition/snowboy/snowboydecoder_arecord.py", line 100, in record_proc
    wav = wave.open(process.stdout, 'rb')
  File "/usr/lib/python3.5/wave.py", line 499, in open
    return Wave_read(f)
  File "/usr/lib/python3.5/wave.py", line 163, in __init__
    self.initfp(f)
  File "/usr/lib/python3.5/wave.py", line 128, in initfp
    self._file = Chunk(file, bigendian = 0)
  File "/usr/lib/python3.5/chunk.py", line 63, in __init__
    raise EOFError
EOFError

When I change the first line to

from snowboy import snowboydecoder as snowboydecoder

I get this error:

Listening...
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.front
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.rear
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.center_lfe
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.side
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround21
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround40
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround41
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround50
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround51
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.surround71
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.iec958
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.hdmi
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.modem
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm.c:2495:(snd_pcm_open_noupdate) Unknown PCM cards.pcm.phoneline
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
ALSA lib pcm_hw.c:1713:(_snd_pcm_hw_open) Invalid value for card
Cannot connect to server socket err = No such file or directory
Cannot connect to server request channel
jack server is not running or cannot be started
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock
JackShmReadWritePtr::~JackShmReadWritePtr - Init not done for -1, skipping unlock

I changed my config file like you described in this post: [Solved] Kitt_ai wakeword engine still not working on Matrix VOICE - #19 by loom

#7

no the normal snowboydecoder is not working only the arecord one. but you need the malos service as it is generating the alsa pipes. than you can also use the malos service to commuincate with the everloop on the voice.

maybe you also need to modify the arecord command to use the correct recording device (e.g. micchannel0:

 def record_proc(self):
        CHUNK = 2048
        RECORD_RATE = 16000
        cmd = 'arecord -q -r %d -f S16_LE' % RECORD_RATE
        process = subprocess.Popen(cmd.split(' '),
                                   stdout = subprocess.PIPE,
                                   stderr = subprocess.PIPE)
```
1 Like
#8

Thanks. I just had to install the malos service and set my alsa config back to default.

Now I can run my python script for wakeword detection as well as the python everloop example (https://github.com/matrix-io/matrix-creator-malos/blob/master/src/python_test/test_everloop.py)