Matrix Voice, Node-RED and Watson Conversation

#1

I set up my Matrix Voice to record speech via Node-RED running on the Raspberry Pi. ( Its a one line hack to node-red-contrib-micropi to record mic_channel0 instead of USB Audio plughw )

I pass the streaming speech to node-red-contrib-snowboy to listen for a wake word. In my case “Hey Watson”
That wakes up a Node-RED flow that uses Watson Speech to Text, Watson Conversation and Watson Text to Speech to interact with the user.

Raspbian doesn’t make Audio in via the Matrix Voice and Audio out via a USB speaker easy. I got it to work but it seems a bit kludgey.

Lots of fun. If anyone is interested, I’ll post my flows and screenshots over in github and
http://flows.nodered.org

– John Walicki - Watson IoT Developer Advocate

1 Like
#2

Looks interesting, thanks for your post.

#3

Awesome @johnwalicki,

Please, if you can share the screenshots and flows would be nice!

Thanks

#4

Would you write a tutorial how to do this? I’m very interested!

#5

Hey, what is the “one line hack to node-red-contrib-micropi”

This sounds pretty interesting

#6

The node-red-contrib-micropi node expects to record USB Audio via

arecord -D plughw:1,0

Matrix Voice outputs audio on mic_channel0.
You can test with

$ /usr/bin/arecord -d 5 --device=mic_channel0 -r 16000 -c 1 -f S16_LE /home/pi/test.wav

I had installed node-red-contrib-micropi into /usr/lib/node_modules, so as root, edit
/usr/lib/node_modules/node-red-contrib-micropi/micropi/nodes/micropi/lib/mic.js
and change

_device = options.device || ‘plughw:1,0’;

into

_device = options.device || ‘mic_channel0’;

– John Walicki

#7

I will write up a full tutorial with step by step instructions after Think (IBM’s big shindig in Las Vegas)

For now, I’ve uploaded the Node-RED flow examples and some screenshots to a github repo. I promise to circle back and document my experiments in two weeks.

2 Likes
#8

@johnwalicki great work!!

But i’m running into trouble…
Fail to open input file "/home/pi/snowboy/rpi-arm-raspbian-8.0-1.1.1/resources/common.res"

can u help ?

Is it ok to use this file?
pi@raspberrypi:~/node_modules/snowboy/resources/common.res

#9

I followed the first part of these instructions to set up Kitt.ai Snowboy wake words on my Raspberry Pi

#10

Thanks John,
If we can combine that wanleg/snowboyPi piece with this for the Matrix microphone hardware (https://www.hackster.io/matrix-labs/matrix-voice-and-matrix-creator-running-google-assistant-e9751e), then we might have a winner imho.

#11

If did fresh setup and writing down everything, malos node red etc. Will write a howto on sunday evening.

Im about to do couching and tomorrow i will have to do social beer drinking with the neighbours ^^.

Stay Tuned :slight_smile:

1 Like
#12

Priorities man, priorities :wink:

1 Like
#14

1m
you got me :joy:

First steps to get it working.

download a fresh image and flash it.

Update the system

sudo apt update && sudo apt upgrade

Install node red and npm by this script.

bash <(curl -sL https://raw.githubusercontent.com/node-red/raspbian-deb-package/master/resources/update-nodejs-and-nodered)

Credits to bitluni.net and his node red tutorial.

Let node red start on boot

sudo systemctl enable nodered.service

Perform a reboot and look if node-red starts

reboot sudo init 6

http://your_rpi_ip:1880

Install some missing dependencies

sudo apt-get install libmagic-dev libatlas-base-dev

Install dependencies for snowboy

npm install nan

Install snowboy

npm install snowboy

Setup the matrix repository

curl https://apt.matrix.one/doc/apt-key.gpg | sudo apt-key add -

echo "deb https://apt.matrix.one/raspbian $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list

Update the apt cache

sudo apt update

Install MALOS

sudo apt-get install matrixio-malos

followed by a reboot.

reboot sudo init 6

Now install missing node red packages. I did that over the Node-Red interface Manage palette.

node-red-dashboard, node-red-contrib-micropi, node-red-contrib-snowboy, node-red-node-weather-underground, node-red-node-watson

Perform a reboot to load all modules correctly.

reboot sudo init 6

Start node red over the browser and insert the flow’s from @johnwalicki

Allready tested:

arecord with Matrix Voice

Simple Speech Dashboard

Did create allready my own wakeword.pmdl and do further work on sunday.

Hope this helps someone. If not feel free to ask or improve this post.

NOTE

If you do it that way like i understand from @johnwalicki the file to edit the microphone is not under

/usr/lib/node_modules/node-red-contrib-micropi/micropi/nodes/micropi/lib/mic.js

you can find the file under

/home/pi/.node-red/node_modules/node-red-contrib-micropi/micropi/nodes/micropi/lib/mic.js

But not tested the corresponding flow yet. :beers:

#15

My next steps are:

Testing Watson, testing wakeword, testing own wakeword, setting up modipy server and control it over node-red-contrib-mopidy :slight_smile:

Crossing fingers.

#16

Great stuff, this really helps me, though my use case is to get a custom wake word running with the Matrix Voice, HAL and the kernel modules (Malos breaks that). No need for Node Red yet, but will come. I dont want to hijack this thread, but it gave me some good pointers, also for my usage.
Good work bro, now you deserve a beer :stuck_out_tongue_winking_eye:

#17

Will do that now. :smiley:

#18

I was able to get node-red-contrib-micropi working with the kernel modules by editing the following…alsamixer shows the matrix voices as device 2

/home/pi/.node-red/node_modules/node-red-contrib-micropi/micropi/nodes/micropi/lib/mic.js
_device = options.device || ‘plughw:2,0’;

1 Like