[Solved] Reseting Matrix Voice ESP32 software in order to run Alexa demo?

Sandman · January 24, 2018, 6:28pm

Hi,

I received my voice a few days ago, and took it for its first spin with the instructions found from here: https://www.hackster.io/matrix-labs/get-started-w-esp32-on-the-matrix-voice-d01e0d
That mic energy demo worked fine, but now I’d like to try the Alexa demo.

I got the AVS sample stack up and running, but matrix voice does not seem to respond to my voice commands, and I’m suspecting its because of the mic energy demo? So… maybe I need to reset the software?
I installed matrix os but to be honest I’m not entirely sure what to do next.

yoelrc88 · January 24, 2018, 7:04pm

Regarding the error with Alexa demo. Can you share the output of the 3 services (auth app, java client, wakeword) you need to run?

spacemishka · January 24, 2018, 7:58pm

@Sandman

How did you get ESP-IDF development framework to work? The site itself provides the package not for ARM just for x86/x64 only,…

yoelrc88 · January 24, 2018, 9:55pm

@spacemishka,

You are right, it is just for x86/x64. In our MATRIX Voice ESP32 guide, all the development should happen in a x86/x64 computer. After building the app, the deployment script copies the binaries to the Raspberry Pi and then we program the ESP32 using UART. The steps show you have to do it.

Sandman · January 25, 2018, 2:43pm

@yoelrc88

The outputs are as follows:

companion:

$ npm start

> alexa-voice-service-sample-companion-service@1.0.0 start /home/pi/matrix/alexa-avs-sample-app/samples/companionService
> node ./bin/www

This node service needs to be running to store token information memory and vend them for the AVS app.

Listening on port 3000
Successfully retrieved registration code for my_device / 123456789

javaClient

$ mvn exec:exec
[INFO] Scanning for projects...
[INFO]
[INFO] ------------------------------------------------------------------------
[INFO] Building Alexa Voice Service Sample Java Client 20160207.7
[INFO] ------------------------------------------------------------------------
[INFO]
[INFO] --- exec-maven-plugin:1.2.1:exec (default-cli) @ sample-java-client ---
Device: my_device, DSN: 123456789
Alexa Voice Service - v20160207.7
LOGIN TO REGISTER/AUTHENTICATE YOUR DEVICE: Please register your device by visiting the following URL in a web browser and follow the instructions:
https://localhost:3000/provision/8298c90376dd1f58f099eea5Continue only once you've authenticated with AVS.

And after logging in with my browser, I get redirected back to localhost with a message “device tokens ready”

wakeWord

$ ./wakeWordAgent -e sensory
INFO:main: Starting Wake Word Agent
INFO:WakeWordAgent: State set to IDLE(2)
INFO:Initializing Sensory library | library name: TrulyHandsfree | library version: 5.0.0-beta.10.2 | model file: ../ext/resources/spot-alexa-rpi.snsr
WARNING:Library expires on: License expires on 28 Apr 2018 00:00:00 GMT
INFO:SensoryWakeWordEngine: mainLoop thread started
INFO:WakeWordIPCSocket::mainLoop thread started
INFO:WakeWordIPCSocket: init socket on port:5123
INFO:WakeWordAgent: thread started
INFO:===> Connected to AVS client <===

After all this, I can shout “Alexa” at my Voice as much as I like, but it just keeps pulsating with red leds from the mic energy thingy I installed previously.

I’m running raspbian Jessie.

Edit: I just realised that the javaClient should actually open some sort of a GUI. It does not. I’m connected to my Pi with real vnc (headless mode).

Sandman · January 25, 2018, 3:42pm

So my first problem was that I did not actually get the java client up and running properly using VNC in headless mode. I booted my raspberry to the GUI and connected to that with VNC and got the java client up and running.

If I use the “push-to-talk” button from the GUI, I can hear the notification sound that prompts me to start speaking and after a few seconds I hear another sound which I guess indicates that my command has been processed? No matter what I say or do, I get no response from alexa. I only get stuff in the wakeWord log if I press the push-to-talk -button. Here are some new logs:

companion

$ npm start

> alexa-voice-service-sample-companion-service@1.0.0 start /home/pi/matrix/alexa-avs-sample-app/samples/companionService
> node ./bin/www

This node service needs to be running to store token information memory and vend them for the AVS app.

Listening on port 3000
Successfully retrieved registration code for my_device / 123456789
Successfully retrieved access token for session id: ca264da0-2006-478f-8c7c-ab53d8d46976

javaClient

 $ mvn exec:exec
[INFO] Scanning for projects...
[INFO]                                                                         
[INFO] ------------------------------------------------------------------------
[INFO] Building Alexa Voice Service Sample Java Client 20160207.7
[INFO] ------------------------------------------------------------------------
[INFO] 
[INFO] --- exec-maven-plugin:1.2.1:exec (default-cli) @ sample-java-client ---

wakeWord

$ ./wakeWordAgent -e sensory
INFO:main: Starting Wake Word Agent
INFO:WakeWordAgent: State set to IDLE(2)
INFO:Initializing Sensory library | library name: TrulyHandsfree | library version: 5.0.0-beta.10.2 | model file: ../ext/resources/spot-alexa-rpi.snsr
WARNING:Library expires on: License expires on 28 Apr 2018 00:00:00 GMT
INFO:SensoryWakeWordEngine: mainLoop thread started
INFO:WakeWordIPCSocket::mainLoop thread started
INFO:WakeWordIPCSocket: init socket on port:5123
INFO:===> Connected to AVS client <===
INFO:WakeWordAgent: thread started
INFO:WakeWordAgent: IPC Command received:3
INFO:WakeWordAgent: State set to WAKE_WORD_PAUSE_REQUESTED(5)
INFO:SensoryWakeWordEngine: handling pause
INFO: *** THREAD JOINING: Sensory ***
INFO:SensoryWakeWordEngine: mainLoop thread ended
INFO:WakeWordAgent: State set to WAKE_WORD_PAUSED(6)
INFO:WakeWordAgent: IPC Command received:4
INFO:WakeWordAgent: State set to WAKE_WORD_RESUME_REQUESTED(7)
INFO:SensoryWakeWordEngine: handling resume
INFO:SensoryWakeWordEngine: mainLoop thread started
INFO:WakeWordAgent: State set to IDLE(2)
INFO:WakeWordAgent: IPC Command received:3
INFO:WakeWordAgent: State set to WAKE_WORD_PAUSE_REQUESTED(5)
INFO:SensoryWakeWordEngine: handling pause

Sandman · January 27, 2018, 4:20pm

Just gave the micarray_recorder demo (https://matrix-io.github.io/matrix-documentation/matrix-hal/components/microphone/) a go, and it does record my voice. However, there is really much noise and the recording itself is quite low volumen even when I speak 10 cm from the device.

yoelrc88 · January 29, 2018, 5:26pm

@Sandman,

Regarding the micarray_recorder demo is important to note that while running HAL examples you should stop MALOS service to prevent errors in the data.

To kill MALOS service use :
sudo systemctl stop matrixio-malos.service

Also, please check this post about a fix that is coming in the next release related to this.

Let me know if it helps

Sandman · January 29, 2018, 6:35pm

@yoelrc88

Yeah I did stop MALOS before running it, but I don’t mind the quality in that demo at the moment. What I’d like to know is what else can I provide for you that might get me closer to get the Alexa demo working?

unicornmafia · January 30, 2018, 7:22pm

Hello,
I have a similar problem with the matrix voice: esp32-hal demos worked great, but now I can’t get any of the NON-esp32 hal demos working.

Here is what I did (exactly this, and nothing else, so if I missed a setup step, please let me know!):

installed raspi lite
ran the esp32 demos with no problem using this guide exactly:
https://github.com/matrix-io/matrixio_hal_esp32
installed the NON-esp32 demos using this guide exactly:
https://github.com/matrix-io/matrix-creator-hal

When I run the NON-esp32 hal demos I get weird things:

mic_energy was just giving me a red ring, but now it gives floating point exception
everloop_demo gives me red pixels at weirdly random intensities (not like the esp32 one which pulsates)
mic_demo just gives me 0’s and 1’s regardless of what sounds I make (Actually sometimes it gives me floating point errors too)

kevindpatino · January 30, 2018, 10:01pm

Hello @Sandman @unicornmafia @spacemishka

We already released the following packages:

Please run the following steps to upgrade your packages:

# Update packages and install
sudo apt-get update
sudo apt-get upgrade

sudo reboot

Now we fix the issue with ESP32 and Raspberry running at the same time.

Also if you want to reset your ESP32 in /usr/bin/ are located https://github.com/matrix-io/matrix-creator-init/blob/master/voice_esp32_reset. So you could disable the ESP32 with the GPIO25 (RESET pin)

Thank You for your support, let me know how the thing is working.

Regards,

Kevin Patino

unicornmafia · January 30, 2018, 11:04pm

Thanks Kevin,
I ran voice_esp32_reset, did the update/upgrade, git pull’d, and rebuilt/installed hal from source. still same issues.

anything I may have missed? e.g. there used to be a script you had to run on the creator for resetting the fpga or something? do I need to do something like that? I’ve done very little initialization stuff, don’t have malos installed etc.

thanks again for your help!

kevindpatino · January 30, 2018, 11:11pm

Hello @unicornmafia

Did you reboot your device?

You could test the FPGA version with:

pi@voice:~ $ /usr/share/matrixlabs/matrixio-devices/fpga_info 
IDENTIFY = 6032bad2
VERSION = 10004

Thank You

unicornmafia · January 30, 2018, 11:19pm

pi@raspberrypi:~ $ /usr/share/matrixlabs/matrixio-devices/fpga_info 
IDENTIFY = 6032bad2
VERSION = 10004

looks good. I had NOT rebooted. O_o did so, and now the mic_energy seems to react to sounds, but it’s still very strange. it stays on medium intensity, and when I clap my hands, it pulses/strobes a few times brightly, slows down and goes darker, and then goes back to being on medium intensity.

andres.calderon · January 30, 2018, 11:44pm

You can test with:

echo 0 > /sys/class/gpio/gpio25/value

This command will disable the ESP32 module.

–afc

Sandman · February 3, 2018, 10:03am

Thanks. Upgrading and running voice_esp32_reset fixed my issues. The Alexa demo works now.