Matrix Voice, Snips & Home Assistant working together


#21

Armin. Man Thanks for your help. Just at work. I Test it thus evening.


#22

OK, I could reproduce the error. There was a small issue remaing which I forgot to mention

  Remove DUMMY from matrix-mics.conf 
  	sudo nano /etc/modules-load.d/matrix-mics.conf 
    comment out snd-dummy:
			snd_bcm2835
			#snd-dummy

Full steps:

Step 1:
	Install Raspbian stretch
Step 2:
  Install Matrix Kernel Modules: https://github.com/matrix-io/matrixio-kernel-modules
		curl https://apt.matrix.one/doc/apt-key.gpg | sudo apt-key add -
		echo "deb https://apt.matrix.one/raspbian $(lsb_release -sc) main" | sudo tee /etc/apt/sources.list.d/matrixlabs.list
		sudo apt-get update
		sudo apt-get upgrade
		sudo reboot
		sudo apt-get update ->> Not stated in on the webpage but otherwise gives error:
			The following packages have unmet dependencies:
				 matrixio-kernel-modules : Depends: matrixio-creator-init but it is not going to be installed
        Depends: dkms but it is not installable
			E: Unable to correct problems, you have held broken packages.		
		sudo apt install matrixio-kernel-modules
		sudo reboot
		arecord -l:
			**** List of CAPTURE Hardware Devices ****
			card 1: Dummy [Dummy], device 0: Dummy PCM [Dummy PCM]
			  Subdevices: 8/8
			  Subdevice #0: subdevice #0
			  Subdevice #1: subdevice #1
			  Subdevice #2: subdevice #2
			  Subdevice #3: subdevice #3
			  Subdevice #4: subdevice #4
			  Subdevice #5: subdevice #5
			  Subdevice #6: subdevice #6
			  Subdevice #7: subdevice #7
			card 2: SOUND [MATRIXIO SOUND], device 0: matrixio.mic.0 snd-soc-dummy-dai-0 []
			  Subdevices: 1/1
			  Subdevice #0: subdevice #0			
Step 3:
  Install Snips: https://github.com/snipsco/snips-platform-documentation/wiki
  	From: https://github.com/snipsco/snips-platform-documentation/wiki/1.-Setup-the-Snips-Voice-Platform
  	Start with Step 3: Install the Snips Platform
  	sudo apt-get update
  	sudo apt-get install -y dirmngr
  	sudo bash -c  'echo "deb https://raspbian.snips.ai/$(lsb_release -cs) stable main" > /etc/apt/sources.list.d/snips.list'
  	sudo apt-key adv --keyserver pgp.mit.edu --recv-keys D4F50CDCA10A2849
  		OR 
    sudo apt-key adv --keyserver pgp.surfnet.nl --recv-keys D4F50CDCA10A2849
  	sudo apt-get update
		sudo apt-get install -y snips-platform-voice
		- skip configuration part, should be OK already
	Install demo assistant (or your own)
		sudo apt-get install snips-platform-demo
	Install Snips-Watch
		sudo apt-get install snips-watch
		Check status:
			sudo systemctl status "snips-*" =>> all should be running
		Stop Audio Server
			sudo systemctl stop snips-audio-server
		Open another terminal and tail syslog
			sudo tail -f /var/log/syslog
		Start the audio-server from the other terminal
			sudo systemctl start snips-audio-server
	  	You will see these errors: 	  	
				Apr 23 08:43:11 raspberrypi snips-audio-server[3266]: INFO:snips_audio_portaudio: Use input device: "default"
				Apr 23 08:43:16 raspberrypi snips-audio-server[3266]: ERROR:snips_audio_server   : Error: Source stopped. Bailing out.
				Apr 23 08:43:16 raspberrypi systemd[1]: snips-audio-server.service: Main process exited, code=exited, status=1/FAILURE
				Apr 23 08:43:16 raspberrypi systemd[1]: snips-audio-server.service: Unit entered failed state.
				Apr 23 08:43:16 raspberrypi systemd[1]: snips-audio-server.service: Failed with result 'exit-code'.
	IMPORTANT TO FIX THE ISSUE:
	  Remove DUMMY from matrix-mics.conf 
	  	sudo nano /etc/modules-load.d/matrix-mics.conf 
	    comment out snd-dummy:
				snd_bcm2835
				#snd-dummy
    Reboot, after reboot:
		Stop Audio Server
			sudo systemctl stop snips-audio-server
		Open another terminal and tail syslog
			sudo tail -f /var/log/syslog
		Start the audio-server from the other terminal
			sudo systemctl start snips-audio-server
 		In systail terminal you should see:
			Apr 23 08:49:17 raspberrypi snips-audio-server[1040]: INFO:snips_audio_portaudio: Use input device: "MATRIXIO SOUND: - (hw:1,0)"

#23

The repo for controlling the leds is here:


#24

Many Many thanks! Now it works. The hints from Romkabouter was the solution.
Now i test snips :slight_smile:


#25

@Romkabouter:
Great Job. Again: Many thanks!
Your Service is working well.
Had to install “sudo pip install paho-mqtt”

Lights are blue while waiting for hotword an red when listening.


#26

Yeah, I have update the readme of the my repo. Thnx
You can change the red into green by changing:
They should have been green, but I think Matrix has changed the byte order

self.set_everloop_color(0,10,0,0)
into
self.set_everloop_color(10,0,0,0)


#27

Thanks. Changed the color.
Sorry:Off topic for matrix.

But i cannot found where i can activate the sounds like “start_of_input.wav”.
The line “INFO :snips_audio_portaudio : Playing “/usr/share/snips/dialogue/sound/end_of_input.wav” using output “default”, wav spec : WavSpec { channels: 2, sample_rate: 22050, bits_per_sample: 16, sample_format: Int }” apears not in my logs. do you know where i can configure them?


#28

Do you want other sounds or just to hear them?
By default, Snips has no feedback sounds, you can enable then by publishing your site id to the topic: hermes/feedback/sound/toggleOn

See, https://github.com/snipsco/snips-platform-documentation/wiki/6.--Miscellaneous

So, post {"siteId":"default"} to the topic hermes/feedback/sound/toggleOn of your Snips instance
You can do that with any client, including the commandline
If you want to enable them after a restart, I suggest posting it retained :slight_smile:

In my video, around 0.48 you can see that I have a Node Red running, which whenever the hotword is toggled, a more human response is given (yes / waiting)


#29

Hi Romkabouter,

your video looks great and the usage of node-red is great. Would it be possible that you share your node-red flow? Thanks Armin


#30

Hi,Romkabouter
It’s so great what you’ve done.
I’m doing also now a projet with Matrix creator and home asistant.
Could you tell me more about your window section? Which product you’ve used? How you connect with MATRIX? And program step for open and close the window?


#31

Sure:

[{“id”:“f6cf64cd.aaf828”,“type”:“json”,“z”:“f842304.6ee8cd”,“name”:"",“pretty”:false,“x”:315,“y”:55,“wires”:[[“a0648b6c.86ffc8”]]},{“id”:“a0648b6c.86ffc8”,“type”:“function”,“z”:“f842304.6ee8cd”,“name”:“Yes”,“func”:“var p = msg.payload;\np.text =“Yes?”;\nreturn msg;”,“outputs”:1,“noerr”:0,“x”:450,“y”:55,“wires”:[[“e828889f.f5fc68”]]},{“id”:“e828889f.f5fc68”,“type”:“mqtt out”,“z”:“f842304.6ee8cd”,“name”:"",“topic”:“hermes/tts/say”,“qos”:"",“retain”:"",“broker”:“64478e5.440557”,“x”:685,“y”:75,“wires”:[]},{“id”:“a80fd4bc.fb4168”,“type”:“json”,“z”:“f842304.6ee8cd”,“name”:"",“pretty”:false,“x”:315,“y”:115,“wires”:[[“ee6f1c2f.c6373”]]},{“id”:“ee6f1c2f.c6373”,“type”:“function”,“z”:“f842304.6ee8cd”,“name”:“Waiting”,“func”:“var p = msg.payload;\np.text =“Waiting”;\nreturn msg;”,“outputs”:1,“noerr”:0,“x”:460,“y”:115,“wires”:[[“e828889f.f5fc68”]]},{“id”:“1b901fea.f4343”,“type”:“mqtt in”,“z”:“f842304.6ee8cd”,“name”:"",“topic”:“hermes/hotword/toggleOff”,“qos”:“2”,“broker”:“64478e5.440557”,“x”:125,“y”:55,“wires”:[[“f6cf64cd.aaf828”]]},{“id”:“3553178f.b0f8a8”,“type”:“mqtt in”,“z”:“f842304.6ee8cd”,“name”:"",“topic”:“hermes/hotword/toggleOn”,“qos”:“2”,“broker”:“64478e5.440557”,“x”:125,“y”:115,“wires”:[[“a80fd4bc.fb4168”]]},{“id”:“64478e5.440557”,“type”:“mqtt-broker”,“z”:"",“name”:"",“broker”:“192.168.178.172”,“port”:“1883”,“clientid”:"",“usetls”:false,“compatmode”:true,“keepalive”:“60”,“cleansession”:true,“willTopic”:"",“willQos”:“0”,“willPayload”:"",“birthTopic”:"",“birthQos”:“0”,“birthRetain”:“true”,“birthPayload”:""}]

You have to change the IP adress and port of the broker most probably


#32

Here is the part of my Home Assistant config (configuration.yaml):

> snips:
> intent_script:
>   Covers:
>     speech:
>       type: plain
>       text: 'Allright, {{ actionType }}ing the {{ whichCover }} cover.'
>     action:
>       - service_template: cover.{{ actionType | lower }}_cover
>         data_template:
>           entity_id: cover.{{ whichCover | lower }}_cover

If have some covers, mine are Somfy on rfxtx but all will do if you name them.

cover:

  • platform: rfxtrx
    automatic_add: True
    devices:
    071a0000 01 00 00 01:
    name: back cover
    071a0000 00 01 00 02:
    name: left cover
    071a0000 00 01 00 03:
    name: right cover
    071a0000 00 01 00 04:
    name: front cover

Sorry about the messed up markup.

In the Snips console, I have create a Cover intent, with actionType and whichCover as slot types, see the docs of Snips for a tutorials, that is quite easy.
Use that assistant as your Snips assistant and you are good to go


#33

Hi,

don´t know why. But in my case this works not. Activated with “sudo mosquitto_pub -h 192.168.178.59 -p 1883 -t hermes/feedback/sound/toggleOn -m “{“siteId”:“default”}”’” but no recation. With snips-watch i see no logging:

[00:44:07] Watching on localhost:1883 (MQTT)
[00:44:13] [Hotword] detected on site default, for model default
[00:44:13] [Asr] was asked to stop listening on site default

Also have errors but can switch the lights.
Another point is. Have a Assistant created with Hotword "Jarvis" but snips listens still for "Hey Snips".
-----
Apr 30 00:46:40 raspberrypi systemd[1]: Started Snips Skill Server.
Apr 30 00:46:40 raspberrypi snips-skill-server[1877]: ERROR:snips_skill_server: no actions found !
Apr 30 00:46:40 raspberrypi systemd[1]: snips-skill-server.service: Main process exited, code=exited, status=1/FAILURE
Apr 30 00:46:40 raspberrypi systemd[1]: snips-skill-server.service: Unit entered failed state.
Apr 30 00:46:40 raspberrypi systemd[1]: snips-skill-server.service: Failed with result 'exit-code'.
Apr 30 00:46:45 raspberrypi systemd[1]: snips-skill-server.service: Service hold-off time over, scheduling restart.
Apr 30 00:46:45 raspberrypi systemd[1]: Stopped Snips Skill Server.
----

Hints?


#34

Hi, I had again the same problem and you need to change the path to the hotword in snips.toml. Change the hotword part in the snips.toml hotword section to to „custom_hotword“ path of your new assistant instead of the default setting. This will change hey snips to Jarvis. I also chooses Jarvis :wink:


#35

I do not use the skill server, so I can’t help you with that.

The sound toggle on message will not give a reaction, but the next time the hotword is detected, the sound should play over your output device.
Snips Watch should also log something like “audio server was asked to play …”

I see some double quotes in your message, maybe that is the problem?
I use MQTT.fx as client


#36

Hi Armin,

i tried this last night but without success. When i download the assistant project zip, there ist no custom hotword folder in the zip. Strange…


#37

Hi Romkabouter. Thanks again for your hints. The quotes was the problem :slight_smile:


#38

You should not have to update anything for the Jarvis hotword if you chose that one in the console
Most probably you did not (correctly) update you assisant and Snips is still listening to the old one.


#39

It seems so Jarvis is only available in english. But i saw it in german.
I make a english assistant ant test-


#40

I am going to try this on the creator today. Wish me luck