Voice activated YouTube Music search

[gmurnane]

I wanted to be able to request songs verbally, without any prior interaction with those songs (not in the library, not favorited, not in a playlist), and get results pulled from YT Music. The approach I used is probably extensible to albums, and "Play songs by this artist" pretty easily, but so far the fully open source voice matching is pretty unreliable for exactly matching an artist's name, so I started with single songs. Luckily, YT Music does a pretty good job of guessing what was meant with whatever it's handed, and fuzzy matching between title and artist has been good enough for me.

I now have something like this working:
"Hey Jarvis ... play Magic Forest by Amberian Dawn" (Then Magic Forest by Amberian Dawn plays)

I am aware that Music Assistant already has python, with ytmusicapi, and authentication set up, and if I were a better developer, this might be best handled as an integrated feature.

Prerequisites:

I'm running the YT music search on an always-on Linux server, on my local network. This is required only because I don't know enough about running python as an addon for HA, so there's likely a way to remove the need for that Linux system entirely (and maybe improve the speed of this operation), but I don't expect to do it.

You need a configured voice assistant/wake word for HA. I followed the Year of the Voice $13 voice assistant tutorial. You also must have music assistant working, and connected to/able to play from YT Music.

Directions:

On the linux system, set up a new python venv, and install some pip packages:

mkdir yt_music_search
cd yt_music_search
python -m venv .
./bin/python -m pip install --upgrade pip
./bin/pip install ytmusicapi
./bin/pip install thefuzz
./bin/pip install flask
# configure the oauth.json for ytmusicapi access:
./bin/ytmusicapi oauth

Get a long-lived access token from your home assistant, and configure your home assistant URL, authorization token, and music assistant player_id in this script, and save it as yt_music_search_server.py:

from ytmusicapi import YTMusic
from thefuzz import fuzz
from pprint import pprint
from requests import post
from flask import Flask, request, jsonify

app = Flask(__name__)

@app.route('/songbyartist', methods=['POST'])
def call_function():
  title = request.get_json()['title']
  artist = request.get_json()['artist']
  result = play_title_artist(title, artist)
  return jsonify({'result': result})

yt=YTMusic('oauth.json')

url="http://192.168.1.10:8123/api/services/mass/queue_command"
headers={
  "Authorization": "Bearer eyJhbGciOiJI_a_whole_lot_more_v6yvB3fO6Mtc7ESo",
  "content-type": "application/json",
}

def play_title_artist(title, artist):
  print("Got a request to play " + title + " by " + artist)

  data={"player_id":"a8:f8:4d:be:ef:1d","command":"clear"}
  response = post(url, headers=headers, json=data)

  sr=yt.search(title + " by " + artist)

  max_score=0
  max_pscore=0
  found_title=''
  found_artist=''
  found_videoId=''

  for r in sr:
    if r['resultType'] == 'song':
      score=fuzz.ratio(r['title'],title) + max([0]+[fuzz.ratio(a['name'],artist) for a in r['artists']])
      pscore=fuzz.partial_ratio(r['title'],title) + max([0]+[fuzz.partial_ratio(a['name'],artist) for a in r['artists']])
      if score > max_score:
        max_score=score
        max_pscore=pscore
        found_title=r['title']
        ascore=0
        for a in r['artists']:
          if fuzz.ratio(a['name'],artist) > ascore:
            ascore = fuzz.ratio(a['name'],artist)
            found_artist=a['name']
        if ascore == 0:
          found_artist=''
        found_videoId=r['videoId']
      elif score == max_score and pscore > max_pscore:
        max_pscore=pscore
        found_title=r['title']
        ascore=0
        for a in r['artists']:
          if fuzz.ratio(a['name'],artist) > ascore:
            ascore = fuzz.ratio(a['name'],artist)
            found_artist=a['name']
        if ascore == 0:
          found_artist=''
        found_videoId=r['videoId']

  if found_videoId == '':
    print("I couldn't find anything on youtube music when searching for " + title + " by " + artist)
    print("Here's what did turn up:")
    pprint(sr)
    return 404
  else:
    print("Sending a request to play " + found_title + " by " + found_artist)
    data={"player_id":"a8:f8:4d:be:ef:1d","command":"play_media","uri":"ytmusic://track/"+found_videoId,"enqueue_mode":"play"}
    response = post(url, headers=headers, json=data)
    return response.status_code


if __name__ == '__main__':
  app.run(
    debug=True,
    host="0.0.0.0",
    port=5000,
  )

Then launch that python server. It's probably more reliable to set it to launch at system start with a systemd unit file or something, but ideally this would eventually wind up on the HA device.

./bin/python yt_music_search_server.py

Now in home assistant, add this to /config/configuration.yaml:

intent_script:
  PlaySong:
    action:
      service: "shell_command.play_song"
      data:
        title: "{{ title }}"
        artist: "{{ artist }}"
    speech:
      text: "Playing {{ title }} by {{ artist }}"

shell_command:
  play_song: bash /config/shell/play_song.sh "{{ title }}" "{{ artist }}"

Create the file /config/shell/play_song.sh, modifying the IP to match your linux server. This is broken out into a separate script because YAML didn't like my attempts to quote JSON with template values.

curl -sS -XPOST -H 'content-type: application/json' 192.168.1.11:5000/songbyartist -d '{"title":"'"${1}"'","artist":"'"${2}"'"}'

Finally, create /config/custom_sentences/en/play_music.yaml:

language: "en"
intents:
  PlaySong:
     data:
      - sentences:
        - "play song {title} by {artist}"
        - "play {title} by {artist}"
lists:
  title:
    wildcard: true
  artist:
    wildcard: true

After HA is reloaded, the new sentence/intent should be working. I found it helpful to type to the Assist engine first to verify, then try the voice commands once that was working.

[marcelveldt]

Also have a look at the new beta 5 of the HA integration, I included a new search service that returns data so that saves you maybe some of the duplicate code. Also the "play media" service now accepts search phrases as the content_id such as "Queen - Innuendo" or "Innuendo by Queen".