Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

musixmatch source is broken on lyrics plugin (404) #4585

Closed
2V3EvG4LMJFdRe opened this issue Dec 15, 2022 · 7 comments · Fixed by #4616
Closed

musixmatch source is broken on lyrics plugin (404) #4585

2V3EvG4LMJFdRe opened this issue Dec 15, 2022 · 7 comments · Fixed by #4616
Labels
bug bugs that are confirmed and actionable

Comments

@2V3EvG4LMJFdRe
Copy link

Problem

lyrics:
    auto: yes
    force: yes
    sources: musixmatch
    fallback: ''
    import.write: yes

is giving me no results for any query that I give the plugin. For instance:

beet lyrics keren ann
lyrics: lyrics not found: Keren Ann - 101 - My Name Is Trouble
lyrics: lyrics not found: Keren Ann - 101 - Run With You
lyrics: lyrics not found: Keren Ann - 101 - All the Beautiful Girls
lyrics: lyrics not found: Keren Ann - 101 - Sugar Mama
lyrics: lyrics not found: Keren Ann - 101 - She Won't Trade It for Nothing
lyrics: lyrics not found: Keren Ann - 101 - You Were on Fire
lyrics: lyrics not found: Keren Ann - 101 - Blood on My Hands
lyrics: lyrics not found: Keren Ann - 101 - Song From a Tour Bus
lyrics: lyrics not found: Keren Ann - 101 - Strange Weather
lyrics: lyrics not found: Keren Ann - 101 - 101
lyrics: lyrics not found: Keren Ann - 101 - Daddy, You Been On My Mind
lyrics: lyrics not found: Keren Ann - Bleue - Les Jours heureux
lyrics: lyrics not found: Keren Ann - Bleue - Bleu
lyrics: lyrics not found: Keren Ann - Bleue - Le Fleuve doux
lyrics: lyrics not found: Keren Ann - Bleue - Nager la nuit
lyrics: lyrics not found: Keren Ann - Bleue - Sous l'eau
lyrics: lyrics not found: Keren Ann - Bleue - Ton île prison
lyrics: lyrics not found: Keren Ann - Bleue - Odessa, odyssée
lyrics: lyrics not found: Keren Ann - Bleue - La Mauvaise Fortune
lyrics: lyrics not found: Keren Ann - Bleue - Le goût était acide
lyrics: lyrics not found: Keren Ann - Keren Ann - It's All a Lie
lyrics: lyrics not found: Keren Ann - Keren Ann - Lay Your Head Down
lyrics: lyrics not found: Keren Ann - Keren Ann - In Your Back
lyrics: lyrics not found: Keren Ann - Keren Ann - The Harder Ships of the World
lyrics: lyrics not found: Keren Ann - Keren Ann - It Ain't No Crime
lyrics: lyrics not found: Keren Ann - Keren Ann - Where No Endings End
lyrics: lyrics not found: Keren Ann - Keren Ann - Liberty
lyrics: lyrics not found: Keren Ann - Keren Ann - Between the Flatland and the Caspian Sea
lyrics: lyrics not found: Keren Ann - Keren Ann - Caspia
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Dimanche en hiver
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Dans ma ville
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Seule
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - On est loin
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Sur le fil
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Peut-être
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Reste là
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Décrocher les étoiles
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Jardin d'hiver
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Aéroplane
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Deux
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Les mercenaires
lyrics: lyrics not found: Keren Ann - La biographie de Luka Philipsen - Autour de l'arbre
lyrics: lyrics not found: Keren Ann - La disparition - Au coin du monde
lyrics: lyrics not found: Keren Ann - La disparition - Le sable mouvant
lyrics: lyrics not found: Keren Ann - La disparition - Les rivières de janvier
lyrics: lyrics not found: Keren Ann - La disparition - La corde et les chaussons
lyrics: lyrics not found: Keren Ann - La disparition - Surannée
lyrics: lyrics not found: Keren Ann - La disparition - Ailleurs
lyrics: lyrics not found: Keren Ann - La disparition - L'illusioniste
lyrics: lyrics not found: Keren Ann - La disparition - La tentation
lyrics: lyrics not found: Keren Ann - La disparition - Mes pas dans la neige
lyrics: lyrics not found: Keren Ann - La disparition - Le chien d'avant-garde
lyrics: lyrics not found: Keren Ann - La disparition - La disparition
lyrics: lyrics not found: Keren Ann - Nolita - Que n'ai-je?
lyrics: lyrics not found: Keren Ann - Nolita - Greatest You Can Find
lyrics: lyrics not found: Keren Ann - Nolita - Chelsea Burns
lyrics: lyrics not found: Keren Ann - Nolita - One Day Without
lyrics: lyrics not found: Keren Ann - Nolita - La forme et le fond
lyrics: lyrics not found: Keren Ann - Nolita - Nolita
lyrics: lyrics not found: Keren Ann - Nolita - Roses & Hips
lyrics: lyrics not found: Keren Ann - Nolita - Midi dans le salon de la Duchesse
lyrics: lyrics not found: Keren Ann - Nolita - L'onde amère
lyrics: lyrics not found: Keren Ann - Nolita - For You and I
lyrics: lyrics not found: Keren Ann - Nolita - Song of Alice (feat. Sean Gullette)
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Not Going Anywhere
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Polly
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Road Bin
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - End of May
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Sailor & Widow
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Sit in the Sun
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Right Now & Right Here
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Seventeen
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - Spanish Song Bird
lyrics: lyrics not found: Keren Ann - Not Going Anywhere - By the Cathedral
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You're Gonna Get Love
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Bring Back
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - The Separated Twin
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Insensible World
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Where Did You Go?
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Easy Money
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - My Man Is Wanted but I Ain't Gonna Turn Him In
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You Knew Me Then
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - Again and Again
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - The River That Swallows All the Rivers
lyrics: lyrics not found: Keren Ann - You're Gonna Get Love - You Have It All to Lose
lyrics: lyrics not found: Keren Ann feat. Barði Jóhannsson - Not Going Anywhere - Ending Song
lyrics: lyrics not found: Keren Ann feat. David Byrne - Bleue - Le Goût d'inachevé

plugin paths: 
artresizer: method is (2, (7, 0, 10), False)
Sending event: pluginload
library database: /Users/redacted/.config/beets/library.db
library directory: /Users/redacted/Music
Sending event: library_opened
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Que-N%27Ai-Je%3F (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Que-N%27Ai-Je%3F (404)
lyrics: lyrics not found: Keren Ann - Nolita - Que n'ai-je?
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Greatest-You-Can-Find (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Greatest-You-Can-Find (404)
lyrics: lyrics not found: Keren Ann - Nolita - Greatest You Can Find
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Chelsea-Burns (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Chelsea-Burns (404)
lyrics: lyrics not found: Keren Ann - Nolita - Chelsea Burns
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/One-Day-Without (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//One-Day-Without (404)
lyrics: lyrics not found: Keren Ann - Nolita - One Day Without
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/La-Forme-Et-Le-Fond (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//La-Forme-Et-Le-Fond (404)
lyrics: lyrics not found: Keren Ann - Nolita - La forme et le fond
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Nolita (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Nolita (404)
lyrics: lyrics not found: Keren Ann - Nolita - Nolita
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Roses-%26-Hips (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Roses-%26-Hips (404)
lyrics: lyrics not found: Keren Ann - Nolita - Roses & Hips
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Midi-Dans-Le-Salon-De-La-Duchesse (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Midi-Dans-Le-Salon-De-La-Duchesse (404)
lyrics: lyrics not found: Keren Ann - Nolita - Midi dans le salon de la Duchesse
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/L%27Onde-Am%C3%A8re (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//L%27Onde-Am%C3%A8re (404)
lyrics: lyrics not found: Keren Ann - Nolita - L'onde amère
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/For-You-And-I (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//For-You-And-I (404)
lyrics: lyrics not found: Keren Ann - Nolita - For You and I
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Song-Of-Alice-%28Feat.-Sean-Gullette%29 (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics/Keren-Ann/Song-Of-Alice (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Song-Of-Alice-%28Feat.-Sean-Gullette%29 (404)
lyrics: failed to fetch: https://www.musixmatch.com/lyrics//Song-Of-Alice (404)
lyrics: lyrics not found: Keren Ann - Nolita - Song of Alice (feat. Sean Gullette)

Setup

  • OS: macOS 10.13.6
  • Python version: 3
  • beets version: 1.6.0

My configuration (output of beet config) is:

lyrics:
    bing_lang_from: []
    auto: yes
    force: yes
    sources: musixmatch
    fallback: ''
    import.write: yes
    bing_client_secret: REDACTED
    bing_lang_to:
    google_API_key: REDACTED
    google_engine_ID: REDACTED
    genius_api_key: REDACTED
    local: no

plugins: embedart fetchart lyrics
autotag: yes
ignore_hidden: yes
original_date: yes
languages: en
va_name: Compilation

import:
    copy: no
    write: yes
    move: no
convert:
    tmpdir: /Volumes/Expansion-1/Media/Audio/
    auto: yes
    delete_originals: no
    no_convert: path::\.(m4a)$
    command: ffmpeg -y -i $source -vsync 0 -q:a 0 -map_metadata 0 $dest
    extension: m4a
discogs:
    user_token: yhRjvGwdihHSyuSMebbwfkDCovmrNHkodhTEdOuk
fetchart:
    auto: yes
    sources: fanarttv itunes amazon
    fanarttv_key: REDACTED
    minwidth: 0
    maxwidth: 0
    quality: 0
    max_filesize: 0
    enforce_ratio: no
    cautious: no
    cover_names:
    - cover
    - front
    - art
    - album
    - folder
    google_key: REDACTED
    google_engine: 001442825323518660753:hrh5ch1gjzm
    lastfm_key: REDACTED
    store_source: no
    high_resolution: no
    deinterlace: no
    cover_format:
embedart:
    auto: yes
    maxwidth: 1000
    remove_art_file: yes
    compare_threshold: 0
    ifempty: no
    quality: 0
zero:
    auto: yes
    update_database: yes
    keep_fields:
    - artist
    - title
    - album
    - year
    - albumartist
    - track
    - tracktotal
    - disc
    - disctotal
    - label
    - unsyncedlyrics
    - images
    - rating
    - first played
    - last played
    - play count
@2V3EvG4LMJFdRe
Copy link
Author

Broken on 1.6.1 also.

@jackwilsdon
Copy link
Member

It seems like Musixmatch have blocked the beets/... user agent (actually any UA containing beets):

$ curl -H "User-Agent: beets/1.6.1" 'https://www.musixmatch.com/lyrics/Disclosure/White-Noise'
Not Allowed

$ curl -H "User-Agent: boots/1.6.1" 'https://www.musixmatch.com/lyrics/Disclosure/White-Noise'
<normal response here>

Musixmatch do offer an API but their free tier is for evaluation only and doesn't give full access to lyrics. We may have to just remove Musixmatch as a backend, which is a bit of a shame.

@jackwilsdon jackwilsdon added the bug bugs that are confirmed and actionable label Dec 15, 2022
@2V3EvG4LMJFdRe
Copy link
Author

2V3EvG4LMJFdRe commented Dec 15, 2022 •
edited
Loading

Man, that is indeed a shame 😞 They had the best lyrics from what I've gathered, I know Spotify uses them and just doing a quick comparison Genius has a lot of straight up garbage on their lyrics that Musixmatch doesn't (Publishers: ©Sony/ATV Music Publishing LLC Popularity : 0 users have visited this page Length: 3:46 Sponsored links, repeating the [artist name] Lyrics header into the actual lyrics, messed up unicode etc)

@sampsyo
Copy link
Member

sampsyo commented Dec 15, 2022

Indeed; this is too bad. I think the right solution is probably to leave the code in place, in case something changes in the future, but disable it by default and remove it from the documentation?

@2V3EvG4LMJFdRe
Copy link
Author

Would it be possible to change the UA identifier like you can in a browser, perhaps at the user's side only if not on the actual beets bundle? Maybe adding some function to slow the requests per second. At some point between those two it wouldn't be any more specific than normal/accepted behaviour of opening up their site from a browser and looking up lyrics.

jackwilsdon added a commit to jackwilsdon/beets that referenced this issue Dec 31, 2022
@jackwilsdon
Remove Musixmatch from default enabled sources
0c7e779 
Musixmatch are currently blocking all requests with the beets user
agent. See beetbox#4585.
@jackwilsdon jackwilsdon mentioned this issue Dec 31, 2022
Merged
3 tasks
@jackwilsdon
Copy link
Member

It feels a bit dishonest to allow UA spoofing, as we are then actively attempting (or at least offering a way) to circumvent their block.

You could edit the code yourself to change the UA, but I don't think it's something we'd want to offer officially. I've opened #4616 to do as @sampsyo has suggested.

jackwilsdon added a commit to jackwilsdon/beets that referenced this issue Dec 31, 2022
@jackwilsdon
Remove Musixmatch from default enabled sources
dad9d23 
Musixmatch are currently blocking all requests with the beets user
agent. See beetbox#4585.
@sampsyo
Copy link
Member

sampsyo commented Jan 2, 2023

I agree with @jackwilsdon here: it feels a little icky to work around a limitation that MusixMatch is clearly imposing with the intent to block beets. And I also don't think it would be effective: any new UA string we choose would likely also get blocked. I would much rather leave it up to individual users to do themselves if they're very motivated to do so.

@snejus snejus mentioned this issue Jun 4, 2024
Merged
@2V3EvG4LMJFdRe 2V3EvG4LMJFdRe mentioned this issue Jul 4, 2024
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug bugs that are confirmed and actionable
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Remove Musixmatch from default enabled sources jackwilsdon/beets
3 participants
@sampsyo @jackwilsdon @2V3EvG4LMJFdRe