Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

lyrics: Genius backend needs to handle networking errors #2771

Closed
BinsonBuzz opened this issue Dec 29, 2017 · 8 comments
Closed

lyrics: Genius backend needs to handle networking errors #2771

BinsonBuzz opened this issue Dec 29, 2017 · 8 comments
Labels
bug bugs that are confirmed and actionable good first issue https://github.com/beetbox/beets/pull/5479

Comments

@BinsonBuzz
Copy link

BinsonBuzz commented Dec 29, 2017 •
edited
Loading

Problem

please see https://discourse.beets.io/t/crashing-on-resuming-import-help-please/257 for more info

Importing my library. After running:

docker exec -it beets /bin/bash -c 'beet import /music_library/adults'

get the following error intermittently, which ends import


Traceback (most recent call last):
  File "/usr/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.6', 'console_scripts', 'beet')()
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 937, in import_func
    import_files(lib, paths, query)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 914, in import_files
    session.run()
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 327, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 1506, in plugin_stage
    func(session, task)
  File "/usr/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 805, in imported
    False, self.config['force'])
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 818, in fetch_item_lyrics
    lyrics = [self.get_lyrics(artist, title) for title in titles]
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 851, in get_lyrics
    lyrics = backend.fetch(artist, title)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 388, in fetch
    return self.lyrics_from_song_api_path(song_api_path)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 356, in lyrics_from_song_api_path
    response = requests.get(song_url, headers=self.headers)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', error(104, 'Connection reset by peer'))

Setup

  • OS: unRAID 6.4RC17b
  • Python version: 2.7.14
  • beets version: 1.4.6
  • Turning off plugins made problem go away (yes/no): no

My configuration (output of beet config) is:
beets version 1.4.6
Python version 2.7.14
plugins: chroma, convert, duplicates, embedart, fetchart, fromfilename, ftintitle, lastgenre, lyrics, missing, replaygain, scrub, web
config: line 1: docker: command not found



directory: /music_library/adults
library: /config/musiclibrary.blb
art_filename: albumart
threaded: yes
original_date: no
per_disc_numbering: no

import:
    write: yes
    move: yes
    copy: no
    resume: ask
    incremental: yes
    quiet_fallback: skip
    autotag: yes
    timid: no
    log: /config/beet.log

musicbrainz:
    host: 172.35.12.88:5000
    ratelimit: 100

########### DOCKER DEFUALTS #################

paths:
    default: $albumartist/$album%aunique{}/$track - $title
    singleton: Non-Album/$artist - $title
    comp: Compilations/$album%aunique{}/$track - $title
    albumtype_soundtrack: Soundtracks/$album/$track $title

replace:
    '^\.': _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    '[\xE8-\xEB]': e
    '[\xEC-\xEF]': i
    '[\xE2-\xE6]': a
    '[\xF2-\xF6]': o
    '[\xF8]': o
    '\.$': _
    '\s+$': ''

web:
    host: 0.0.0.0
    port: 8337

########### My additions #####################

# import folders are deleted if only files matching these patterns are left in folders 
clutter: ["Thumbs.DB", ".DS_Store", "*.m3u", ".pls", "*.jpg"]

########### PLUGINS #####################
# chroma and web installed with docker #

plugins: fetchart lyrics lastgenre embedart ftintitle convert scrub replaygain missing chroma web duplicates missing fromfilename

fetchart:
    auto: yes
    maxwidth: 300
    cautious: true
    sources: amazon filesystem coverart albumart google  wikipedia
    fanarttv_key: REMOVED

lyrics:
    fallback: ''
    force: yes

lastgenre:
    auto: yes
    source: album
    canonical: yes
    canoncical: /config/genres-tree.yaml
    fallback: unknown
    whitelist: /config/genres.txt

embedart:
    auto: yes
    maxwidth: 300

ftintitle:
    auto: yes

convert:
    auto: no
    ffmpeg: /usr/bin/ffmpeg
    opts: -ab 320k -ac 2 -ar 48000
    max_bitrate: 320
    threads: 1

scrub:
    auto: yes

replaygain:
    auto: yes
    overwrite: yes

#duplicates:
#    move: /duplicates

missing:
    format: $albumartist - $album - $track - $title
    count: no
    total: yes
@sampsyo sampsyo changed the title Crashing on import lyrics: Genius backend needs to handle networking errors Dec 29, 2017
@sampsyo sampsyo added the bug bugs that are confirmed and actionable label Dec 29, 2017
@sampsyo
Copy link
Member

sampsyo commented Dec 29, 2017

Indeed; thanks! The problem is the Genius backend, introduced in #2709 by @lmagno. It needs to catch exceptions raised by the requests.get call to avoid bringing down the import if something goes wrong with the connection.

Any interest in adding that try block and logging, @lmagno?

@sampsyo sampsyo added the good first issue https://github.com/beetbox/beets/pull/5479 label Dec 29, 2017
@BinsonBuzz
Copy link
Author

any updates? I disabled lyrics in my config but I'm still getting errors:

Traceback (most recent call last):
  File "/usr/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.6', 'console_scripts', 'beet')()
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 937, in import_func
    import_files(lib, paths, query)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 914, in import_files
    session.run()
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 327, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 1506, in plugin_stage
    func(session, task)
  File "/usr/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 805, in imported
    False, self.config['force'])
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 818, in fetch_item_lyrics
    lyrics = [self.get_lyrics(artist, title) for title in titles]
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 851, in get_lyrics
    lyrics = backend.fetch(artist, title)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 377, in fetch
    response = requests.get(search_url, data=data, headers=self.headers)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 72, in get
    return request('get', url, params=params, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/api.py", line 58, in request
    return session.request(method=method, url=url, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 508, in request
    resp = self.send(prep, **send_kwargs)
  File "/usr/lib/python2.7/site-packages/requests/sessions.py", line 618, in send
    r = adapter.send(request, **kwargs)
  File "/usr/lib/python2.7/site-packages/requests/adapters.py", line 490, in send
    raise ConnectionError(err, request=request)
requests.exceptions.ConnectionError: ('Connection aborted.', BadStatusLine("''",))

@sampsyo
Copy link
Member

sampsyo commented Jan 6, 2018

Hmm; it looks like the genius backend is still active for some reason. Can you paste your config?

@BinsonBuzz
Copy link
Author

sorry, I thought I'd restarted the docker properly - restarted with lyrics disabled and it's working ok.

@BinsonBuzz
Copy link
Author

sorry, to mess you around - but the error is genuine. It just popped up again after around 30 mins of tagging:

Traceback (most recent call last):
  File "/usr/bin/beet", line 11, in <module>
    load_entry_point('beets==1.4.6', 'console_scripts', 'beet')()
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1256, in main
    _raw_main(args)
  File "/usr/lib/python2.7/site-packages/beets/ui/__init__.py", line 1243, in _raw_main
    subcommand.func(lib, suboptions, subargs)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 937, in import_func
    import_files(lib, paths, query)
  File "/usr/lib/python2.7/site-packages/beets/ui/commands.py", line 914, in import_files
    session.run()
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 327, in run
    pl.run_parallel(QUEUE_SIZE)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 445, in run_parallel
    six.reraise(exc_info[0], exc_info[1], exc_info[2])
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 312, in run
    out = self.coro.send(msg)
  File "/usr/lib/python2.7/site-packages/beets/util/pipeline.py", line 194, in coro
    func(*(args + (task,)))
  File "/usr/lib/python2.7/site-packages/beets/importer.py", line 1506, in plugin_stage
    func(session, task)
  File "/usr/lib/python2.7/site-packages/beets/plugins.py", line 124, in wrapper
    return func(*args, **kwargs)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 805, in imported
    False, self.config['force'])
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 818, in fetch_item_lyrics
    lyrics = [self.get_lyrics(artist, title) for title in titles]
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 851, in get_lyrics
    lyrics = backend.fetch(artist, title)
  File "/usr/lib/python2.7/site-packages/beetsplug/lyrics.py", line 378, in fetch
    json = response.json()
  File "/usr/lib/python2.7/site-packages/requests/models.py", line 892, in json
    return complexjson.loads(self.text, **kwargs)
  File "/usr/lib/python2.7/json/__init__.py", line 339, in loads
    return _default_decoder.decode(s)
  File "/usr/lib/python2.7/json/decoder.py", line 364, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
  File "/usr/lib/python2.7/json/decoder.py", line 382, in raw_decode
    raise ValueError("No JSON object could be decoded")
ValueError: No JSON object could be decoded

My config:

directory: /music_library/adults
library: /config/musiclibrary.blb
art_filename: albumart
threaded: yes
original_date: no
per_disc_numbering: no

import:
    write: yes
    move: yes
    copy: no
    resume: ask
    incremental: yes
    quiet_fallback: skip
    autotag: yes
    timid: no
    log: /config/beet.log

# musicbrainz:
#    host: 172.35.12.88:5000
#    ratelimit: 100

########### DOCKER DEFUALTS #################

paths:
    default: $albumartist/$album%aunique{}/$track - $title
    singleton: Non-Album/$artist - $title
    comp: Compilations/$album%aunique{}/$track - $title
    albumtype_soundtrack: Soundtracks/$album/$track $title

replace:
    '^\.': _
    '[\x00-\x1f]': _
    '[<>:"\?\*\|]': _
    '[\xE8-\xEB]': e
    '[\xEC-\xEF]': i
    '[\xE2-\xE6]': a
    '[\xF2-\xF6]': o
    '[\xF8]': o
    '\.$': _
    '\s+$': ''

web:
    host: 0.0.0.0
    port: 8337

########### My additions #####################

# import folders are deleted if only files matching these patterns are left in folders 
clutter: ["Thumbs.DB", ".DS_Store", "*.m3u", ".pls", "*.jpg"]

########### PLUGINS #####################
# chroma and web installed with docker #

plugins: fetchart lyrics lastgenre embedart ftintitle convert scrub replaygain missing chroma web duplicates missing fromfilename

fetchart:
    auto: yes
    maxwidth: 300
    cautious: true
    sources: amazon filesystem coverart albumart google  wikipedia
    fanarttv_key: a013aa2ee126dfbe268c1a51fff7b2a7

# lyrics:
#    fallback: ''
#    force: yes

lastgenre:
    auto: yes
    source: album
    canonical: yes
    canoncical: /config/genres-tree.yaml
    fallback: unknown
    whitelist: /config/genres.txt

embedart:
    auto: yes
    maxwidth: 300

ftintitle:
    auto: yes

convert:
    auto: no
    ffmpeg: /usr/bin/ffmpeg
    opts: -ab 320k -ac 2 -ar 48000
    max_bitrate: 320
    threads: 1

scrub:
    auto: yes

replaygain:
    auto: yes
    overwrite: yes

#duplicates:
#    move: /duplicates

missing:
    format: $albumartist - $album - $track - $title
    count: no
    total: yes

@sampsyo
Copy link
Member

sampsyo commented Jan 6, 2018

Hmm; I don't see anything in your config that would disable the Genius backend for the lyrics plugin. Perhaps you still need to add that?

@BinsonBuzz
Copy link
Author

BinsonBuzz commented Jan 6, 2018 •
edited
Loading

duh just realised I'd removed the custom lyrics plugin options, but left the plugin still enabled.

Will re-enable once the lyrics issue is fixed to help test

@sampsyo
Copy link
Member

sampsyo commented Jan 31, 2018

Hi! I think the above should fix these errors. If you get a chance, could you try re-enabling the genius source and trying that out?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug bugs that are confirmed and actionable good first issue https://github.com/beetbox/beets/pull/5479
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@sampsyo @BinsonBuzz