Conflict between behaviour & documentation of incremental_skipped_later flag

[rulimanurung]

See original discussion here: https://discourse.beets.io/t/importing-skipped-albums-incremental-skipped-later-not-being-observed-properly/815

There is a conflict between the behaviour of the incremental_skip_later flag as stated in the documentation here and in the actual implementation here.

The documentation states:

Either yes or no, controlling whether skipped directories are recorded in the incremental list. When set to yes, skipped directories will be recorded, and skipped later. When set to no, skipped directories won’t be recorded, and beets will try to import them again later. Defaults to no.

However, the boolean condition in importer.py is currently:

            # Should we skip recording to incremental list?
            self.skip and session.config['incremental_skip_later']

Thus, if incremental_skip_later is true/yes then it will NOT record skipped directories in the incremental list (and thus NOT skipped upon subsequent imports), but if it is false/no then it WILL record skipped directories in the incremental list (and thus WILL be skipped upon subsequent imports). It looks like this error was introduced in this commit by @jroitgrund.

Problem

$ beet import -qil log.txt /my/audio/folder/
$ beet import -il log.txt /my/audio/folder/

The expectation is that on the second import, since the quiet flag is turned off, beets will try to import the albums that were skipped during the first import. However, it just states No files imported from /my/audio/folder, and skips over everything.

Verified that if I add the following to config.yaml, then it works as expected.

import:
  incremental_skip_later: yes

I suppose this can be fixed by either changing the documentation or changing the boolean condition in importer.py. To be honest, I'm not entirely sure what should be the expected behaviour given the flag name "incremental_skip_later". It's quite confusing.

[sampsyo]

Arg! What a mess. This is indeed very confusing and we’ve obviously messed up the docs.

Here's a brief history of where all this came from:

• Before all this started, we always recorded everything, regardless of whether or not the user skipped it.
• The original implementation of the config option comes from When importer is in incremental mode, offer an option to avoid recording the album #2773. That seems to have been incorrect, and when the option was turned on, it never recorded anything.
• The current behavior was introduced in 5f26ec6. This means that when the option is off (the default), the behavior is the same as it was before the option even existed (i.e., we record everything all the time). And when it's on, you get a chance to come back to skipped music later. This seems to match the name.
• The docs were somewhat unclear. So Clarify description of incremental_skip_later #3187 attempted to make it clearer, but in doing so, it got the behavior exactly backward.

So I think we should fix the docs, not the code. It needs to say the opposite of what it currently says. And hopefully we can make the actual use case more obvious in the process.

[RobbieG3]

This is closed, but I think the doc is still incorrect. I ran my first import -q with incremental: yes and no settings for incremental_skip_later (default: no, if I understand correctly). When it finished, I re-ran import to try to manually address the skipped albums, but it just said "No files imported from" each skipped folder. So either the default for incremental_skip_later is set incorrectly, or it needs to be set to yes in order to try to import skipped albums later.

I am also unclear about what to do at this point to import these skipped albums. I can run with incremental: no, but I only want to try to import the previously skipped albums. Do I have to completely start over from scratch?

[spencermathews]

The wording has been fixed in the latest version of the docs. The correction has apparently not made it to the stable version of the RTD docs, which is the version where the root URL redirects.

I believe you should have set incremental: yes and incremental_skip_later: yes to be able to revisit skipped directories only. I'd be curious if someone has a better solution, but if I were you I'd just start over and redo the quiet import with these new config settings.