diff --git a/.gitignore b/.gitignore index f00b44a..3c40ffa 100644 --- a/.gitignore +++ b/.gitignore @@ -7,3 +7,8 @@ musicbrainz2 /*.toc *.pyc .tox +.idea +venv +.eggs +/build/scripts-3.9/isrcsubmit.py +/build/scripts-3.9/isrcDigitalSubmit.py diff --git a/AUTHORS b/AUTHORS index cf90d91..deadfcc 100644 --- a/AUTHORS +++ b/AUTHORS @@ -1,6 +1,7 @@ This is just one isrcsubmit. There are other isrc submission tools by different people. - +This project also contains isrcDigitalSubmit, a derived tool +which extracts ISRC codes from digital media instead of CDs. The following people have contributed to this isrcsubmit.py: @@ -27,3 +28,7 @@ The following people have contributed to this isrcsubmit.py: Jim Patterson Guilherme Danno - minor fixes + +The following people have contributed to isrcDigitalSubmit.py: + Jim Patterson + diff --git a/Makefile b/Makefile index 51969bc..6cfd9c2 100644 --- a/Makefile +++ b/Makefile @@ -1,5 +1,5 @@ #version := 2.1.0-dev -version := 2.1.0 +version := 2.2.2 build: ./setup.py build @@ -17,7 +17,7 @@ upload: version: sed -i -e 's/\(Isrcsubmit\s\)[0-9.]\+[0-9a-z.-]*/\1$(version)/' README.rst sed -i -e 's/\(__version__\s=\s"\)[0-9.]\+[0-9a-z.-]*/\1$(version)/' \ - isrcsubmit.py + isrcshared.py sed -i -e 's/\(version="\)[0-9.]\+[0-9a-z.-]*/\1$(version)/' \ setup.py diff --git a/README.rst b/README.rst index 88f75a1..e548378 100644 --- a/README.rst +++ b/README.rst @@ -1,11 +1,17 @@ -Isrcsubmit 2.1.0 for MusicBrainz +Isrcsubmit 2.2.2 for MusicBrainz ================================ -This python script extracts ISRCs_ from audio CDs -and submits them to MusicBrainz_. +This project includes two python scripts that extracts ISRCs +and submits them to MusicBrainz_. isrcsubmit extracts ISRCs from +Audio CDs. isrcDigitalSubmit extracts ISRCs from audio files. -This script uses python-musicbrainzngs to access the MusicBrainz API -and python-discid to create an identifier for the disc. +These scripts use python-musicbrainzngs to access the MusicBrainz API. + +isrcsubmit uses python-discid to create an identifier for the disc which +is used to locate corresponding releases in MusicBrainz. + +isrcDigitalSubmit uses mutagen to analyze audio files, and locates matching +MusicBrainz releases based on embedded Artist, Album and AlbumArtist tags. The script works for Linux, Mac OS X and Windows. @@ -15,11 +21,11 @@ The script works for Linux, Mac OS X and Windows. Features: --------- -* read ISRCs from disc -* search for releases with the TOC of the disc +* read ISRCs from disc or audio files +* search for releases with the TOC of the disc or embedded tags * display release information from MusicBrainz * submit ISRCs -* submit discIds / TOCs +* submit discIds / TOCs (isrcsubmit) * duplicate ISRC detection (local and on server) @@ -29,23 +35,32 @@ Dependencies: * Python 2 >= 2.6 or Python 3 >= 3.1 * python-discid_ >= 1.0.0 (or python-libdiscid_ >= 0.2.0) * python-musicbrainzngs_ >= 0.4 +* mutagen_ >= 1.45.1 +* unidecode >= 1.2.0 * keyring_ (optional) .. _python-discid: http://python-discid.readthedocs.org/ .. _python-libdiscid: http://pythonhosted.org/python-libdiscid/ .. _python-musicbrainzngs: http://python-musicbrainzngs.readthedocs.org/ -.. _keyring: https://github.com/jaraco/keyring - +.. _keyring: https://github.com/jaraco/keyring/ +.. _mutaagen: https://mutagen.readthedocs.io/ +.. _unicode: https://github.com/avian2/unidecode Usage: ------ :: isrcsubmit.py [options] [username] [device] + isrcDigitalSubmit.py [options] [username] audioFiles ... -All arguments are optional. For detailed usage see:: +All arguments are optional other than audioFiles for isrcDigitalSubmit. +AudioFiles may be zipped (so a ZIP file as distributed by the vendor can +usually be passed directly to the script). + +For detailed usage see:: isrcsubmit.py -h + isrcDigitalSubmit.py -h Windows Usage: @@ -54,6 +69,7 @@ Windows Usage: Windows users should use:: isrcsubmit.bat + isrcDigitalSubmit.bat Mac Usage: @@ -62,6 +78,7 @@ Mac Usage: Mac users should rather use:: isrcsubmit.sh + isrcDigitalSubmit.sh This also works on Linux. @@ -83,8 +100,36 @@ is fetched from MusicBrainz. If the disc is unknown, you will be given the chance to submit the ID to the server. +Identifying Releases from Tags +------------------------------ + +A Digital Media release has no equivalent of a DiscID as defined for CDs. +Instead, it uses tags that are embedded in the digital media files. The +tags of interest are Album, AlbumArtist, Artist and TrackNumber. (Other +formats eg. MP3 might name these different; this discussion will use the +Vorbis tag names). + +Identifying a release is most straightforward if the albumartist tag is found. +The same albumartist tag must be on all tracks. isrcDigitalSubmit will look for +a Digital Media release with that album artist and title. If more than one is found, +the user will be asked to choose. + +If no Album Artist is found, the artist will be treated as an album artist. +An attempt is made to strip away any "featured artist" from tracks, either by looking +for connectors like "feat." or "duet with", or by checking for a common name that +is found on all artist name strings. If it can't identify a single artist, it will +look for a release without specifying an artist. + +If all else fails, the user can point the script to a specific release by use of +the --release-id= option. + +isrcDigitalRelease always verifies that the digital media release matches the MusicBrainz +release by checking for matching titles and artists and similar track times. It also +verifies that the format is "Digital Media"; it will not attach +ISRCs to releases with a different format. + -"Installation": +Installation: --------------- If you downloaded isrcsubmit as a zip package for your platform @@ -103,7 +148,7 @@ but "python setup.py install" might work for you. However, the backends and libraries should get installed so that the script has access to them. -On Linux you just install the above mentioneed dependencies with +On Linux you just install the above mentioned dependencies with the package manager of your distribution. For Ubuntu all dependencies should be in the MusicBrainz Stable PPA, if not in the official repositories. @@ -133,7 +178,7 @@ You might find additional information about this script at the `MusicBrainz forums`_. .. _web page: http://jonnyjd.github.io/musicbrainz-isrcsubmit/backends -.. _MusicBrainz forums: http://forums.musicbrainz.org/viewtopic.php?id=3444 +.. _MusicBrainz forums: https://community.metabrainz.org/search?q=isrcsubmit Bugs: diff --git a/doc/conf.py b/doc/conf.py index a38ba99..0d3e862 100644 --- a/doc/conf.py +++ b/doc/conf.py @@ -175,7 +175,8 @@ # (source start file, name, description, authors, manual section). man_pages = [ ('isrcsubmit.1', 'isrcsubmit', u'isrcsubmit Documentation', [], 1), - ('isrcsubmit-config.5', 'isrcsubmit-config', u'isrcsubmit Documentation', + ('isrcDigitalSubmit.1', 'isrcDigitalSubmit', u'isrcDigitalSubmit Documentation', [], 1), + ('isrcsubmit-config.5', 'isrcsubmit-config', u'isrcsubmit and isrcDigitalSubmit Configuration Documentation', [], 5) ] diff --git a/doc/isrcDigitalSubmit.1.rst b/doc/isrcDigitalSubmit.1.rst new file mode 100644 index 0000000..7688e7b --- /dev/null +++ b/doc/isrcDigitalSubmit.1.rst @@ -0,0 +1,50 @@ +isrcDigitalSubmit manual page +====================== + +Synopsis +-------- + +**isrcDigitalSubmit** [*options*] [*user*] *audioFile*... + +Description +----------- + +:program:`isrcDigitalSubmit` extracts International Standard Recording Codes (ISRC) +from audio files and allows one to submit the data to MusicBrainz. ISRCs are used +to uniquely identifiy sound and music video recordings. + +Options +------- + +--version + Show program's version number and exit. +-h + Short usage help. +--help + Complete help for the program. +--debug + Show debug messages. +-u , --user= + MusicBrainz username, if not given as argument. +--release-id= + Optional MusicBrainz ID of the release. This will be gathered if not given. +--browser= + Program to open URLs. This will be automatically deteced for most setups, + if not chosen manually. +--server= + Server to send ISRCs to. If not given, musicbrainz.org is used. +--keyring + Use keyring if it is available. +--no-keyring + Do not use keyring. + +See also +-------- + +:manpage:`isrcsubmit-config(5)` + +Author +------ + +This manual was written by Sebastian Ramacher and Johannes Dewender. +:program:`isrcDigitalSubmit` was written by Jim Patterson adapted from isrcsubmit by Johannes Dewender. diff --git a/doc/isrcsubmit-config.5.rst b/doc/isrcsubmit-config.5.rst index 67c1f41..5e2f9b9 100644 --- a/doc/isrcsubmit-config.5.rst +++ b/doc/isrcsubmit-config.5.rst @@ -12,8 +12,8 @@ Description ----------- The configuration file contains various options controlling the behavior of -:program:`isrcsubmit`. All the options given here can be overridden by passing -command line arguments to :program:`isrcsubmit`. +:program:`isrcsubmit` and :program:`isrcDigitalubmit`. All the options given here can be overridden by passing +command line arguments. If **$XDG_CONFIG_HOME** is not set, **%APPDATA%** is used on Windows and **~/.config** for all other systems. @@ -25,7 +25,7 @@ This refers to the ``[general]`` section of the configuration file. backend ^^^^^^^ -Force using a specific backend to extract ISRCs from the disc. Possible +(:program:`isrcsubmit` only). Force using a specific backend to extract ISRCs from the disc. Possible backends are: mediatools, media_info, cdrdao, libdiscid, discisrc. browser @@ -34,7 +34,7 @@ Program to open URLs. device ^^^^^^ -CD device with a loaded audio CD. +(:program:`isrcsubmit` only). CD device with a loaded audio CD. keyring ^^^^^^^ diff --git a/isrcDigitalSubmit.bat b/isrcDigitalSubmit.bat new file mode 100644 index 0000000..96b5674 --- /dev/null +++ b/isrcDigitalSubmit.bat @@ -0,0 +1,5 @@ +@echo off +for /f "tokens=2 delims=:." %%x in ('chcp') do set cp=%%x +chcp 65001>NUL & cmd /c "isrcDigitalSubmit.py %*" & chcp %cp%>NUL +echo. +pause diff --git a/isrcDigitalSubmit.py b/isrcDigitalSubmit.py new file mode 100755 index 0000000..065384f --- /dev/null +++ b/isrcDigitalSubmit.py @@ -0,0 +1,666 @@ +#!/usr/bin/env python +# Copyright (C) 2009-2015 Johannes Dewender +# Copyright (C) 2021 Jim Patterson +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . +"""This is a tool to submit ISRCs from audio files to MusicBrainz. + +Mutagen is used to gather the ISRCs +and python-musicbrainz2 to submit them. +The project is hosted on +https://github.com/SheamusPatt/musicbrainz-isrcDigitalSubmit +""" + +import operator +import re + +from unidecode import unidecode + +import isrcshared + +AGENT_NAME = "isrcDigitalSubmit.py" + +import glob +import logging +import musicbrainzngs +import mutagen +import os +import sys +import zipfile +from musicbrainzngs import ResponseError, WebServiceError +from optparse import OptionParser + +from isrcshared import __version__, open_browser, config_path, \ + user_input, printf, print_encoded, print_error, print_release, \ + setDefaultOptions + +# Maximum time difference between MB and audio files, +# else they will be considered different +max_time_diff = 3 + +try: + from configparser import ConfigParser +except ImportError: + from ConfigParser import ConfigParser + +if os.name == "nt": + SHELLNAME = "isrcDigitalSubmit.bat" +else: + SHELLNAME = "isrcDigitalSubmit.sh" +if os.path.isfile(SHELLNAME): + SCRIPTNAME = SHELLNAME +else: + SCRIPTNAME = os.path.basename(sys.argv[0]) + + +# global variables +ws2 = None +logger = logging.getLogger("isrcDigitalSubmit") + + +def script_version(): + return "isrcDigitalSubmit %s by SheamusPatt" % (__version__) + + +def print_help(option=None, opt=None, value=None, parser=None): + print("%s" % script_version()) + print( +""" +This python script extracts ISRCs from audio files and submits them +to MusicBrainz (musicbrainz.org). You need to have a MusicBrainz account, +specify the username and will be asked for your password when you +submit ISRCs (keyring may be used to retain credentials between invocations). + +IsrcDigitalSubmit will warn you if there are any problems and won't actually +submit anything to MusicBrainz without giving a final choice. + +IsrcDigitalSubmit will warn you if any duplicate ISRCs are detected and help +you fix previously inserted duplicate ISRCs. The ISRC-track relationship we +found in the audio files is taken as our correct evaluation. +""") + parser.print_usage() + print("""\ +Please report bugs on https://github.com/SheamusPatt/musicbrainz-isrcsubmit""") + sys.exit(0) + + +def print_usage(option=None, opt=None, value=None, parser=None): + print("%s\n" % script_version()) + parser.print_help() + sys.exit(0) + + +class Isrc(object): + def __init__(self, isrc, track=None): + self._id = isrc + self._tracks = [] + if track is not None: + self._tracks.append(track) + + def add_track(self, track): + if track not in self._tracks: + self._tracks.append(track) + + def get_tracks(self): + return self._tracks + + def get_track_numbers(self): + numbers = [] + for track in self._tracks: + numbers.append(track["position"]) + return ", ".join(numbers) + + +trackNumMatcher = re.compile(r'(^|[/ -])(\d\d)\D[^/]+$') + + +class Track(dict): + """track with equality checking + + This makes it easy to check if this track is already in a collection. + Only the element already in the collection needs to be hashable. + """ + def __init__(self, track, trackName): + self._recording = track + self._trackName = trackName + self._number = None + if track.__class__ == mutagen.oggvorbis.OggVorbis or track.__class__ == mutagen.flac.FLAC : + if not self._number: + if track.get("tracknumber") and track.get("tracknumber")[0]: + self._number = int(track.get("tracknumber")[0]) + if (track.get('artist')): + self._artist = track.get('artist')[0] + else: + self._artist = None + if (track.get('albumartist')): + self._albumartist = track.tags.get('albumartist')[0] + else: + self._albumartist = None + self._album = track.get("album")[0] if track.get("album") else None + self._title = track.get("title")[0] if track.get("title") else None + self._isrc = track.get("ISRC") if track.get("ISRC") \ + else track.get("93a74bea-ce97-5571-a56a-c5084dba9873") + elif track.__class__ == mutagen.mp3.MP3: + if not self._number: + trck = track.get("TRCK") or track.get("TRK") + if trck: + self._number = int(trck.text[0].partition("/")[0]) + if track.get("TPE1"): + self._artist = track.get("TPE1")[0] + elif track.get("TP1"): + self._artist = track.get("TP1")[0] + else: + self._artist = None + self._albumartist = None + if track.get("TALB"): + self._album = track.get("TALB")[0] + elif track.get("TAL"): + self._album = track.get("TAL")[0] + else: + self._album = None + if track.get("TIT2"): + self._title = track.get("TIT2")[0] + elif track.get("TT2"): + self._title = track.get("TT2")[0] + else: + self._title = None + self._isrc = track.get("TSRC") + elif track.__class__ == mutagen.mp4.MP4: + if not self._number: + trck = track.get("trkn") + if trck: + self._number = trck[0][0] + if track.get("\xa9ART"): + self._artist = track.get("\xa9ART")[0] + else: + self._artist = None + if track.get("aART"): + self._albumartist = track.get("aART")[0] + else: + self._albumartist = None + if track.get("\xa9alb"): + self._album = track.get("\xa9alb")[0] + else: + self._album = None + if track.get("\xa9nam"): + self._title = track.get("\xa9nam")[0] + else: + self._title = None + self._isrc = track.get("----:com.apple.iTunes:ISRC") + if (self._isrc and len(self._isrc)>0): + # The M4A class wraps ISRC values - we need to convert + # them to simple strings. + self._isrc = (str(self._isrc[0],'UTF-8'),) + if not self._number: + match = trackNumMatcher.search(trackName) + if match: + self._number = match.group(2) + else: + print_error("All tracks must have a Track Number. None found on {}" + .format(trackName)) + + def __eq__(self, other): + return self["id"] == other["id"] + + def __getitem__(self, item): + return self._recording[item] + + def get(self, item, default=None): + return self._recording.get(item, default) + + +def gather_options(argv): + global options + + config = ConfigParser() + config.read(config_path()) + + parser = OptionParser(version=script_version(), add_help_option=False) + parser.set_usage( + "{prog} [options] [user] audioFile...\n {prog} -h".format( + prog=SCRIPTNAME)) + parser.add_option("-h", action="callback", callback=print_usage, + help="Short usage help") + parser.add_option("--help", action="callback", callback=print_help, + help="Complete help for the script") + parser.add_option("-u", "--user", metavar="USERNAME", + help="MusicBrainz username, if not given as argument.") + parser.add_option("--release-id", metavar="RELEASE_ID", + help="Optional MusicBrainz ID of the release." + + " This will be gathered if not given.") + parser.add_option("--browser", metavar="BROWSER", + help="Program to open URLs. This will be automatically detected" + " for most setups, if not chosen manually.") + parser.add_option("--force-submit", action="store_true", default=False, + help="Always open TOC/disc ID in browser.") + parser.add_option("--server", metavar="SERVER", + help="Server to send ISRCs to. Default: %s" % isrcshared.DEFAULT_SERVER) + parser.add_option("--debug", action="store_true", default=False, + help="Show debug messages." + + " Currently shows some backend messages.") + parser.add_option("--keyring", action="store_true", dest="keyring", + help="Use keyring if available.") + parser.add_option("--no-keyring", action="store_false", dest="keyring", + help="Disable keyring.") + (options, args) = parser.parse_args(argv[1:]) + + print("%s" % script_version()) + + # assign positional arguments to options + if options.user is None and args: + options.user = args[0] + args = args[1:] + if not args: + print_error("No audio files to process") + sys.exit(1) + options.audioFiles = [] + for arg in args: + for f in glob.glob(arg): + options.audioFiles.append(f) + + setDefaultOptions(config, options) + + return options + + +class WebService2(isrcshared.WebService2): + """A web service wrapper that asks for a password when first needed. + + This uses musicbrainzngs as a wrapper itself. + """ + + def __init__(self, username=None): + isrcshared.WebService2.__init__(self, username) + musicbrainzngs.set_useragent(AGENT_NAME, __version__, + "http://github.com/SheamusPatt/musicbrainz-isrcsubmit") + + def tracks_match(self, mb_tracks, tracks): + if len(mb_tracks) != len(tracks): + return False + mbIter = mb_tracks.iter() + trackIter = tracks.iter() + mbTrack = mbIter.next() + while mbTrack: + track = trackIter.next() + if abs(track.info.length - mbTrack.length) > max_time_diff: + return False + if mbTrack.get("artist").lower() != track._artist.lower(): + return False + if mbTrack.get("album").lower() != track._album.lower(): return False + + return True + + def get_releases_by_name(self, title, tracks, extra=""): + query = '{} AND tracks:{}{}'.format(title, len(tracks), extra); + try: + response = musicbrainzngs.search_releases(query, strict=True) + except ResponseError as err: + if err.cause.code == 404: + return [] + else: + print_error("Couldn't fetch release: %s" % err) + sys.exit(1) + except WebServiceError as err: + print_error("Couldn't fetch release: %s" % err) + sys.exit(1) + return response['release-list'] + + +def gather_tracks(audioFiles): + """read the disc in the device with the backend and extract the ISRCs + """ + tracks = [] + + for file in audioFiles: + if zipfile.is_zipfile(file): + zf = zipfile.ZipFile(file) + for info in zf.infolist(): + if not info.is_dir() and not info.filename.startswith('__MACOSX/'): + # Ignore AppleDouble resource files + member = zf.open(info.filename) + try: + track = mutagen.File(member) + if (track): + tracks.append(Track(track, info.filename)) + else: + printf("Ignoring non-music member %s\n" % info.filename) + except Exception as e: + print_error("Failed to load track {}: {}".format(info.filename, e)) + else: + try: + track = mutagen.File(file) + if (track): + tracks.append(Track(track, file)) + else: + printf("Ignoring non-music file %s\n" % file) + except Exception as e: + print_error("Failed to load track {}: {}".format(file, e)) + tracks.sort(key=operator.attrgetter("_number")) + return tracks + + +"""Return an ASCII version of str with all punctuation replaced by spaces +""" +def simplestr(str): + result = unidecode(str) + result = result.lower() + words = re.split(r'\W+', result) + return " ".join(words) + + +"""Do a fuzzy compare, ignoring case, accents and punctuation. +Returns true if strings are the same under those conditions. +""" +def fuzzycompare(str1, str2): + return simplestr(str1) == simplestr(str2) + + +def check_isrcs_local(tracks, mb_tracks): + """check tracls for (local) duplicates and inconsistencies + Also verify that local tracks match those in MusicBrainz release + """ + isrcs = dict() # isrcs found on disc + tracks2isrcs = dict() # isrcs to be submitted + errors = 0 + + for track in tracks: + track_number = track._number + if track._isrc == None or not track._isrc[0]: + print("Track %s %s has no ISRC" % (track_number, track._title)) + elif len(track._isrc) > 1: + print_error("Track %s %s has multiple ISRCs. Has this file been tagged already?" + % (track_number, track._title)) + errors += 1 + else: + isrc = track._isrc[0] + mb_track = mb_tracks[track_number - 1] + if not fuzzycompare(track._artist, mb_track["artist-credit-phrase"])\ + and track._artist.lower() != "various artists": + print_error("Track {} credited to {} which does not match credit in MusicBrainz: {}" + .format(track_number, track._artist, mb_track["artist-credit-phrase"])) + errors += 1 + if not fuzzycompare(track._title, mb_track["recording"]["title"]): + print_error("Track {} title {} does not match that in MusicBrainz: {}" + .format(track_number, track._title, mb_track["recording"]["title"])) + errors += 1 + if abs(track._recording.info.length - float(mb_track["recording"]["length"])/1000)\ + > max_time_diff: + print_error("Track {} recording length {} more than {} seconds different than length in MusicBrainz: {}" + .format(track_number, track._recording.info.length, + max_time_diff, float(mb_track["recording"]["length"])/1000)) + errors += 1 + if isrc in isrcs: + # found this ISRC for multiple tracks + isrcs[isrc].add_track(mb_track) + else: + isrcs[isrc] = Isrc(isrc, mb_track) + # check if the ISRC was already added to the track + isrc_attached = False + mbIsrcList = mb_track["recording"].get("isrc-list") + if mbIsrcList: + for mb_isrc in mbIsrcList: + if isrc == mb_isrc: + print("%s is already attached to track %d" + % (isrc, track_number)) + isrc_attached = True + break + if not isrc_attached: + # single isrcs work in python-musicbrainzngs 0.4, but not 0.3 + # lists of isrcs don't work in 0.4 though, see pymbngs #113 + tracks2isrcs[mb_track.get("recording")["id"]] = isrc + print("found new ISRC for track %d: %s" + % (track_number, isrc)) + + # Check for multiple ISRCs on different tracks + for isrc in isrcs.items(): + if len(isrc[1].get_tracks()) > 1: + print("ISRC %s found on more than one track" % isrc._id) + for track in isrc.get_tracks(): + print("-> %s" % track) + errors += 1 + + return isrcs, tracks2isrcs, errors + + +def check_global_duplicates(release, mb_tracks, isrcs): + """Help cleaning up global duplicates of any of the isrcs from this digital release + """ + duplicates = [] + # add already attached ISRCs + for isrc in isrcs: + recordings = ws2.get_recordings_by_isrc(isrc) + if len(recordings) > 1: + duplicates.append(isrc._id) + + if len(duplicates) > 0: + printf("\nThere were %d ISRCs ", len(duplicates)) + print("that are attached to multiple tracks.") + choice = user_input("Do you want to help clean those up? [y/N] ") + if choice.lower() == "y": + cleanup_isrcs(release, duplicates) + + +def cleanup_isrcs(release, isrcs): + """Show information about duplicate ISRCs + + Our attached ISRCs should be correct -> helps to delete from other tracks + """ + global options + for isrc in isrcs: + tracks = isrcs[isrc].get_tracks() + print("\nISRC %s attached to:" % isrc) + for track in tracks: + printf("\t") + artist = track.get("artist-credit-phrase") + if artist and artist != release.get("artist-credit-phrase"): + string = "%s - %s" % (artist, track["title"]) + else: + string = "%s" % track["title"] + print_encoded(string) + # tab alignment + if len(string) >= 32: + printf("\n%s", " " * 40) + else: + if len(string) < 7: + printf("\t") + if len(string) < 15: + printf("\t") + if len(string) < 23: + printf("\t") + if len(string) < 31: + printf("\t") + + printf("\t track %s", track["position"]) + + url = "http://%s/isrc/%s" % (options.server, isrc) + if user_input("Open ISRC in the browser? [Y/n] ").lower() != "n": + open_browser(url) + user_input("(press when done with this ISRC) ") + +FEATURED_ARTIST_SEP = [' feat. ', ' duet with '] + +def find_release(tracks, common_includes): + global options + global ws2 + albumartists = set() + artists = set() + albums = set() + for track in tracks: + if (track._artist): + artists.add(track._artist.strip()) + if (track._albumartist): + albumartists.add(track._albumartist.strip()) + albumtitle = track._album + if not albumtitle: + print_error("Track {} is missing an album name. \ +We cannot search for an album unless all tracks have an album title".format(track._number)) + sys.exit(1) + albums.add(albumtitle.strip().lower()) + if len(albumartists)>1: + print("Release should have just one Album Artist, found %d: %s", len(artists), str(artists)) + sys.exit(1) + elif len(albumartists)==1: + albumartist = albumartists.pop() + elif len(artists) == 1: + albumartist = artists.pop() + else: + # Might still be a single artist album with 'featured' artists. Look for a common prefix + # after breaking at common featured artist tags + albumartist = None + for otherartist in artists: + for feat in FEATURED_ARTIST_SEP: + otherartist = otherartist.partition(feat)[0] + if not albumartist or len(otherartist) < len(albumartist): + albumartist = otherartist + for otherartist in artists: + if len(albumartist) > len(otherartist) and albumartist.startswith(otherartist): + albumartist = otherartist + elif not otherartist.startswith(albumartist): + albumartist = 'Various Artists' + break + + if len(albums)>1: + print("Release should have just one Album, found %d: %s" % (len(albums), str(albums))) + sys.exit(1) + # We have a unique artist and album. See if we can locate it in MB + formatQuery = ' AND format:"Digital Media"' + if albumartist: + artistQuery = ' AND artist:"{}"'.format(albumartist) + results = ws2.get_releases_by_name(albumtitle, tracks, artistQuery + formatQuery) + if len(results) == 0: + results = ws2.get_releases_by_name(albumtitle, tracks, artistQuery) + if len(results) > 0: + print_error("Found a release but format is not Digital Media") + print_release(results[0]) + sys.exit(1) + else: + results = [] + if len(results) == 0: + # Try without the artist name. This can find Various Artists releases under odd names + # or cases where the artist on tracks doesn't match MusicBrainz (it happens) + results = ws2.get_releases_by_name(albumtitle, tracks, formatQuery) + if len(results) == 0: + results = ws2.get_releases_by_name(albumtitle, tracks) + if len(results) > 0: + print_error("Found a release but format is not Digital Media") + print_release(results[0]) + sys.exit(1) + num_results = len(results) + if num_results == 0: + print("\n\"Cannot find {}\" by \"{}\" with {} tracks in the database." + .format(albumartist, albumtitle, len(tracks))) + sys.exit(1) + elif num_results > 1: + print("\nMultiple releases found for '{0}' ({1} tracks) by '{2}', please pick one:" + .format(albumtitle, len(tracks), albumartist)) + print(" 0: none of these\n") + for i in range(num_results): + release = results[i] + # printed list is 1..n, not 0..n-1 ! + print_release(release, i + 1) + try: + num = user_input("Which one do you want? [0-%d] " + % num_results) + if int(num) not in range(0, num_results + 1): + raise IndexError + if int(num) == 0: + print("Release not found. You can resubmit using the --release-id option\n") + sys.exit(1) + else: + selected_release = results[int(num) - 1] + except (ValueError, IndexError): + print_error("Invalid Choice") + sys.exit(1) + except KeyboardInterrupt: + print("\nexiting..") + sys.exit(1) + else: + print("Found unique release\n") + selected_release = results[0] + print_release(selected_release) + + return ws2.get_release_by_id(selected_release["id"], common_includes) + + +def main(argv): + global options + global ws2 + common_includes = ["artists", "labels", "recordings", "isrcs", + "artist-credits"] + # preset logger + stream_handler = logging.StreamHandler() + stream_handler.setFormatter(logging.Formatter("%(levelname)s: %(message)s")) + logging.getLogger().addHandler(stream_handler) # add to root handler + + # global variables + options = isrcshared.options = gather_options(argv) + + ws2 = WebService2(options.user) + + if options.debug: + logging.getLogger().setLevel(logging.DEBUG) + stream_handler.setLevel(logging.INFO) + + # adding log file + logfile = "isrcDigitalSubmit.log" + file_handler = logging.FileHandler(logfile, mode='w', + encoding="utf8", delay=True) + formatter = logging.Formatter("%(levelname)s:%(name)s: %(message)s") + file_handler.setFormatter(formatter) + file_handler.setLevel(logging.DEBUG) + logger.info("Writing debug log to %s", logfile) + logging.getLogger().addHandler(file_handler) + + # add context to log file (DEBUG only added there) + logger.debug(script_version()) + + tracks = gather_tracks(options.audioFiles) + if (len(tracks) == 0): + print_error("No valid tracks found") + sys.exit(1) + if (options.release_id): + release = ws2.get_release_by_id(options.release_id, common_includes) + print_release(release.get('release')) + else: + release = find_release(tracks, common_includes) + + # list, dict + mb_tracks = release.get('release')['medium-list'][0]['track-list'] + isrcs, tracks2isrcs, errors = check_isrcs_local(tracks, mb_tracks) + + if isrcs: + print("") + # try to submit the ISRCs + update_intention = True + if not tracks2isrcs: + print("No new ISRCs could be found.") + else: + if errors > 0: + print_error("%d problems detected" % errors) + if user_input("Do you want to submit? [y/N] ").lower() == "y": + ws2.submit_isrcs(tracks2isrcs) + else: + update_intention = False + print("Nothing was submitted to the server.") + + # check for overall duplicate ISRCs, including server provided + if update_intention: + # the ISRCs are deemed correct, so we can use them to check others + check_global_duplicates(release, mb_tracks, isrcs) + +if __name__ == "__main__": + main(sys.argv) + + +# vim:set shiftwidth=4 smarttab expandtab: diff --git a/isrcDigitalSubmit.sh b/isrcDigitalSubmit.sh new file mode 100755 index 0000000..62bc1ba --- /dev/null +++ b/isrcDigitalSubmit.sh @@ -0,0 +1,4 @@ +#!/bin/sh + +export PATH=$PATH:. +exec ./isrcDigitalSubmit.py $@ diff --git a/isrcshared.py b/isrcshared.py new file mode 100644 index 0000000..e7db162 --- /dev/null +++ b/isrcshared.py @@ -0,0 +1,383 @@ +# Copyright (C) 2009-2015 Johannes Dewender +# +# This program is free software: you can redistribute it and/or modify +# it under the terms of the GNU General Public License as published by +# the Free Software Foundation, either version 3 of the License, or +# (at your option) any later version. +# +# This program is distributed in the hope that it will be useful, +# but WITHOUT ANY WARRANTY; without even the implied warranty of +# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the +# GNU General Public License for more details. +# +# You should have received a copy of the GNU General Public License +# along with this program. If not, see . + +# This module contains shared functionality used by both +# the CD based tool isrcsubmit +# and the audio file tool isrcDigitalSubmit. +import logging +import os +import sys +import codecs +import webbrowser +import getpass +from subprocess import Popen, PIPE, call + +__version__ = "2.2.2" + +import musicbrainzngs +from musicbrainzngs import AuthenticationError, WebServiceError + +DEFAULT_SERVER = "musicbrainz.org" +BROWSERS = ["xdg-open", "x-www-browser", + "firefox", "chromium", "chrome", "opera"] +# The webbrowser module is used when nothing is found in this list. +# This especially happens on Windows and Mac OS X (browser mostly not in PATH) + +options = None + +try: + import keyring +except ImportError: + keyring = None + +# make code run on Python 2 and 3 +try: + user_input = raw_input +except NameError: + user_input = input + +try: + unicode_string = unicode +except NameError: + unicode_string = str + +logger = logging.getLogger("isrcsubmit") + +def cp65001(name): + """This might be buggy, but better than just a LookupError + """ + if name.lower() == "cp65001": + return codecs.lookup("utf-8") + + +codecs.register(cp65001) + +def test_which(): + """There are some old/buggy "which" versions on Windows. + We want to know if the user has a "sane" which we can trust. + Unxutils has a broken 2.4 version. Which >= 2.16 should be fine. + """ + with open(os.devnull, "w") as devnull: + try: + # "which" should at least find itself + return_code = call(["which", "which"], + stdout=devnull, stderr=devnull) + except OSError: + return False # no which at all + else: + if (return_code == 0): + return True + else: + print('warning: your version of the tool "which"' + ' is buggy/outdated') + if os.name == "nt": + print(' unxutils is old/broken, GnuWin32 is good.') + return False + + +def has_program(program, choices, strict=False): + """When the backend is only a symlink to another backend, + we will return False, unless we strictly want to use this backend. + """ + if not hasattr(has_program, "sane_which"): + has_program.sane_which = test_which() + with open(os.devnull, "w") as devnull: + if has_program.sane_which: + p_which = Popen(["which", program], stdout=PIPE, stderr=devnull) + program_path = p_which.communicate()[0].strip() + if p_which.returncode == 0: + # check if it is only a symlink to another backend + real_program = os.path.basename(os.path.realpath(program_path)) + if program != real_program and ( + real_program in choices): + if strict: + print("WARNING: %s is a symlink to %s" + % (program, real_program)) + return True + else: + return False # use real program (target) instead + return True + else: + return False + elif program in choices: + try: + # we just try to start these non-interactive console apps + call([program], stdout=devnull, stderr=devnull) + except OSError: + return False + else: + return True + else: + return False + + +class WebService2(): + """A web service wrapper that asks for a password when first needed. + + This uses musicbrainzngs as a wrapper itself. + """ + + def __init__(self, username=None): + self.auth = False + self.keyring_failed = False + self.username = username + self.server = options.server + self.keyring = options.keyring + musicbrainzngs.set_hostname(options.server) + + def authenticate(self): + """Sets the password if not set already + """ + if not self.auth: + print("") + if self.username is None: + printf("Please input your MusicBrainz username (empty=abort): ") + self.username = user_input() + if len(self.username) == 0: + print("(aborted)") + sys.exit(1) + password = None + if keyring is not None and self.keyring and not self.keyring_failed: + password = keyring.get_password(self.server, self.username) + if password is None: + password = getpass.getpass( + "Please input your MusicBrainz password: ") + print("") + musicbrainzngs.auth(self.username, password) + self.auth = True + self.keyring_failed = False + if keyring is not None and self.keyring: + keyring.set_password(self.server, self.username, password) + + def get_release_by_id(self, release_id, includes=[]): + try: + return musicbrainzngs.get_release_by_id(release_id, + includes=includes) + except WebServiceError as err: + print_error("Couldn't fetch release: %s" % err) + sys.exit(1) + + def get_recordings_by_isrc(self, isrc, includes=[]): + try: + response = musicbrainzngs.get_recordings_by_isrc(isrc, includes) + except WebServiceError as err: + print_error("Couldn't fetch recordings for isrc %s: %s" % (isrc, err)) + sys.exit(1) + return response.get('isrc')['recording-list'] + + def submit_isrcs(self, tracks2isrcs): + logger.info("tracks2isrcs: %s", tracks2isrcs) + while True: + try: + self.authenticate() + musicbrainzngs.submit_isrcs(tracks2isrcs) + except AuthenticationError as err: + print_error("Invalid credentials: %s" % err) + self.auth = False + self.keyring_failed = True + self.username = None + continue + except WebServiceError as err: + print_error("Couldn't send ISRCs: %s" % err) + sys.exit(1) + else: + print("Successfully submitted %d ISRCS." % len(tracks2isrcs)) + break + + + +def printf(format_string, *args): + """Print with the % and without additional spaces or newlines + """ + if not args: + # make it convenient to use without args -> different to C + args = (format_string, ) + format_string = "%s" + sys.stdout.write(format_string % args) + + +def decode(msg): + """This will replace unsuitable characters and use stdin encoding + """ + if isinstance(msg, bytes): + return msg.decode(sys.stdin.encoding, "replace") + else: + return unicode_string(msg) + + +def encode(msg): + """This will replace unsuitable characters and use stdout encoding + """ + if isinstance(msg, unicode_string): + return msg.encode(sys.stdout.encoding, "replace") + else: + return bytes(msg) + + +def print_encoded(*args): + """This will replace unsuitable characters and doesn't append a newline + """ + stringArgs = () + for arg in args: + stringArgs += encode(arg), + msg = b" ".join(stringArgs) + if not msg.endswith(b"\n"): + msg += b" " + if os.name == "nt": + os.write(sys.stdout.fileno(), msg) + else: + try: + sys.stdout.buffer.write(msg) + except AttributeError: + sys.stdout.write(msg) + + +def print_release(release, position=None): + """Print information about a release. + + If the position is given, this should be an entry + in a list of releases (choice) + """ + country = (release.get("country") or "").ljust(2) + date = (release.get("date") or "").ljust(10) + barcode = (release.get("barcode") or "").rjust(13) + label_list = release.get("label-info-list") or [] + catnumber_list = [] + for label in label_list: + cat_number = label.get("catalog-number") + if cat_number: + catnumber_list.append(cat_number) + catnumbers = ", ".join(catnumber_list) + + if position is None: + print_encoded("Artist:\t\t%s\n" % release["artist-credit-phrase"]) + print_encoded("Release:\t%s" % release["title"]) + else: + print_encoded("%#2d:" % position) + print_encoded("%s - %s" % ( + release["artist-credit-phrase"], release["title"])) + if release.get("status"): + print("(%s)" % release["status"]) + else: + print("") + if position is None: + print_encoded("Release Event:\t%s\t%s\n" % (date, country)) + print_encoded("Barcode:\t%s\n" % release.get("barcode") or "") + print_encoded("Catalog No.:\t%s\n" % catnumbers) + print_encoded("MusicBrainz ID:\t%s\n" % release["id"]) + else: + print_encoded("\t%s\t%s\t%s\t%s\n" % ( + country, date, barcode, catnumbers)) + + +def print_error(*args): + string_args = tuple([str(arg) for arg in args]) + logger.error("\n ".join(string_args)) + + +def setDefaultOptions(config, options): + # If an option is set in the config and not overriden on the command line, + # assign them to options. + if options.keyring is None and config.has_option("general", "keyring"): + options.keyring = config.getboolean("general", "keyring") + if options.browser is None and config.has_option("general", "browser"): + options.browser = config.get("general", "browser") + if options.server is None and config.has_option("musicbrainz", "server"): + options.server = config.get("musicbrainz", "server") + if options.user is None and config.has_option("musicbrainz", "user"): + options.user = config.get("musicbrainz", "user") + options.sane_which = test_which() + if options.browser is None: + options.browser = find_browser() + if options.server is None: + options.server = DEFAULT_SERVER + if options.keyring is None: + options.keyring = True + + +def get_config_home(tool="isrcsubmit"): + """Returns the base directory for isrcsubmit's configuration files.""" + + if os.name == "nt": + default_location = os.environ.get("APPDATA") + else: + default_location = os.path.expanduser("~/.config") + + xdg_config_home = os.environ.get("XDG_CONFIG_HOME", default_location) + return os.path.join(xdg_config_home, tool) + + +def config_path(tool="isrcsubmit"): + """Returns isrsubmit's config file location.""" + return os.path.join(get_config_home(tool), "config") + + +def find_browser(): + """search for an available browser + """ + for browser in BROWSERS: + if has_program(browser, BROWSERS): + return browser + + # This will use the webbrowser module to find a default + return None + + +def open_browser(url, exit=False, submit=False): + """open url in the selected browser, default if none + """ + if options.browser: + if exit: + try: + if os.name == "nt": + # silly but necessary for spaces in the path + os.execlp(options.browser, '"' + options.browser + '"', url) + else: + # linux/unix works fine with spaces + os.execlp(options.browser, options.browser, url) + except OSError as err: + error = ["Couldn't open the url in %s: %s" + % (options.browser, str(err))] + if submit: + error.append("Please submit via: %s" % url) + print_error(*error) + sys.exit(1) + else: + try: + if options.debug: + Popen([options.browser, url]) + else: + with open(os.devnull, "w") as devnull: + Popen([options.browser, url], stdout=devnull) + except OSError as err: + error = ["Couldn't open the url in %s: %s" + % (options.browser, str(err))] + if submit: + error.append("Please submit via: %s" % url) + print_error(*error) + else: + try: + if options.debug: + webbrowser.open(url) + else: + # this supresses stdout + webbrowser.get().open(url) + except webbrowser.Error as err: + error = ["Couldn't open the url: %s" % str(err)] + if submit: + error.append("Please submit via: %s" % url) + print_error(*error) + if exit: + sys.exit(1) diff --git a/isrcsubmit.py b/isrcsubmit.py index 1bd5eb7..3c31572 100755 --- a/isrcsubmit.py +++ b/isrcsubmit.py @@ -21,27 +21,24 @@ https://github.com/JonnyJD/musicbrainz-isrcsubmit """ -__version__ = "2.1.0" AGENT_NAME = "isrcsubmit.py" -DEFAULT_SERVER = "musicbrainz.org" # starting with highest priority BACKENDS = ["mediatools", "media_info", "cdrdao", "libdiscid", "discisrc"] -BROWSERS = ["xdg-open", "x-www-browser", - "firefox", "chromium", "chrome", "opera"] -# The webbrowser module is used when nothing is found in this list. -# This especially happens on Windows and Mac OS X (browser mostly not in PATH) + import os import re import sys -import codecs import logging -import getpass import tempfile -import webbrowser from datetime import datetime from optparse import OptionParser -from subprocess import Popen, PIPE, call +from subprocess import Popen, PIPE +import isrcshared +from isrcshared import __version__, print_error, decode, \ + printf, user_input, open_browser, print_release, \ + has_program, logger, print_encoded, WebService2, \ + config_path, setDefaultOptions, encode try: import discid @@ -55,12 +52,7 @@ import discid import musicbrainzngs -from musicbrainzngs import AuthenticationError, ResponseError, WebServiceError - -try: - import keyring -except ImportError: - keyring = None +from musicbrainzngs import ResponseError, WebServiceError try: from configparser import ConfigParser @@ -76,28 +68,18 @@ else: SCRIPTNAME = os.path.basename(sys.argv[0]) -# make code run on Python 2 and 3 -try: - user_input = raw_input -except NameError: - user_input = input - -try: - unicode_string = unicode -except NameError: - unicode_string = str - # global variables options = None ws2 = None -logger = logging.getLogger("isrcsubmit") + def script_version(): return "isrcsubmit %s by JonnyJD for MusicBrainz" % __version__ + def print_help(option=None, opt=None, value=None, parser=None): print("%s" % script_version()) - print(\ + print( """ This python script extracts ISRCs from audio cds and submits them to MusicBrainz (musicbrainz.org). You need to have a MusicBrainz account, specify the username and will be asked for your password every time you execute the script. @@ -112,11 +94,13 @@ def print_help(option=None, opt=None, value=None, parser=None): Please report bugs on https://github.com/JonnyJD/musicbrainz-isrcsubmit""") sys.exit(0) + def print_usage(option=None, opt=None, value=None, parser=None): print("%s\n" % script_version()) parser.print_help() sys.exit(0) + class Isrc(object): def __init__(self, isrc, track=None): self._id = isrc @@ -166,25 +150,11 @@ def get(self, item, default=None): except KeyError: return self._track.get(item, default) + class OwnTrack(Track): """A track found on an analyzed (own) disc""" pass -def get_config_home(): - """Returns the base directory for isrcsubmit's configuration files.""" - - if os.name == "nt": - default_location = os.environ.get("APPDATA") - else: - default_location = os.path.expanduser("~/.config") - - xdg_config_home = os.environ.get("XDG_CONFIG_HOME", default_location) - return os.path.join(xdg_config_home, "isrcsubmit") - -def config_path(): - """Returns isrsubmit's config file location.""" - - return os.path.join(get_config_home(), "config") def gather_options(argv): global options @@ -227,7 +197,7 @@ def gather_options(argv): parser.add_option("--force-submit", action="store_true", default=False, help="Always open TOC/disc ID in browser.") parser.add_option("--server", metavar="SERVER", - help="Server to send ISRCs to. Default: %s" % DEFAULT_SERVER) + help="Server to send ISRCs to. Default: %s" % isrcshared.DEFAULT_SERVER) parser.add_option("--debug", action="store_true", default=False, help="Show debug messages." + " Currently shows some backend messages.") @@ -249,36 +219,21 @@ def gather_options(argv): if args: logger.warning("Superfluous arguments: %s", ", ".join(args)) - # If an option is set in the config and not overriden on the command line, - # assign them to options. - if options.keyring is None and config.has_option("general", "keyring"): - options.keyring = config.getboolean("general", "keyring") + setDefaultOptions(config, options) + + # assign remaining options automatically + if options.device is None and config.has_option("general", "device"): + options.device = config.get("general", "device") + if options.device is None: + options.device = default_device + if options.backend is None and config.has_option("general", "backend"): options.backend = config.get("general", "backend") if options.backend not in BACKENDS: print_error("Backend given in config file is not a valid choice.", "Choose a backend from %s" % ", ".join(BACKENDS)) sys.exit(-1) - if options.browser is None and config.has_option("general", "browser"): - options.browser = config.get("general", "browser") - if options.device is None and config.has_option("general", "device"): - options.device = config.get("general", "device") - if options.server is None and config.has_option("musicbrainz", "server"): - options.server = config.get("musicbrainz", "server") - if options.user is None and config.has_option("musicbrainz", "user"): - options.user = config.get("musicbrainz", "user") - - # assign remaining options automatically - if options.device is None: - options.device = default_device - options.sane_which = test_which() - if options.browser is None: - options.browser = find_browser() - if options.server is None: - options.server = DEFAULT_SERVER - if options.keyring is None: - options.keyring = True - if options.backend and not has_program(options.backend, strict=True): + if options.backend and not has_program(options.backend, BACKENDS, strict=True): print_error("Chosen backend not found. No ISRC extraction possible!", "Make sure that %s is installed." % options.backend) sys.exit(-1) @@ -288,28 +243,6 @@ def gather_options(argv): return options -def test_which(): - """There are some old/buggy "which" versions on Windows. - We want to know if the user has a "sane" which we can trust. - Unxutils has a broken 2.4 version. Which >= 2.16 should be fine. - """ - with open(os.devnull, "w") as devnull: - try: - # "which" should at least find itself - return_code = call(["which", "which"], - stdout=devnull, stderr=devnull) - except OSError: - return False # no which at all - else: - if (return_code == 0): - return True - else: - print('warning: your version of the tool "which"' - ' is buggy/outdated') - if os.name == "nt": - print(' unxutils is old/broken, GnuWin32 is good.') - return False - def get_prog_version(prog): if prog == "libdiscid": version = discid.LIBDISCID_VERSION_STRING @@ -321,48 +254,15 @@ def get_prog_version(prog): return decode(version) -def has_program(program, strict=False): - """When the backend is only a symlink to another backend, - we will return False, unless we strictly want to use this backend. - """ - if program == "libdiscid": - return "isrc" in discid.FEATURES - - with open(os.devnull, "w") as devnull: - if options.sane_which: - p_which = Popen(["which", program], stdout=PIPE, stderr=devnull) - program_path = p_which.communicate()[0].strip() - if p_which.returncode == 0: - # check if it is only a symlink to another backend - real_program = os.path.basename(os.path.realpath(program_path)) - if program != real_program and ( - real_program in BACKENDS or real_program in BROWSERS): - if strict: - print("WARNING: %s is a symlink to %s" - % (program, real_program)) - return True - else: - return False # use real program (target) instead - return True - else: - return False - elif program in BACKENDS: - try: - # we just try to start these non-interactive console apps - call([program], stdout=devnull, stderr=devnull) - except OSError: - return False - else: - return True - else: - return False def find_backend(): """search for an available backend """ backend = None for prog in BACKENDS: - if has_program(prog): + if prog == "libdiscid": + return "isrc" in discid.FEATURES + elif has_program(prog, BACKENDS): backend = prog break @@ -374,62 +274,6 @@ def find_backend(): return backend -def find_browser(): - """search for an available browser - """ - for browser in BROWSERS: - if has_program(browser): - return browser - - # This will use the webbrowser module to find a default - return None - -def open_browser(url, exit=False, submit=False): - """open url in the selected browser, default if none - """ - if options.browser: - if exit: - try: - if os.name == "nt": - # silly but necessary for spaces in the path - os.execlp(options.browser, '"' + options.browser + '"', url) - else: - # linux/unix works fine with spaces - os.execlp(options.browser, options.browser, url) - except OSError as err: - error = ["Couldn't open the url in %s: %s" - % (options.browser, str(err))] - if submit: - error.append("Please submit via: %s" % url) - print_error(*error) - sys.exit(1) - else: - try: - if options.debug: - Popen([options.browser, url]) - else: - with open(os.devnull, "w") as devnull: - Popen([options.browser, url], stdout=devnull) - except OSError as err: - error = ["Couldn't open the url in %s: %s" - % (options.browser, str(err))] - if submit: - error.append("Please submit via: %s" % url) - print_error(*error) - else: - try: - if options.debug: - webbrowser.open(url) - else: - # this supresses stdout - webbrowser.get().open(url) - except webbrowser.Error as err: - error = ["Couldn't open the url: %s" % str(err)] - if submit: - error.append("Please submit via: %s" % url) - print_error(*error) - if exit: - sys.exit(1) def get_real_mac_device(option_device): """drutil takes numbers as drives. @@ -447,103 +291,13 @@ def get_real_mac_device(option_device): # libdiscid needs the "raw" version return given.replace("/disk", "/rdisk") -def cp65001(name): - """This might be buggy, but better than just a LookupError - """ - if name.lower() == "cp65001": - return codecs.lookup("utf-8") - -codecs.register(cp65001) - -def printf(format_string, *args): - """Print with the % and without additional spaces or newlines - """ - if not args: - # make it convenient to use without args -> different to C - args = (format_string, ) - format_string = "%s" - sys.stdout.write(format_string % args) - -def decode(msg): - """This will replace unsuitable characters and use stdin encoding - """ - if isinstance(msg, bytes): - return msg.decode(sys.stdin.encoding, "replace") - else: - return unicode_string(msg) - -def encode(msg): - """This will replace unsuitable characters and use stdout encoding - """ - if isinstance(msg, unicode_string): - return msg.encode(sys.stdout.encoding, "replace") - else: - return bytes(msg) - -def print_encoded(*args): - """This will replace unsuitable characters and doesn't append a newline - """ - stringArgs = () - for arg in args: - stringArgs += encode(arg), - msg = b" ".join(stringArgs) - if not msg.endswith(b"\n"): - msg += b" " - if os.name == "nt": - os.write(sys.stdout.fileno(), msg) - else: - try: - sys.stdout.buffer.write(msg) - except AttributeError: - sys.stdout.write(msg) - -def print_release(release, position=None): - """Print information about a release. - - If the position is given, this should be an entry - in a list of releases (choice) - """ - country = (release.get("country") or "").ljust(2) - date = (release.get("date") or "").ljust(10) - barcode = (release.get("barcode") or "").rjust(13) - label_list = release["label-info-list"] - catnumber_list = [] - for label in label_list: - cat_number = label.get("catalog-number") - if cat_number: - catnumber_list.append(cat_number) - catnumbers = ", ".join(catnumber_list) - - if position is None: - print_encoded("Artist:\t\t%s\n" % release["artist-credit-phrase"]) - print_encoded("Release:\t%s" % release["title"]) - else: - print_encoded("%#2d:" % position) - print_encoded("%s - %s" % ( - release["artist-credit-phrase"], release["title"])) - if release.get("status"): - print("(%s)" % release["status"]) - else: - print("") - if position is None: - print_encoded("Release Event:\t%s\t%s\n" % (date, country)) - print_encoded("Barcode:\t%s\n" % release.get("barcode") or "") - print_encoded("Catalog No.:\t%s\n" % catnumbers) - print_encoded("MusicBrainz ID:\t%s\n" % release["id"]) - else: - print_encoded("\t%s\t%s\t%s\t%s\n" % ( - country, date, barcode, catnumbers)) - -def print_error(*args): - string_args = tuple([str(arg) for arg in args]) - logger.error("\n ".join(string_args)) - def backend_error(err): print_error("Couldn't gather ISRCs with %s: %i - %s" % (options.backend, err.errno, err.strerror)) sys.exit(1) + def ask_for_submission(url, print_url=False): if options.force_submit: submit_requested = True @@ -557,43 +311,17 @@ def ask_for_submission(url, print_url=False): print("Please submit the Disc ID with this url:") print(url) -class WebService2(): + +class WebService2(isrcshared.WebService2): """A web service wrapper that asks for a password when first needed. This uses musicbrainzngs as a wrapper itself. """ def __init__(self, username=None): - self.auth = False - self.keyring_failed = False - self.username = username - musicbrainzngs.set_hostname(options.server) + isrcshared.WebService2.__init__(self, username) musicbrainzngs.set_useragent(AGENT_NAME, __version__, - "http://github.com/JonnyJD/musicbrainz-isrcsubmit") - - def authenticate(self): - """Sets the password if not set already - """ - if not self.auth: - print("") - if self.username is None: - printf("Please input your MusicBrainz username (empty=abort): ") - self.username = user_input() - if len(self.username) == 0: - print("(aborted)") - sys.exit(1) - password = None - if keyring is not None and options.keyring and not self.keyring_failed: - password = keyring.get_password(options.server, self.username) - if password is None: - password = getpass.getpass( - "Please input your MusicBrainz password: ") - print("") - musicbrainzngs.auth(self.username, password) - self.auth = True - self.keyring_failed = False - if keyring is not None and options.keyring: - keyring.set_password(options.server, self.username, password) + "http://github.com/SheamusPatt/musicbrainz-isrcsubmit") def get_releases_by_discid(self, disc_id, includes=[]): try: @@ -614,34 +342,6 @@ def get_releases_by_discid(self, disc_id, includes=[]): else: return [] - def get_release_by_id(self, release_id, includes=[]): - try: - return musicbrainzngs.get_release_by_id(release_id, - includes=includes) - except WebServiceError as err: - print_error("Couldn't fetch release: %s" % err) - sys.exit(1) - - def submit_isrcs(self, tracks2isrcs): - logger.info("tracks2isrcs: %s", tracks2isrcs) - while True: - try: - self.authenticate() - musicbrainzngs.submit_isrcs(tracks2isrcs) - except AuthenticationError as err: - print_error("Invalid credentials: %s" % err) - self.auth = False - self.keyring_failed = True - self.username = None - continue - except WebServiceError as err: - print_error("Couldn't send ISRCs: %s" % err) - sys.exit(1) - else: - print("Successfully submitted %d ISRCS." % len(tracks2isrcs)) - break - - class Disc(object): def read_disc(self): @@ -768,7 +468,6 @@ def select_release(self): return selected_release - def get_release(self, verified=False): """This will get a release the ISRCs will be added to. """ @@ -841,7 +540,7 @@ def gather_isrcs(disc, backend, device): except OSError as err: backend_error(err) for line in isrcout: - line = decode(line) # explicitely decode from pipe + line = decode(line) # explicitly decode from pipe ext_logger = logging.getLogger("discisrc") ext_logger.debug(line.rstrip()) # rstrip newline if line.startswith("Track") and len(line) > 12: @@ -869,7 +568,7 @@ def gather_isrcs(disc, backend, device): except OSError as err: backend_error(err) for line in isrcout: - line = decode(line) # explicitely decode from pipe + line = decode(line) # explicitly decode from pipe ext_logger = logging.getLogger("mediatools") ext_logger.debug(line.rstrip()) # rstrip newline if line.startswith("ISRC") and not line.startswith("ISRCS"): @@ -938,6 +637,7 @@ def gather_isrcs(disc, backend, device): devnull.close() return backend_output + def check_isrcs_local(backend_output, mb_tracks): """check backend_output for (local) duplicates and inconsistencies """ @@ -979,6 +679,7 @@ def check_isrcs_local(backend_output, mb_tracks): return isrcs, tracks2isrcs, errors + def check_global_duplicates(release, mb_tracks, isrcs): """Help cleaning up global duplicates with the information we got from our disc. @@ -1005,6 +706,7 @@ def check_global_duplicates(release, mb_tracks, isrcs): if choice.lower() == "y": cleanup_isrcs(release, isrcs) + def cleanup_isrcs(release, isrcs): """Show information about duplicate ISRCs @@ -1057,7 +759,7 @@ def main(argv): logging.getLogger().addHandler(stream_handler) # add to root handler # global variables - options = gather_options(argv) + options = isrcshared.options = gather_options(argv) ws2 = WebService2(options.user) if options.debug: diff --git a/pkg/.gitignore b/pkg/.gitignore index 847c4d4..a3c252a 100644 --- a/pkg/.gitignore +++ b/pkg/.gitignore @@ -7,3 +7,5 @@ libdiscid-*/ musicbrainzngs-*/ isrcsubmit/ mediatools/ +mutagen-*/ +unidecode-*/ diff --git a/pkg/Makefile b/pkg/Makefile index e81167e..e3f5abd 100644 --- a/pkg/Makefile +++ b/pkg/Makefile @@ -1,4 +1,4 @@ -version := 2.1.0 +version := 2.2.0 current := isrcsubmit-$(version) downloads := ../web/isrcsubmit.jonnyjd.net/downloads/ @@ -6,8 +6,12 @@ upload := ftp://isrcsubmit.jonnyjd.net/downloads/ musicbrainzngs := musicbrainzngs-0.6 discid := discid-1.1.1 libdiscid := libdiscid-0.6.1 +unidecode := unidecode-1.2.0 +mutagen_ver := 1.45.1 +mutagen := mutagen-$(mutagen_ver) -scripts := isrcsubmit.py $(current).py +scripts := isrcsubmit.py isrcsubmit-$(version).py \ + isrcDigitalSubmit.py isrcDigitalSubmit-$(version).py #scripts := $(current).py tar := $(current) win32 := $(current)-win32 @@ -39,7 +43,7 @@ define cp_source = rm -rf $(1)/.git endef -base_libs := $(musicbrainzngs) $(discid) $(libdiscid) +base_libs := $(musicbrainzngs) $(discid) $(libdiscid) $(unidecode) $(mutagen) define cp_base_libs = mv $(1)/COPYING $(1)/COPYING.isrcsubmit mv $(1)/AUTHORS $(1)/AUTHORS.isrcsubmit @@ -49,6 +53,10 @@ define cp_base_libs = cp -ar $(discid)/discid $(1)/ cp -ar $(discid) $(1)/src/ cp -ar $(libdiscid) $(1)/src/ + cp -ar $(mutagen)/mutagen $(1)/ + cp -ar $(mutagen) $(1)/src/ + cp -ar unidecode-$(unidecode)/unidecode $(1)/ + cp -ar unidecode-$(unidecode) $(1)/src/ endef @@ -63,8 +71,14 @@ upload: $(all) isrcsubmit.py: $(source_ref) cp -a $(source_dir)/isrcsubmit.py . -$(current).py: $(source_ref) - cp -a $(source_dir)/isrcsubmit.py $(current).py +isrcsubmit-$(version).py: $(source_ref) + cp -a $(source_dir)/isrcsubmit.py isrcsubmit-$(version).py + +isrcDigitalSubmit.py: $(source_ref) + cp -a $(source_dir)/isrcDigitalSubmit.py . + +isrcDigitalSubmit-$(version).py: $(source_ref) + cp -a $(source_dir)/isrcDigitalSubmit.py isrcDigitalSubmit-$(version).py tar: $(tar).tar.gz $(tar).tar.gz: $(source_ref) @@ -73,13 +87,11 @@ $(tar).tar.gz: $(source_ref) rm -rf $(tar) win32: $(win32).zip -$(win32).zip: $(source_ref) $(base_libs) $(libdiscid)-win32.zip mediatools +$(win32).zip: $(source_ref) $(base_libs) $(libdiscid)-win32.zip $(call cp_source, $(win32)) $(call cp_base_libs, $(win32)) unzip $(libdiscid)-win32.zip cp -a $(libdiscid)-win32/discid.dll $(win32)/ - cp -ar mediatools $(win32)/ - cp -a mediatools.exe $(win32)/ zip -qr $@ $(win32) rm -rf $(libdiscid)-win32 rm -rf $(win32) @@ -126,17 +138,23 @@ $(libdiscid).tar.gz: $(libdiscid)-%.zip: wget http://ftp.musicbrainz.org/pub/musicbrainz/libdiscid/$@ -mediatools: mediatools.zip - unzip -o $^ +$(unidecode).tar.gz: + wget https://github.com/avian2/unidecode/archive/refs/tags/$@ + +unidecode: $(unidecode) +$(unidecode): $(unidecode).tar.gz + tar xfz $^ -mediatools.zip: - wget http://www.flanagan-family.com/$@ +$(mutagen).tar.gz: + wget https://github.com/quodlibet/mutagen/releases/download/release-$(mutagen_ver)/$@ +mutagen: $(mutagen) +$(mutagen): $(mutagen).tar.gz + tar xfz $^ clean: rm -rf $(source_dir) rm -f $(all) rm -rf $(base_libs) - rm -rf mediatools mediatools.exe .PHONY: $(source_ref) diff --git a/requirements.txt b/requirements.txt new file mode 100644 index 0000000..b0324e2 --- /dev/null +++ b/requirements.txt @@ -0,0 +1,6 @@ +setuptools~=44.1.1 +musicbrainzngs~=0.7.1 +discid~=1.1.0 +argparse~=1.2.1 +mutagen>=1.40.0 +Unidecode~=1.2.0 diff --git a/setup.py b/setup.py index a313b80..4eb63c0 100755 --- a/setup.py +++ b/setup.py @@ -40,6 +40,7 @@ def initialize_options(self): import os.path man_pages = [ (os.path.join(man_dir, 'man1'), ['build/man/isrcsubmit.1']), + (os.path.join(man_dir, 'man1'), ['build/man/isrcDigitalSubmit.1']), (os.path.join(man_dir, 'man5'), ['build/man/isrcsubmit-config.5']) ] else: @@ -48,7 +49,8 @@ def initialize_options(self): args = {} if have_setuptools: - args["install_requires"] = ["discid >=1.0.0", "musicbrainzngs >=0.4"], + args["install_requires"] = ["discid >=1.0.0", "musicbrainzngs >=0.4", + "Unidecode >=1.2.0", "mutagen >=1.45.1" ] # we load isrcsubmit on setup args["setup_requires"] = args["install_requires"], @@ -94,15 +96,16 @@ def run(self): long_description = readme.read() setup(name="isrcsubmit", - version="2.1.0", + version="2.2.2", description="submit ISRCs from disc to MusicBrainz", long_description=long_description, author="Johannes Dewender", author_email="brainz@JonnyJD.net", url="https://github.com/JonnyJD/musicbrainz-isrcsubmit", - requires=["discid(>=1.0.0)", "musicbrainzngs(>=0.4)"], + requires=["discid(>=1.0.0)", "musicbrainzngs(>=0.4)", + "mutagen(>=1.45)", "unidecode(>=1.20)"], python_requires='>=2.6', - scripts=["isrcsubmit.py"], + scripts=["isrcsubmit.py", "isrcDigitalSubmit.py"], license="GPLv3+", classifiers=[ "Development Status :: 5 - Production/Stable", diff --git a/test_isrcsubmit.py b/test_isrcsubmit.py index f5529aa..2be697d 100755 --- a/test_isrcsubmit.py +++ b/test_isrcsubmit.py @@ -27,7 +27,6 @@ TEST_DATA = "test_data/" SAVE_RUN = False - class TestInternal(unittest.TestCase): def setUp(self): # suppress output @@ -188,7 +187,7 @@ def _open(name, mode): _isrcsubmit_has_program = isrcsubmit.has_program _isrcsubmit_get_prog_version = isrcsubmit.get_prog_version -def _has_program(program, strict=False): +def _has_program(program, choices=(), strict=False): if program == "libdiscid": # we mock it anyways # libdiscid >= 0.2.2 still needed to load discid @@ -197,7 +196,7 @@ def _has_program(program, strict=False): # also mocked return True else: - return _isrcsubmit_has_program(program, strict) + return _isrcsubmit_has_program(program, choices, strict) def _get_prog_version(prog): if prog == "libdiscid":