MARVYN

MARVYN is Mastering Audio, Ripping & Video tasks automaticallY & Nicely.

Key features

MARVYN automates multimedia tasks:

• Rip and decrypt Audio CDs, DVDs & BDs without a single click or keypress
• Compress videos keeping only the main video and a single audio track
• Convert video and audio into smaller formats (e.g. h.264, mp3)
• Remultiplex video directories to files losslessly (Blu-ray or DVD to MKV)

MARVYN is nice:

• Runs in a Docker container to keep the system clean of dependencies
• Uses VAAPI hardware acceleration for video transcoding
• Runs with low cpu priority
• Allows to send a mail notification after a disc is ripped
• Does not need root permissions

TL;DR

• Pre-requisites: the current user has permissions to
  • create the directory /opt/docker/marvyn
  • build a Docker image
• Install:

  $ sudo apt-get install docker-compose-plugin
  $ wget -O - https://github.com/git-developer/marvyn/tarball/main | tar --one-top-level=/opt/docker/marvyn --strip-components 1 -xz
  $ /opt/docker/marvyn/image/build

• Enable autorip:

  $ sudo ln -s /opt/docker/marvyn/udev/rules.d/disc-detection.rules /etc/udev/rules.d/disc-detection.rules
  $ sudo udevadm control --reload

• Run: Insert disc

Pre-requisites

• A linux system with Docker Compose.

The Docker Compose documentation contains a comprehensive guide explaining several install options. On debian-based systems, Docker Compose may be installed by calling

$ sudo apt-get install docker-compose-plugin

Installation

1. Download to /opt/docker/marvyn, e.g.

   $ (groups | grep -oP users || id -gn) | xargs -I {} sudo sh -c 'mkdir -p -m 0775 /opt/docker && chown ":{}" /opt/docker'
   $ wget -O - https://github.com/git-developer/marvyn/tarball/main | tar --one-top-level=/opt/docker/marvyn --strip-components 1 -xz

2. Create Docker image (this will take a few minutes):

    $ /opt/docker/marvyn/image/build

   If you wish to customize the tags and labels, change the values in image/.labelsand etc/base.yml accordingly.

Usage

Autorip

• Insert a Blu-ray, DVD or Audio CD into your drive.

When autorip is enabled (disabled by default), MARVYN will automatically rip the disc to your output directory, remove copy protection and tag audio files. If mail notification is enabled (disabled by default), MARVYN will send a mail with a log file on finish.

Video Disc Conversion

• bin/transcode-disc-directory <path-to-directory>

The directory must contain a BD or DVD structure (i.e. VIDEO_TS or BDMV). All videos will be converted to the configured target format (default: h.264) containing a single language (default: German) and written to the output directory as separate files. Video conversion uses hardware transcoding.

Audio Conversion

• bin/transcode-audio-directory <path-to-directory>

The directory or its subdirectories must contain audio files. The audio files will be converted to the target format (default: mp3) while keeping directory structure and metadata (tags).

Video Series Conversion

• bin/transcode-series-directory <path-to-directory>

The directory is supposed to hold a (TV or movie) series; it must contain subdirectories, each holding a Blu-ray or DVD structure. Every subdirectory will be converted by the Video Disc Conversion.

Video Remultiplexing

• bin/remux-directory <path-to-directory>

The directory must contain a Blu-ray or DVD structure (i.e. VIDEO_TS or BDMV). Every contained video will be converted to a separate MKV file losslessly and written to the output directory.

Video Transcoding

• bin/vaapi-transcode <path-to-video-file>

The input video will be converted to the configured target format (default: h.264) containing a single language (default: German). Video conversion uses hardware transcoding.

Configuration

General

• File: etc/base.yml

Option	Description	Examples	Default
Environment variable MARVYN_MAKEMKV_KEY	MakeMKV registration key. When unset, a beta key is downloaded and used.	T-wN2...AE5z	none
Environment variable MARVYN_NICENESS	CPU priority (in terms of the nice command)	-19..20	none (effectively: Default of nice command)
Environment variable TZ	Timezone	Europe/Berlin	none
Environment variable FAKETIME	System time that MARVYN uses. This option allows to circumvent expiration issues. See libfaketime for format specification and details.	-14d, 2020-12-24 20:30:00	none
Service Option cpu_shares	CPU priority. 1024 means normal, lower value mean lower priority. See CPU share constraint for details.	512	512

Autorip

Enable Autorip:

$ sudo ln -s /opt/docker/marvyn/udev/rules.d/disc-detection.rules /etc/udev/rules.d/disc-detection.rules
$ sudo udevadm control --reload

You have to to modify the file content only if one or more of the following cases apply:

• MARVYN is not located in the default directory /opt/docker/marvyn.
• Your disc drive is not mounted at the default locations /dev/sg1 and /dev/sr0; you also have to modify etc/rip-disc.yml in this case. Run lsscsi -g to find your optical drives and their mount locations.
• You want to run Autorip as a non-root user; change the command to /usr/bin/sudo -E -u <user> /opt/docker/marvyn/bin/rip-disc in this case.

Main configuration

File etc/rip-disc.yml

Option	Description	Examples	Default
Device Options	Available disc drives	/dev/cdrom:/dev/cdrom	see etc/rip-disc.yml
Environment variable MARVYN_CD_TITLE_DEPTH	Starting in the output directory, how many subdirectories does MARVYN have to descend to find the directory containing the CD title? Should be set according to the directory count used in OUTPUTFORMAT and VAOUTPUTFORMAT in the file data/.abcde.conf	1	3
Environment variable MARVYN_EJECT_DISC	Eject disc on finish	yes	no
Environment variable MARVYN_OVERWRITE	Overwrite files in the output directory	yes	no

Mail notification

• File: etc/mail.env

Notification mails are sent only when MARVYN_MAIL_RECIPIENT is set.

Option	Description	Examples
MARVYN_MAIL_RECIPIENT	Recipient address	[email protected]
MARVYN_MAIL_SENDER	Sender address	[email protected]
MARVYN_MAIL_SERVER	Mail server and optional port	mail.example.com:587
MARVYN_MAIL_USER	Username for login on mail server	[email protected]
MARVYN_MAIL_PASSWORD	Password for login on mail server	p4§§w0rD

Audio CD ripping

• File: data/.abcde.conf

See the abcde documentation for details.

Video Disc Conversion

• File: etc/transcode-disc-directory.yml

Option	Description	Examples	Default
Environment variable MARVYN_CLEANUP	Delete temporary files after successful finish?	yes, no	yes

This configuration uses the Video Transcoding configuration as default. Values in etc/vaapi-transcode.yml will be applied to all video transcoding tasks, including Video Disc Conversion. Values in etc/transcode-disc-directory.yml will be applied to Video Conversion only.

Audio Conversion

• File: etc/transcode-audio-directory.yml

Option	Description	Examples	Default
Environment variable MARVYN_FFMPEG_OUTPUT_FORMAT	Output format	mp3, ogg	mp3
Environment variable MARVYN_FFMPEG_OUTPUT_OPTIONS	Output options for FFmpeg	see FFmpeg documentation	-q 5.5 -id3v2_version 3 -c:v copy
Environment variable MARVYN_AUDIO_INCLUDES	Comma-separated list of filenames that are considered for conversion. Glob expressions are supported.	*.flac,*.ogg	*.flac,*.ogg,*.mp3
Environment variable MARVYN_AUDIO_EXCLUDES	Comma-separated list of filenames that are ignored for conversion. Glob expressions are supported. This option has no effect when MARVYN_AUDIO_INCLUDES is set.	*.jpg,*.png,*.txt	none

Video Series Conversion

• File: etc/transcode-series-directory.yml

This configuration uses the Video Disc Conversion configuration as default. Values in etc/transcode-disc-directory.yml will be applied to all video disc conversion tasks, including Video Series Conversion. Values in etc/transcode-series-directory.yml will be applied to Video Servies Conversion only.

Video Remultiplexing

• File: etc/remux-directory.yml

Video Transcoding

• File: etc/vaapi-transcode.yml

Option	Description	Examples	Default
Environment variable MARVYN_VIDEO_BITRATE	Video bitrate	1280k	1280k
Environment variable MARVYN_VIDEO_MAXRATE	Maximum video bitrate	1280k	Video bitrate
Environment variable MARVYN_VIDEO_BUFFERSIZE	Video buffer size	2560k	2x video bitrate
Environment variable MARVYN_VIDEO_CODEC	Video format supported by VAAPI	h264_vaapi	h264_vaapi
Environment variable MARVYN_VIDEO_WIDTH	Video width	720	720
Environment variable MARVYN_AUDIO_BITRATE	Audio bitrate	128k	128k
Environment variable MARVYN_AUDIO_CODEC	Audio format	aac	aac
Environment variable MARVYN_AUDIO_CHANNELS	Audio channels	2	2
Environment variable MARVYN_AUDIO_LANGUAGES	List of allowed audio languages in descending priority. The first language that is found in the video is selected for output. If the video contains only a single language, that one is used and this setting is ignored.	deu,ger,eng	deu,ger,eng
Environment variable MARVYN_VAAPI_DEVICE	VAAPI device for hardware transcoding	/dev/dri/renderD128	/dev/dri/renderD128
Device Options	Available VAAPI devices	/dev/dri/renderD128:/dev/dri/renderD128	/dev/dri/renderD128:/dev/dri/renderD128

Advanced Topics

Management tasks

• Start a service (e.g. Audio Conversion):

  $ bin/transcode-audio-directory /media/audio/
  marvyn-transcode-audio-directory-20200312-061525

• See the log:

  $ docker logs marvyn-transcode-audio-directory-20200312-061525

• Check status:

  $ docker ps -a --format={{.Status}} -f name=marvyn-transcode-audio-directory-20200312-061525

• Stop service:

  $ docker stop marvyn-transcode-audio-directory-20200312-061525

• Remove a service container (including its logs):

  $ docker rm marvyn-transcode-audio-directory-20200312-061525

• Remove all MARVYN containers:

  $ docker container ls -a -q -f name=marvyn | xargs docker rm

• Start a shell within MARVYN:

  $ docker compose -f etc/base.yml run --rm marvyn-base bash

Environment variables

Environment variables may either be declared in a configuration file or as environment variable. For all variables prefixed with MARVYN_, the environment value takes precedence over the configuration file. For all other variables, the default Docker Compose behavior is applied:

• if a variable is not declared in the configuration file, the value is ignored.
• if a variable is declared in the configuration file without value, the value of the environment variable is used.
• if a variable is declared in the configuration file with value, this value is used and the environment value is ignored.

Configuration of output and work directories

By default, the directories for output and work are located in the root directory of the installation (on the same level as bin and etc). If that's not appropriate, you have the choice to:

1. Hand them over as command line arguments to a service:

   $ bin/transcode-disc-directory /media/input.mp4 /media/output/
   $ bin/transcode-disc-directory /media/input.mp4 /media/output/ /tmp/work/

2. Set environment variables to change the location:

   $ export MARVYN_OUTPUT_DIRECTORY=/media/output/
   $ export MARVYN_WORK_DIRECTORY=/tmp/work/
   $ bin/transcode-disc-directory /media/input.mp4

3. Use symlinks to redirect them to a different location:

   $ ln -s /media/output output
   $ ln -s /tmp/work work
   $ bin/transcode-disc-directory /media/input.mp4

4. Use bind mounts to mount them to a different location:

   $ sudo mount --bind /media/output output
   $ sudo mount --bind /tmp/work work
   $ bin/transcode-disc-directory /media/input.mp4

Users, groups and permissions

MARVYN services run with userid and groupid of the current user. Autorip is called by udev which runs as root by default. A different user may be configured by using sudo -u within the udev rule.

Home directory

The directory data is used as home directory for MARVYN. It contains all configuration and data needed at runtime, e.g.

• MakeMKV configuration
• abcde configuration
• Keys for disc decryption

Base image (Linux distribution)

The default Docker image is based on Ubuntu, an alternative Docker image based on Debian is available in image/Dockerfile.debian.

Ripping of encrypted discs

Detailed information about ripping of encrypted discs may be found in the resources listed in the file README.keys.

DVD duplicate file detection (requires support for hard links)

A few DVDs appear to be huge (e.g. 70 GB) because they contain multiple VOB files that reference the same data. MARVYN integrates a DVD duplicate detector to rip only the necessary files and use hard links for the rest. This mechanism requires that the target file system supports hard links and does not work otherwise (e.g. for samba mounts).

Extending MARVYN

Example: Create a new service called process-video

1. Create start script bin/process-video:

   ln -s .start bin/process-video

2. Create configuration file etc/process-video.yml:

   ---
   version: "2.2"
   services:
     marvyn-process-video:
       extends:
         file: base.yml
         service: marvyn-base
   
       <custom configuration here>

3. Create executable service script services/process-video, e.g.:

   #!/bin/sh
   here="$(dirname ${0})"
   . "${here}/.prepare"
   if [ -z "${1-}" ] || [ ! -d "${1}" ] || [ -z "${2-}" ]; then
     cancel "Syntax: $(basename $0) <input-directory> <output-directory>"
   fi
   mkdir -p "${output_fullpath}"
   
   <service implementation>

Components

MARVYN integrates the following applications and libraries:

• Docker Compose
• MakeMKV
• dvdbackup
• abcde
• ffmpeg
• sendEmail
• libdvdcss
• libfaketime
• udev
• DVD duplicate detector

Examples

Given the output directory /media/inbox/ and the following example directory structure:

media
+-inbox
| +-CD
| +-DVD
| +-BD
+-audio
| +-single-track.mp3
| +-album
|   +-01.ogg
|   +-02.flac
|   +-folder.jpg
+-video
| +-movie.mp4
| +-movie
| | +-VIDEO_TS
| +-series
|   +-episode-01
|   | +-VIDEO_TS
|   +-episode-02
|     +-VIDEO_TS
+-phone

• Rip a Blu-ray disc (1:1 copy) lossless to /media/inbox/BD and receive an e-mail notification on finish:

  Insert the disc into the drive

• Compress DVD directory /media/video/movie/ to small MKV files containing a single language, using VAAPI hardware transcoding:

  $ bin/transcode-disc-directory /media/video/movie/

• The same for a directory containing several DVDs of a series in directory /media/video/series:

  $ bin/transcode-series-directory /media/video/series/

• For a MP4 file /media/video/movie.mp4 with output format MP4:

  $ bin/vaapi-transcode /media/video/movie.mp4

• For a MP4 file /media/video/movie.mp4 with output format MKV:

  $ bin/vaapi-transcode /media/video/movie.mp4 /media/phone/movie.mkv

• Convert DVD directory /media/video/movie losslessly to MKV format:

  $ bin/remux-directory /media/video/movie/

• Convert audio directory media/audio to mp3, keeping tags:

  $ bin/transcode-audio-directory /media/audio/