Issue with Recording Object Not Capturing MIDI Events in DryWetMidi

[dblondin]

Hi there. I’ve been using the Melanchall.DryWetMidi library successfully for about a month, specifically utilizing its recording capabilities to capture MIDI events. Initially, everything worked as expected, and I was able to save MIDI recordings without any issues. However, recently I've encountered a problem where the library's event capture mechanism seems to have stopped working. Despite no memory of significant changes on my end, the Recording object no longer captures events from their associated InputDevice, resulting in empty recordings.

Here’s a snippet of how I’m setting up the recording objects:

foreach (var inputDevice in _inputDevices)
{
    inputDevice.StartEventsListening();
    recording = new Recording(tempoMap, inputDevice);
}

To diagnose the issue, I've confirmed that MIDI events are indeed being received by verifying through debug logs in the EventReceived event handler. However, these events do not appear to be captured by the Recording object, as indicated by the recording being empty upon saving.

• No changes were made to the setup or code that correlates with the onset of this issue.
• Attempted troubleshooting steps include verifying input device connections, ensuring no other software is monopolizing the MIDI devices, and restarting the app and computer
• System Configuration: Running on macOS Sonoma 14.3 on an M1 Mac, with the library version being the last known working without any updates to the library itself prior to the issue.

Any suggestions on further troubleshooting steps or fixes would be greatly appreciated. I’m stumped!

Here are two MIDI test files I recorded with it. One when it was working and the other being empty:

MIDI_files.zip

Here’s the entire MIDI recording class:

using Melanchall.DryWetMidi.Common;
using Melanchall.DryWetMidi.Core;
using Melanchall.DryWetMidi.Interaction;
using Melanchall.DryWetMidi.Multimedia;
using Melanchall.DryWetMidi.MusicTheory;
using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using UnityEngine;
using System.Threading.Tasks;
using Melanchall.DryWetMidi.Tools;

public class MidiRecorder : MonoBehaviour
{
    public bool recordNoteOnEvents = true;
    public bool recordNoteOffEvents = true;
    public bool recordControlChangeEvents = false;
    public bool recordProgramChangeEvents = false;
    public bool recordPitchBendEvents = false;
    public bool recordClockEvents = false;
    public bool recordStartMessages = false;
    public bool recordStopMessages = false;

    private Recording recording;
    private InputDevice[] _inputDevices;

    private TempoMap tempoMap;

    public delegate void RecordingStateChanged(bool isRecording);
    public event RecordingStateChanged OnRecordingStateChanged;

    private bool _isRecording = false;
    public bool IsRecording
    {
        get { return _isRecording; }
        private set
        {
            if (_isRecording != value)
            {
                _isRecording = value;
                //Debug.Log($"Recording state changed: {_isRecording}");
                OnRecordingStateChanged?.Invoke(_isRecording);
            }
        }
    }

    static List<(string Note, string Time, string Length)> GetNoteDetails(MidiFile midiFile)
    {
        var notes = midiFile.GetNotes();
        var tempoMap = midiFile.GetTempoMap();
        var noteDetails = new List<(string, string, string)>();

        foreach (var note in notes)
        {
            var time = note.TimeAs<MusicalTimeSpan>(tempoMap).ToString();
            var length = note.LengthAs<MusicalTimeSpan>(tempoMap).ToString();
            noteDetails.Add((note.ToString(), time, length));
        }

        return noteDetails;
    }

    void Start()
    {
        tempoMap = TempoMap.Default;

        _inputDevices = InputDevice.GetAll().ToArray();

        foreach (var inputDevice in _inputDevices)
        {
            inputDevice.StartEventsListening();
            //inputDevice.SilentNoteOnPolicy = SilentNoteOnPolicy.NoteOff;//necessary for silent note offs (velocity zero)
            recording = new Recording(tempoMap, inputDevice);
        }
    }

    void Update()
    {
        CheckSpriteStatesAndHandleRecording();
    }

    public void CheckSpriteStatesAndHandleRecording()
    {
        bool playing = MidiParameter.spriteState[(int)SelectionManager.SpriteSelection.Sprite1] == MidiParameter.SPRITE_STATE_PLAY ||
                       MidiParameter.spriteState[(int)SelectionManager.SpriteSelection.Sprite2] == MidiParameter.SPRITE_STATE_PLAY;

        bool stopped = MidiParameter.spriteState[(int)SelectionManager.SpriteSelection.Sprite1] == MidiParameter.SPRITE_STATE_STOP &&
                       MidiParameter.spriteState[(int)SelectionManager.SpriteSelection.Sprite2] == MidiParameter.SPRITE_STATE_STOP;

        if (playing && !IsRecording)
        {
            StartRecording();
        }
        else if (stopped && IsRecording)
        {
            StopRecording();
        }
    }

    public void StartRecording()
    {
        recording.Start();
        IsRecording = true;
    }

    public void StopRecording()
    {
        recording.Stop();

        NameDisplay nameDisplay = FindObjectOfType<NameDisplay>();
        string name = nameDisplay?.GetCurrentName();

        if (name != null)
        {
            SaveRecording(name);
        }
        else
        {
            Debug.LogError("NameDisplay is null or no current name available");
        }

        IsRecording = false;
    }

    private int GetChannelNumber(MidiEvent midiEvent)
    {
        return midiEvent is ChannelEvent channelEvent ? channelEvent.Channel : 0;
    }

    private long AddEventToTrackChunk(TrackChunk trackChunk, TimedEvent timedEvent, TempoMap tempoMap, long lastEventTimeInTicks)
    {
        long eventTimeInTicks = TimeConverter.ConvertTo<MidiTimeSpan>(timedEvent.Time, tempoMap).TimeSpan;
        long deltaTime = eventTimeInTicks - lastEventTimeInTicks;

        var midiEvent = timedEvent.Event.Clone();
        midiEvent.DeltaTime = deltaTime;
        trackChunk.Events.Add(midiEvent);

        return eventTimeInTicks;
    }

    public void SaveRecording(string fileName)
    {
        if (!fileName.EndsWith(".mid") && !fileName.EndsWith(".midi"))
        {
            fileName += ".mid";
        }

        string desktopPath = Environment.GetFolderPath(Environment.SpecialFolder.Desktop);
        string folderPath = Path.Combine(desktopPath, "MidiFiles");
        Directory.CreateDirectory(folderPath);

        string originalFilePath = Path.Combine(folderPath, fileName);

        Task.Run(() =>
        {
            string filePath = originalFilePath;
            int fileNumber = 1;

            while (File.Exists(filePath))
            {
                string fileNameWithoutExtension = Path.GetFileNameWithoutExtension(originalFilePath);
                string fileExtension = Path.GetExtension(originalFilePath);
                string fileNumberSuffix = "_" + fileNumber.ToString();
                filePath = Path.Combine(folderPath, fileNameWithoutExtension + fileNumberSuffix + fileExtension);
                fileNumber++;
            }

            IReadOnlyList<TimedEvent> recordedEvents = recording.GetEvents();

            var midiFile = new MidiFile();
            var trackChunks = new Dictionary<int, TrackChunk>();
            var lastEventTimes = new Dictionary<int, long>();

            TempoMap tempoMap = midiFile.GetTempoMap();

            foreach (var timedEvent in recordedEvents)
            {
                if (timedEvent.Event == null)
                    continue;

                //var midiEvent = timedEvent.Event;
                var midiEvent = ConvertToExplicitNoteOffIfNeeded(timedEvent.Event); //TODO: this is untested

                int channelNumber = GetChannelNumber(midiEvent) + 1;

                if (!trackChunks.ContainsKey(channelNumber))
                {
                    var trackChunk = new TrackChunk();
                    trackChunk.Events.Add(new SequenceTrackNameEvent($"Channel {channelNumber}"));
                    trackChunks[channelNumber] = trackChunk;
                    lastEventTimes[channelNumber] = 0;
                }

                lastEventTimes[channelNumber] = AddEventToTrackChunk(trackChunks[channelNumber], timedEvent, tempoMap, lastEventTimes[channelNumber]);
            }

            foreach (var channel in trackChunks.Keys.OrderBy(k => k))
            {
                midiFile.Chunks.Add(trackChunks[channel]);
            }

            midiFile.Write(filePath, overwriteFile: true, format: MidiFileFormat.MultiTrack, settings: new WritingSettings { UseRunningStatus = true, NoteOffAsSilentNoteOn = false });
        });
    }

    private MidiEvent ConvertToExplicitNoteOffIfNeeded(MidiEvent midiEvent)
    {
        if (midiEvent is NoteOnEvent noteOnEvent && noteOnEvent.Velocity == 0)
        {
            // Convert to an explicit note-off event
            return new NoteOffEvent(noteOnEvent.NoteNumber, noteOnEvent.Velocity)
            {
                Channel = noteOnEvent.Channel,
                DeltaTime = noteOnEvent.DeltaTime
            };
        }

        return midiEvent; // Return the event as-is if no conversion is needed
    }

    public MidiFile ShiftAllEvents(MidiFile midiFile, long shiftAmount)
    {
        foreach (var trackChunk in midiFile.Chunks.OfType<TrackChunk>())
        {
            using (var manager = trackChunk.ManageTimedEvents())
            {
                var events = manager.Objects;

                foreach (var timedEvent in events)
                {
                    long newTime = timedEvent.Time + shiftAmount;
                    timedEvent.Time = newTime < 0 ? 0 : newTime; // Ensure the time is not less than zero
                }

                manager.SaveChanges();
            }
        }

        return midiFile;
    }

    void OnDestroy()
    {
        if (_inputDevices != null)
        {
            foreach (var device in _inputDevices)
            {
                if (device != null)
                {
                    device.StopEventsListening();
                    device.Dispose();
                }
            }
        }

        if (recording != null)
        {
            recording.Dispose();
        }
    }
}

[melanchall]

Hi,

Thanks for the info. Your code is pretty strange...

Let's look at the Start method. Do you see that your recording will always target last device of _inputDevices? The whole foreach loop can be replaced with

var inputDevice = _inputDevices.Last();
recording = new Recording(tempoMap, inputDevice);

and nothing will change.

And I suppose it's where your bug is placed. To check my assumption please change Start method to this:

void Start()
{
    tempoMap = TempoMap.Default;

    _inputDevices = InputDevice.GetAll().ToArray();

    foreach (var inputDevice in _inputDevices)
    {
        inputDevice.EventReceived += OnEventReceived;
        inputDevice.StartEventsListening();
        recording = new Recording(tempoMap, inputDevice);
    }
}

private void OnEventReceived(object sender, MidiEventReceivedEventArgs e)
{
    Debug.Log($"'{e.Event}' received from '{((MidiDevice)sender).Name}'");
}

Also change StartRecording:

public void StartRecording()
{
    recording.Start();
    Debug.Log($"Recording started on '{((InputDevice)recording.InputDevice).Name}'");
    IsRecording = true;
}

and StopRecording:

public void StopRecording()
{
    recording.Stop();
    Debug.Log($"Recording stopped on '{((InputDevice)recording.InputDevice).Name}'");

    NameDisplay nameDisplay = FindObjectOfType<NameDisplay>();
    string name = nameDisplay?.GetCurrentName();

    if (name != null)
    {
        SaveRecording(name);
    }
    else
    {
        Debug.LogError("NameDisplay is null or no current name available");
    }

    IsRecording = false;
}

Now

1. run your game;
2. start recording;
3. push some buttons on your input device;
4. stop recording.

And then send me the log from Unity.

Thanks,
Max

[dblondin]

Indeed you are correct and I see the issue now. By chance this code actually recorded the proper device at one time but I see that was extremely unlikely. ::picard_facepalm_here:: The intention was to listen to all devices but I need to manage separate Recording instances for each device. I think the way forward is to detect all the MIDI devices and list them in a dropdown for selection.

Oops report>>>

With your debug, when I had just an iConnectivity mio2 connected:

Recording started on 'mio2 none'
'Note On [12] (64, 67)' received from 'mio2 DIN 1'
'Note Off [11] (57, 0)' received from 'mio2 DIN 1'
Recording stopped on 'mio2 none'

When I had both an iConnectivity mio2 and a Focusrite 18i20 connected and the events coming into the mio2:

Recording started on 'Scarlett 18i20 USB'
'Start' received from 'mio2 DIN 1'
'Note On [10] (29, 67)' received from 'mio2 DIN 1'
Recording stopped on 'Scarlett 18i20 USB'

After simply hard coding it to one of my MIDI interfaces the recordings saved properly.

Thank you again for solving an issue for me. I can’t thank you enough for your library and personal time helping me get DryWetMIDI projects working.