Two-vocal Separation and Singing Pitch Transcription

Interact with the full pipeline on HF Spaces!

Set Up:

1. Install dependencies using Poetry: poetry install
2. Download the dataset MIR-1K

Running

Source Separation Evaluation

poetry run src/source_sep.py

Pitch Transcription

Trained models on Hugging Face:

• CREPE-16:
• CREPE-24:
• CREPE-32:

To run training pipeline: poetry run src/train_pitch_transcription.py

File Descriptions:

Training scripts:

• convtasnet.ipynb: Evaluate pre-trained Conv-TasNet model for vocal sepration
• test_crepe.ipynb : Train CREPE model for pitch transcription
• crepe_model.py: Define CREPE model

Pipeline:

• combined_pipeline.ipynb: combines both Conv-TasNet and trained CREPE models into a single pipeline
• crepe_model.py

Artifacts:

• best_crepe_xx.pkl: Stores best trained CREPE model for each model size
• leon_7_jmzen_5.wav: mixed track used for case study
• gbqq_lwq_mixed.wav: mixed track by me (lol) to test in-sync vocals

Others:

• analyze_pitch_labels: initial analysis on distribution of pitches in dataset

TO-DOs

• Setup Poetry
• Move from notebook to script
• Test individual scripts in src/
• Get intermediate audio samples for writeup post
• Create e2e pipeline
• Host model(s) on HF
• Add gradio on HF Spaces hehe