Skip to content

Commit

Permalink
Couple days of tinkering on Exercise
Browse files Browse the repository at this point in the history
  • Loading branch information
LukeDowell committed Sep 24, 2024
1 parent eea8fc7 commit 5a9b9ec
Show file tree
Hide file tree
Showing 10 changed files with 351 additions and 111 deletions.
72 changes: 72 additions & 0 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -1742,3 +1742,75 @@ As for getting it running on an iPad, I'm still a little boned by the fact that
theoretically move this to some kind of native wrapper, and I might, but I think that is a little bit of a distraction.
There is this wild Jazz-plugin that I am sure I mentioned somewhere above in the last few years (!!!) but I have yet to
try it out.


## 09/20/2024

I'm working through a new exercise component, and I'm considering adding the concept of 'duration' to the `Note` class.
From a domain perspective, does that make sense? The only time duration comes into the app is when we are trying to
generate a multi dimensional collection of notes in order to present them to the player, and honestly at this point it's
also just because Vexflow forces us to provide a duration.

Ideally, I'd like to be able to avoid doing this manually at all. In order to get the amount of varied exercises I am
hoping for, I am going to need to be able to create them from some kind of DSL. Maybe something like this:

```json
{
"title": "Simple Melody",
"tags": [,"Right Hand Only", "Melody"],
"exercise": [ // recipe for creating the content of the exercise, list of measures
[ // measure one
"sd5q", // 5th scale degree quarter note
"sd4q", // 4th scale degree quarter note
"sdb7e", // flat 7th scale degree eighth
"sdb6e", // flat 6th scale degree eighth
"sdb6e", // flat 6th scale degree eighth
]
]
}
```

Already it's a little annoying. Maybe I can instead just augment Vexflow's EasyScore grammar with any additional features
I'd like.


## 09/22/2024

I'm still working on the new exercise. I've tried a few different configurations out, and I think I am getting closer
to settling on something that will be generic enough to handle the first several exercises, and something that is easy
enough to use.

I have also been noodling on a "skill tree" concept for gamification. Khan Academy used to have this amazing galaxy-tree
view of all the skills that they had lessons on. In order to get to "Calculus I", you had to progress up the math tree,
handling arithmetic, geometry, and algebra for example. DuoLingo had (or had, haven't used it in a while) a feature
I like as well, where some skills in the tree would decay, and by doing a daily exercise, you could regain any of the
depleted amount.

I think of working on piano technique a little bit like "eating vegetables". This is definitely unfair to vegetables, but
practicing scales is something I have to do in order to get to what I really want, which is to be able to play music
I enjoy. Not only just able to play, but to play better and to learn quicker as well. When I think of gamification for
this app, I'd like to better show and track that long-term progress. With that in mind, I probably want to do something
between duolingo's 3-star system and runescape's grind-fest.


## 09/23/2024

I'm feeling a little frustrated with Vexflow. The domain terminology is clashing with my internal model of how I'd
describe sheet music. I have been butting my head against a wall with the new exercise component; I wanted to use
Vexflow's high level API to format notes and draw stems and accidentals and such, but I also wanted to have enough control
to animate the staff and set the position of each measure as need be. There is a component of Vexflow called a 'System',
that, due to it's name, I assumed was some higher-level organizational concept that would apply to several measures. Nope,
turns out System wants to be a small set of one or more staves? I'm not really sure, when I try to add a grand staff's
worth of staves to a system, the formatting goes nuts.

In any case, here we are:

![new exercise rendering](./doc/new-exercise-rendering.png)

I think I'm going to skip on breaking this up into newlines right now, because if the vision is that someone can use
this on their iPad or phone or whatever, we will need to deal with small screens.

Next steps for the snappy exercise component:
* Player feedback. Show greyed out notes the user is playing, just like the old component
* Beat indicator. For this go around, I'd like to keep the music -mostly- static, and instead draw at different
locations on the screen.
Binary file added doc/new-exercise-rendering.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
97 changes: 34 additions & 63 deletions src/app/altscale/page.tsx
Original file line number Diff line number Diff line change
@@ -1,88 +1,57 @@
'use client'

import React, {useState} from 'react'
import React, {useEffect, useState} from 'react'
import {styled} from "@mui/material/styles";
import _ from "lodash";
import {ExerciseResult} from "@/components/exercises/NotesExercise";
import {MAJOR_SCALE, SCALES_FOR_ALL_NOTES} from "@/lib/music/Scale";
import {findNoteOnKeyboard, KEYBOARD, Note, placeOnOctave} from "@/lib/music/Note";
import {notesToStaveNote} from "@/lib/vexMusic";
import {StaveNote} from "vexflow";
import {noteToEasyScore} from "@/lib/vexMusic";
import {Autocomplete, TextField} from "@mui/material";
import Exercise from "@/components/exercises/Exercise";
import {getKey} from "@/lib/music/Circle";


const StyledRoot = styled('div')`
display: flex;
flex-direction: column;
align-items: center;
width: 100vw;
.scale-settings {
display: flex;
flex-direction: row;
flex-direction: column;
align-items: center;
justify-content: space-around;
margin-bottom: 2rem;
margin-top: 1.15rem;
}
width: 100vw;
.scale-autocomplete {
width: 40vw;
min-width: 200px;
}
`
.scale-settings {
display: flex;
flex-direction: row;
align-items: center;
justify-content: space-around;
margin-bottom: 2rem;
margin-top: 1.15rem;
}
const NUM_NOTES_EQUAL_DURATION: Record<number, string> = {
1: 'w',
2: 'h',
4: 'q',
8: '8',
16: '16',
32: '32'
}
.scale-autocomplete {
width: 40vw;
min-width: 200px;
}
`

export default function ScalePage() {
export default function AltScalePage() {
const numOctaves = 3
const numNotesPerMeasure = 8
const bpm = 60
const [scale, setScale] = useState(MAJOR_SCALE)
const [rootNote, setRootNote] = useState(Note.of('C'))
const [bassVoice, setBassVoice] = useState('')
const [trebleVoice, setTrebleVoice] = useState('')

const rootIndex = findNoteOnKeyboard(rootNote.withOctave(1))
const scaleNotesWithoutOctave = _.range(0, numOctaves)
.flatMap(_ => scale.semitonesFromRoot.map(semi => KEYBOARD[rootIndex + semi]))
.map(n => n.withOctave(undefined))

const scaleNotes = placeOnOctave(3, [rootNote, ...scaleNotesWithoutOctave])
const scaleNotesGoingDown = _.reverse(placeOnOctave(3, [rootNote, ...scaleNotesWithoutOctave]))
scaleNotes.push(...scaleNotesGoingDown)
useEffect(() => {
const rootIndex = findNoteOnKeyboard(rootNote.withOctave(1))
const scaleNotesWithoutOctave = _.range(0, numOctaves)
.flatMap(_ => scale.semitonesFromRoot.map(semi => KEYBOARD[rootIndex + semi]))
.map(n => n.withOctave(undefined))

const measureSize = numNotesPerMeasure
const measures = _.chain(scaleNotes)
.map(note => notesToStaveNote([note], {duration: NUM_NOTES_EQUAL_DURATION[measureSize]}))
.chunk(measureSize)
.map(notesInMeasure => {
if (notesInMeasure.length !== numNotesPerMeasure) {
const numNotes = notesInMeasure.length
const remainingNotes = measureSize - numNotes
const addRest = (duration: string) => notesInMeasure.push(new StaveNote({keys: ["b/4"], duration}))
if (numNotes === remainingNotes) addRest('hr')
else if (remainingNotes === measureSize / 4) addRest('qr')
else if (remainingNotes === measureSize / 8) addRest('8r')
else if (remainingNotes === measureSize / 16) addRest('16r')
else if (remainingNotes === measureSize / 32) addRest('32r')
}
return notesInMeasure
})
.value()
const scaleNotes = placeOnOctave(3, [rootNote, ...scaleNotesWithoutOctave])
const scaleNotesGoingDown = _.reverse(placeOnOctave(3, [rootNote, ...scaleNotesWithoutOctave]))
scaleNotes.push(...scaleNotesGoingDown)

function reset(r: ExerciseResult) {
const noteAndScale = _.sample(SCALES_FOR_ALL_NOTES)!
setScale(noteAndScale.scale)
setRootNote(noteAndScale.note)
}
setBassVoice(scaleNotes.map((n) => n.withOctave(n.octave! - 1)).map((n) => noteToEasyScore(n, "q")).join(', '))
setTrebleVoice(scaleNotes.map((n) => noteToEasyScore(n, "8")).join(', '))
}, [scale, rootNote])

return <StyledRoot>
<div className={'scale-settings'}>
Expand All @@ -101,6 +70,8 @@ export default function ScalePage() {
}}
/>
</div>
<Exercise musicKey={getKey('C')} bassNotes={'C3/q, C#3/q, D3/q, D#3/q'} trebleNotes={'C4/q, C#4/q, D4/q, D#4/q'}/>
{(bassVoice.length > 0 || trebleVoice.length > 0) &&
<Exercise musicKey={getKey('C')} bassVoice={bassVoice} trebleVoice={trebleVoice}/>
}
</StyledRoot>
}
2 changes: 1 addition & 1 deletion src/components/exercises/DiatonicChordExercise.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -130,5 +130,5 @@ export default function DiatonicChordExercise({musicKey, onEnd, options}: Props)
}

const VexflowOutput = styled('div')`
overflow: hidden;
overflow: hidden;
`
47 changes: 13 additions & 34 deletions src/components/exercises/Exercise.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -5,19 +5,26 @@ import {MidiInputContext} from "@/lib/react/contexts";
import MidiPiano, {NoteEvent, NoteSubscriber} from "@/lib/music/MidiPiano";
import _ from "lodash";
import {Note} from "@/lib/music/Note";
import {Vex, Voice} from "vexflow";
import {MusicKey} from "@/lib/music/Circle";
import {renderVex} from "@/lib/vexRenderer";

const VexflowOutput = styled('div')`
overflow: hidden;
`

interface Props {
// The key this exercise will be in
musicKey: MusicKey,
bassNotes: string,
trebleNotes: string
bassVoice: string,
trebleVoice: string,
}

/**
* Generic exercise that will complete when players have played all provided notes
*
* @param props
* @constructor
*/
export default function Exercise(props: Props) {
// const [context, [contextWidth, contextHeight]] = useVexflowContext('exercise-vexflow-output')
const midiInput = useContext(MidiInputContext)
Expand All @@ -33,37 +40,9 @@ export default function Exercise(props: Props) {
}, [midiInput])

useEffect(() => {
const outputDiv = document.getElementById('exercise-vexflow-output') as HTMLDivElement
if (outputDiv) outputDiv.innerHTML = ''

const vf = new Vex.Flow.Factory({renderer: {elementId: 'exercise-vexflow-output', width: windowWidth, height: 300}})
const score = vf.EasyScore()
const system = vf.System()

let trebleVoice: Voice[] = []
if (props.trebleNotes.length > 0) {
const easyScoreNotes = score.notes(props.trebleNotes, {clef: 'treble'})
trebleVoice = [score.voice(easyScoreNotes)]
}

let bassVoice: Voice[] = []
if (props.bassNotes.length > 0) {
const easyScoreNotes = score.notes(props.bassNotes, {clef: 'bass'})
bassVoice = [score.voice(easyScoreNotes)]
}

// TODO doesn't seem to automatically add more measures, we will have to slice it ourselves
system.addStave({voices: trebleVoice})
.addClef('treble')
.addTimeSignature('4/4')

system.addStave({voices: bassVoice})
.addClef('bass')
.addTimeSignature('4/4')

system.addConnector()
vf.draw()
}, [windowWidth]);
const renderConfig = {width: windowWidth, trebleVoice: props.trebleVoice, bassVoice: props.bassVoice}
renderVex('exercise-vexflow-output', renderConfig)
}, [windowWidth, props]);

const noteSubscriber: NoteSubscriber = (event: NoteEvent, currentActiveNotes: Note[], history: NoteEvent[]) => {
const {note, velocity, flag, time} = event
Expand Down
22 changes: 11 additions & 11 deletions src/components/exercises/NotesExercise.tsx
Original file line number Diff line number Diff line change
Expand Up @@ -41,22 +41,22 @@ const durationToFraction = (duration: string) => {
}

const beatAnimation = keyframes`
from {
transform: scale(1, 1);
}
from {
transform: scale(1, 1);
}
to {
transform: scale(1, .8);
}
to {
transform: scale(1, .8);
}
`

const VexflowOutput = styled('div')`
overflow: hidden;
${beatAnimation};
overflow: hidden;
${beatAnimation};
#vf-beat-indicator {
transform-origin: center center;
}
#vf-beat-indicator {
transform-origin: center center;
}
`

export default function NotesExercise({inputMeasures, onEnd, options}: Props) {
Expand Down
20 changes: 20 additions & 0 deletions src/lib/vexMusic.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@
import {fillWithRests, splitIntoMeasures} from "@/lib/vexMusic";

describe('vex music', () => {
it('should place a list of notes into evenly divided measures', () => {
const input = 'C2/q, D2, E2, F2, G2, A2, B2, C3, D3'
const measures = splitIntoMeasures(input)

// expect(measures.length).toBe(3)
expect(measures[0]).toBe('C2/q, D2, E2, F2')
expect(measures[1]).toBe('G2, A2, B2, C3')
expect(measures[2]).toBe('D3, C4/h/r, C4/q/r')
})

test.each([
["", "/w/r"],
])(
`%s measure should be filled with an %s rest`,
(input: string, expectedRests: string) => expect(fillWithRests(input).includes('')).toBeTruthy()
)
})
Loading

0 comments on commit 5a9b9ec

Please sign in to comment.