Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Locked Figure Aria] Use spoken math in locked figure settings autogen labels #1858

Merged
merged 20 commits into from
Nov 15, 2024

Conversation

nishasy
Copy link
Contributor

@nishasy nishasy commented Nov 14, 2024

Summary:

When auto-generating the aria labels for locked figures, we want it to use
words as if they were spoken rather than math expressions that might be
read incorrectly by the screen reader.

  • Use the generateSpokenMathDetails utility within LockedVectorSettings,
    LockedEllipseSettings, LockedPolygonSettings, and LockedFunctionSettings.
  • Updated LockedFigureAria - remove the prop prePopulatedAriaLabel and
    make getPrepopulatedAriaLabel required.

Issue: https://khanacademy.atlassian.net/browse/LEMS-2548

Test plan:

  • yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-vector-settings.test.tsx
  • yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-ellipse-settings.test.tsx
  • yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-polygon-settings.test.tsx
  • yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-function-settings.test.tsx

Storybook

… label, update type on condenseTextNodes params
…labels

When auto-generating the aria labels for locked figures, we want it to use
words as if they were spoken rather than math expressions that might be
read incorrectly by the screen reader.

- Use the `generateSpokenMathDetails` utility within LockedLineSettings,
  following the pattern used in LockedPointSettings for this.
- Update the locked line autogen function to include the labels
  on the locked points that define the locked line.

Issue: https://khanacademy.atlassian.net/browse/LEMS-2548

Test plan:
- `yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-line-settings.test.tsx`

Storybook
- Go to http://localhost:6006/iframe.html?args=&id=perseuseditor-widgets-interactive-graph--mafs-with-locked-figure-labels-all-flags&viewMode=story
- Open the locked line settings
- Change the visible label to hvae a mix of TeX (with `$...$`) and non-TeX
- Press the "Auto-generate" button
- Confirm that the input changes to include spoken math words for the TeX
- Also try this with no labels, multiple labels, and the labels changed
  for the locked points that define the line
…is clicked rather when the props are changed
…is clicked rather than when the props are changed
…n labels

When auto-generating the aria labels for locked figures, we want it to use
words as if they were spoken rather than math expressions that might be
read incorrectly by the screen reader.

- Use the `generateSpokenMathDetails` utility within LockedVectorSettings,
  LockedEllipseSettings, LockedPolygonSettings, and LockedFunctionSettings.

Issue: https://khanacademy.atlassian.net/browse/LEMS-2548

Test plan:
- `yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-vector-settings.test.tsx`
- `yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-ellipse-settings.test.tsx`
- `yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-polygon-settings.test.tsx`
- `yarn jest packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-function-settings.test.tsx`

Storybook
- Go to http://localhost:6006/iframe.html?args=&id=perseuseditor-widgets-interactive-graph--mafs-with-locked-figure-labels-all-flags&viewMode=story
- For each locked figure:
  - Open the figure settings
  - Change the visible label to have a mix of TeX and non-TeX
  - Press the "Auto-generate" button
  - Confirm that the input changes to include spoken math words for the TeX
  - Also try this with no labels, multiple labels, fully non-TeX, and
    fully teX labels.
@nishasy nishasy self-assigned this Nov 14, 2024
Copy link
Contributor

github-actions bot commented Nov 14, 2024

npm Snapshot: Published

Good news!! We've packaged up the latest commit from this PR (7ee43a8) and published it to npm. You
can install it using the tag PR1858.

Example:

yarn add @khanacademy/perseus@PR1858

If you are working in Khan Academy's webapp, you can run:

./dev/tools/bump_perseus_version.sh -t PR1858

Copy link
Contributor

github-actions bot commented Nov 14, 2024

Size Change: -65 B (-0.01%)

Total Size: 1.29 MB

Filename Size Change
packages/perseus-editor/dist/es/index.js 699 kB -65 B (-0.01%)
ℹ️ View Unchanged
Filename Size
packages/kas/dist/es/index.js 38.9 kB
packages/keypad-context/dist/es/index.js 760 B
packages/kmath/dist/es/index.js 4.27 kB
packages/math-input/dist/es/index.js 77.8 kB
packages/math-input/dist/es/strings.js 1.79 kB
packages/perseus-core/dist/es/index.js 1.48 kB
packages/perseus-linter/dist/es/index.js 22.2 kB
packages/perseus/dist/es/index.js 420 kB
packages/perseus/dist/es/strings.js 3.55 kB
packages/pure-markdown/dist/es/index.js 3.66 kB
packages/simple-markdown/dist/es/index.js 12.4 kB

compressed-size-action

@nishasy nishasy marked this pull request as ready for review November 14, 2024 20:51
@khan-actions-bot
Copy link
Contributor

khan-actions-bot commented Nov 14, 2024

Gerald

Required Reviewers
  • @Khan/perseus for changes to .changeset/rare-lamps-cough.md, .changeset/short-actors-sit.md, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-ellipse-settings.test.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-ellipse-settings.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-figure-aria.test.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-figure-aria.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-function-settings.test.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-function-settings.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-polygon-settings.test.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-polygon-settings.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-vector-settings.test.tsx, packages/perseus-editor/src/widgets/interactive-graph-editor/locked-figures/locked-vector-settings.tsx

Don't want to be involved in this pull request? Comment #removeme and we won't notify you of further changes.

@catandthemachines
Copy link
Member

Was playing around with the experience and I was noticing how it's reading out square roots. Is this expected behavior for it to read out "StartRoot one half EndRoot"? That doesn't feel natural, but maybe I'm not aware of the standard for reading out TeX.
image

Copy link
Member

@catandthemachines catandthemachines left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's ship it!

@@ -24,6 +24,14 @@ const defaultProps = {

const defaultLabel = getDefaultFigureForType("label");

// Mock the async function generateSpokenMathDetails
jest.mock("./util", () => ({
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Love this mocking to help us identify the generated spoken math details in testing ❤️

@nishasy
Copy link
Contributor Author

nishasy commented Nov 15, 2024

@catandthemachines I agree that it's a bit awkward, but I believe that's the intended behavior. I think it's to make it more clear where the square root starts and ends for the cases in which the radicand is particularly complex.

@nishasy nishasy changed the base branch from spoken-aria-lines to main November 15, 2024 19:50
An error occurred while trying to automatically change base from spoken-aria-lines to main November 15, 2024 19:50
@nishasy
Copy link
Contributor Author

nishasy commented Nov 15, 2024

The parent pull-request (#1854) has been merged into main, but this branch (spoken-math-others) now has conflicts with the new base branch. These conflicts must be resolved before checks can complete on this pull-request.

@nishasy nishasy merged commit 5e930ce into main Nov 15, 2024
9 checks passed
@nishasy nishasy deleted the spoken-math-others branch November 15, 2024 19:56
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants