Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

test(e2e): restore last e2e tests #7926

Merged
merged 10 commits into from
Dec 4, 2024
Merged

test(e2e): restore last e2e tests #7926

merged 10 commits into from
Dec 4, 2024

Conversation

RitaDias
Copy link
Contributor

@RitaDias RitaDias commented Dec 2, 2024

Description

  • We had some e2e tests that were disabled due to being unreliable while work with the Actions was on going. As things seem more stable, this PR brings back those tests.
  • Some updates had to be done to data-test ids since a lot of them were breaking (not correctly formatted, hence why the large number of files)

What to review

Review tests and make sure they make sense, I have tried reducing the flakiness but they are e2e tests so 🫠 any pointers would be appreciated.

Testing

This PR is just tests

Notes for release

N/A

Copy link

vercel bot commented Dec 2, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
page-building-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 4, 2024 1:04pm
performance-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 4, 2024 1:04pm
test-next-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 4, 2024 1:04pm
test-studio ✅ Ready (Inspect) Visit Preview 💬 Add feedback Dec 4, 2024 1:04pm
1 Skipped Deployment
Name Status Preview Comments Updated (UTC)
studio-workshop ⬜️ Ignored (Inspect) Visit Preview Dec 4, 2024 1:04pm

Copy link
Contributor

github-actions bot commented Dec 2, 2024

No changes to documentation

Copy link
Contributor

github-actions bot commented Dec 2, 2024

Component Testing Report Updated Dec 4, 2024 1:33 PM (UTC)

✅ All Tests Passed -- expand for details
File Status Duration Passed Skipped Failed
comments/CommentInput.spec.tsx ✅ Passed (Inspect) 1m 10s 15 0 0
formBuilder/ArrayInput.spec.tsx ✅ Passed (Inspect) 12s 3 0 0
formBuilder/inputs/PortableText/Annotations.spec.tsx ✅ Passed (Inspect) 39s 6 0 0
formBuilder/inputs/PortableText/copyPaste/CopyPaste.spec.tsx ✅ Passed (Inspect) 52s 11 7 0
formBuilder/inputs/PortableText/copyPaste/CopyPasteFields.spec.tsx ✅ Passed (Inspect) 0s 0 12 0
formBuilder/inputs/PortableText/Decorators.spec.tsx ✅ Passed (Inspect) 28s 6 0 0
formBuilder/inputs/PortableText/DisableFocusAndUnset.spec.tsx ✅ Passed (Inspect) 15s 3 0 0
formBuilder/inputs/PortableText/DragAndDrop.spec.tsx ✅ Passed (Inspect) 2m 33s 1 0 0
formBuilder/inputs/PortableText/FocusTracking.spec.tsx ✅ Passed (Inspect) 1m 9s 15 0 0
formBuilder/inputs/PortableText/Input.spec.tsx ✅ Passed (Inspect) 2m 43s 21 0 0
formBuilder/inputs/PortableText/ObjectBlock.spec.tsx ✅ Passed (Inspect) 1m 45s 18 0 0
formBuilder/inputs/PortableText/PresenceCursors.spec.tsx ✅ Passed (Inspect) 14s 3 9 0
formBuilder/inputs/PortableText/RangeDecoration.spec.tsx ✅ Passed (Inspect) 40s 9 0 0
formBuilder/inputs/PortableText/Styles.spec.tsx ✅ Passed (Inspect) 26s 6 0 0
formBuilder/inputs/PortableText/Toolbar.spec.tsx ✅ Passed (Inspect) 54s 12 0 0
formBuilder/tree-editing/TreeEditing.spec.tsx ✅ Passed (Inspect) 0s 0 3 0
formBuilder/tree-editing/TreeEditingNestedObjects.spec.tsx ✅ Passed (Inspect) 0s 0 3 0

Copy link
Contributor

github-actions bot commented Dec 2, 2024

⚡️ Editor Performance Report

Updated Wed, 04 Dec 2024 13:00:59 GMT

Benchmark reference
latency of sanity@latest
experiment
latency of this branch
Δ (%)
latency difference
article (title) 26.3 efps (38ms) 22.7 efps (44ms) +6ms (+15.8%)
article (body) 46.1 efps (22ms) 50.5 efps (20ms) -2ms (-8.8%)
article (string inside object) 24.7 efps (41ms) 26.3 efps (38ms) -3ms (-6.2%)
article (string inside array) 24.4 efps (41ms) 22.7 efps (44ms) +3ms (+7.3%)
recipe (name) 46.5 efps (22ms) 45.5 efps (22ms) +1ms (+2.3%)
recipe (description) 54.1 efps (19ms) 55.6 efps (18ms) -1ms (-2.7%)
recipe (instructions) 99.9+ efps (8ms) 99.9+ efps (7ms) -1ms (-/-%)
synthetic (title) 16.9 efps (59ms) 17.2 efps (58ms) -1ms (-1.7%)
synthetic (string inside object) 18.2 efps (55ms) 18.5 efps (54ms) -1ms (-1.8%)

efps — editor "frames per second". The number of updates assumed to be possible within a second.

Derived from input latency. efps = 1000 / input_latency

Detailed information

🏠 Reference result

The performance result of sanity@latest

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 38ms 41ms 45ms 200ms 222ms 10.4s
article (body) 22ms 25ms 34ms 184ms 248ms 6.1s
article (string inside object) 41ms 43ms 47ms 65ms 146ms 7.0s
article (string inside array) 41ms 43ms 52ms 197ms 147ms 7.0s
recipe (name) 22ms 23ms 24ms 37ms 0ms 7.7s
recipe (description) 19ms 20ms 22ms 36ms 0ms 4.7s
recipe (instructions) 8ms 10ms 11ms 18ms 0ms 3.5s
synthetic (title) 59ms 63ms 72ms 334ms 818ms 15.3s
synthetic (string inside object) 55ms 61ms 68ms 424ms 343ms 8.8s

🧪 Experiment result

The performance result of this branch

Benchmark latency p75 p90 p99 blocking time test duration
article (title) 44ms 46ms 49ms 158ms 205ms 11.1s
article (body) 20ms 23ms 27ms 200ms 237ms 6.3s
article (string inside object) 38ms 40ms 44ms 171ms 152ms 6.9s
article (string inside array) 44ms 48ms 56ms 73ms 149ms 7.2s
recipe (name) 22ms 26ms 29ms 46ms 0ms 7.6s
recipe (description) 18ms 20ms 22ms 33ms 0ms 4.7s
recipe (instructions) 7ms 9ms 10ms 17ms 0ms 3.4s
synthetic (title) 58ms 59ms 80ms 346ms 925ms 14.0s
synthetic (string inside object) 54ms 56ms 64ms 300ms 637ms 9.0s

📚 Glossary

column definitions

  • benchmark — the name of the test, e.g. "article", followed by the label of the field being measured, e.g. "(title)".
  • latency — the time between when a key was pressed and when it was rendered. derived from a set of samples. the median (p50) is shown to show the most common latency.
  • p75 — the 75th percentile of the input latency in the test run. 75% of the sampled inputs in this benchmark were processed faster than this value. this provides insight into the upper range of typical performance.
  • p90 — the 90th percentile of the input latency in the test run. 90% of the sampled inputs were faster than this. this metric helps identify slower interactions that occurred less frequently during the benchmark.
  • p99 — the 99th percentile of the input latency in the test run. only 1% of sampled inputs were slower than this. this represents the worst-case scenarios encountered during the benchmark, useful for identifying potential performance outliers.
  • blocking time — the total time during which the main thread was blocked, preventing user input and UI updates. this metric helps identify performance bottlenecks that may cause the interface to feel unresponsive.
  • test duration — how long the test run took to complete.

@RitaDias RitaDias changed the title tests(e2e): restore last e2e tests test(e2e): restore last e2e tests Dec 2, 2024
@RitaDias RitaDias marked this pull request as ready for review December 2, 2024 13:32
@RitaDias RitaDias requested a review from a team as a code owner December 2, 2024 13:32
@RitaDias RitaDias requested review from jordanl17 and juice49 and removed request for a team December 2, 2024 13:32
@jordanl17
Copy link
Member

@RitaDias to confirm - the e2es passing in github actions here on the PR is validation that these tests are all passing?

@RitaDias
Copy link
Contributor Author

RitaDias commented Dec 3, 2024

@RitaDias to confirm - the e2es passing in github actions here on the PR is validation that these tests are all passing?

Yep!

@@ -108,6 +108,7 @@ function PaneContextMenuItem(props: {disabled?: boolean; node: _PaneMenuItem}) {
pressed={node.selected}
text={title}
tone={node.tone}
data-testid={`action-${node.title?.toLocaleLowerCase().replace(' ', '')}`}
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this also be replaceAll instead of single replace?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah yes, it should! Good catch!
I'll fix this!

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Small related thought: would it make sense to move this to a tiny helper function? I noticed it's repeated a few times, and having the code inline like this doesn't make it super clear to me how/why the node title -> test id transformation works.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, that makes sense :)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Done!

@@ -108,6 +109,7 @@ function PaneContextMenuItem(props: {disabled?: boolean; node: _PaneMenuItem}) {
pressed={node.selected}
text={title}
tone={node.tone}
data-testid={`action-${toLowerCaseNoSpaces(node.title)}`}
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Noice 🙂.

juice49
juice49 previously approved these changes Dec 4, 2024
@RitaDias RitaDias enabled auto-merge December 4, 2024 13:27
@RitaDias RitaDias disabled auto-merge December 4, 2024 13:27
@RitaDias RitaDias merged commit b74b50e into next Dec 4, 2024
54 of 55 checks passed
@RitaDias RitaDias deleted the corel-262 branch December 4, 2024 13:34
bjoerge added a commit that referenced this pull request Dec 6, 2024
* next:
  test(sanity): update timeout to avoid slow test from being flaky (#7969)
  v3.66.1
  fix(deps): update dependency @sanity/presentation to v1.19.7 (#7966)
  fix(core): guard against missing process variable (#7967)
  fix(sanity): reset search cursor state when any parameter changes (#7889)
  fix(deps): allow react v19 as peer dependency (#7962)
  v3.66.0
  fix: reword warning about vector images not support crop/hotspot (#7960)
  fix(deps): Update dev-non-major (#7959)
  fix(core): add telemetry debug/noop logger (#7958)
  fix(deps): update dependency @sanity/presentation to v1.19.5 (#7957)
  fix(core): only open one annotation object edit modal at a time (#7898)
  fix(cli): remove v2 commands (#5750)
  feat(cli): generate read token conditionally for remote template (#7953)
  fix(deps): update dependency @sanity/mutate to ^0.11.1 (#7903)
  fix(deps): update dependency @sanity/ui to ^2.9.1 (#7948)
  feat(vision): link to edit documents on `_id`, `_ref` attributes (#6124)
  fix(core): disable scheduledPublishing and tasks if `/features` returns error (#7517)
  test(e2e): restore last e2e tests (#7926)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants