Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

perf: MkImgWithBlurhashとMkMediaImageを最適化 #10782

Merged
merged 65 commits into from
May 19, 2023
Merged

perf: MkImgWithBlurhashとMkMediaImageを最適化 #10782

merged 65 commits into from
May 19, 2023

Conversation

tamaina
Copy link
Contributor

@tamaina tamaina commented May 6, 2023 •
edited
Loading

Resolve #10781
Resolve #10860

What

  • Blurhashをburahaで描画・Workerで描画するように
  • MkImgWithBlurhash周りのパフォーマンスと描画のチューニング
  • MkMediaImageをMkImgWithBlurhashを再利用するように変更

Why

パフォーマンス向上

Additional info (optional)

Checklist

  • Read the contribution guide
  • Test working in a local environment
  • (If needed) Add story of storybook
  • (If needed) Update CHANGELOG.md
  • (If possible) Add tests

@github-actions github-actions bot added the packages/frontend Client side specific issue/PR label May 6, 2023
@codecov
Copy link

codecov bot commented May 6, 2023 •
edited
Loading

Codecov Report

Merging #10782 (104f878) into develop (dd60f1a) will decrease coverage by 0.95%.
The diff coverage is n/a.

@@             Coverage Diff             @@
##           develop   #10782      +/-   ##
===========================================
- Coverage    77.86%   76.91%   -0.95%     
===========================================
  Files          164      736     +572     
  Lines        20290    69802   +49512     
  Branches       298     6277    +5979     
===========================================
+ Hits         15798    53690   +37892     
- Misses        4492    16112   +11620

see 900 files with indirect coverage changes

@tamaina
Copy link
Contributor Author

tamaina commented May 6, 2023

なんかよくわからんけどSafariでOffscreenCanvasがバグり散らかしてる

@tamaina
Copy link
Contributor Author

tamaina commented May 6, 2023

Safariだとこう↓なったりそもそも真っ暗だったりする

image

@tamaina
Copy link
Contributor Author

tamaina commented May 6, 2023

画面上に最初から表示されている場合は正常に表示されるのがさらに謎

@tamaina
Copy link
Contributor Author

tamaina commented May 7, 2023

データセーバー

srcを指定しなければいいしデータセーバー有効時は指定しないようにした

@tamaina
Copy link
Contributor Author

tamaina commented May 7, 2023

バグ

  • Safariでcanvasがバグり散らかす
  • devかつFirefoxで起動するとworkerのimportでエラーが出る

気になったこと

  • v-elseでimgを表示するときにChromeで見るとめちゃくちゃチラチラする（v-showにしてloaderを削除するといい感じになるけどgalleryのトランジションが正しく動作しない）

@tamaina
Copy link
Contributor Author

tamaina commented May 7, 2023

TODO

  • 閲覧注意の背景を暗くする

@tamaina tamaina mentioned this pull request May 7, 2023
Closed
@tamaina
Copy link
Contributor Author

tamaina commented May 7, 2023

Safariだとこう↓なったりそもそも真っ暗だったりする
Safariでcanvasがバグり散らかす

完成したImageBitmapを転送することでとりあえず対処…
転送しない版→ #10790

v-elseでimgを表示するときにChromeで見るとめちゃくちゃチラチラする

duration指定するなどしていい感じに動くようにした

@tamaina tamaina marked this pull request as ready for review May 7, 2023 12:24
@tamaina
Copy link
Contributor Author

tamaina commented May 17, 2023

mount前にbitmap作成はmount後にOffscreenCanvas直接描画より遅いことが判明した

@acid-chicken
Copy link
Member

もう mounted 時に HTMLCanvasElement 直接マウントしようぜ

@tamaina
Copy link
Contributor Author

tamaina commented May 17, 2023

CPU制限なしでChrome/Firefoxを比較したところ結果が逆転した

こんなに遅いイメージがない（ファイル数多いから？）

Chrome.bh-worker.mp4
Firefox.bh-worker.mp4

@tamaina
Copy link
Contributor Author

tamaina commented May 17, 2023

こんなに遅いイメージがない（ファイル数多いから？）

ファイル数を増やすとFirefoxが遅くなりはじめた

（OffscreenCanvasで変更された内容が一斉に適用されるようになっている？？）

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@tamaina
Copy link
Contributor Author

tamaina commented May 17, 2023

最初は average color だけ表示しとく

HTMLCanvasElementではコンテキストを作るのがnative canvas?かOffscreenCanvasかの二択となっており…と思ったけどつまりImageBitmap転送式で最初に描画すればいいということか

もう mounted 時に HTMLCanvasElement 直接マウントしようぜ

TransitionGroupの兼ね合いを考えると無理そう

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

1 similar comment
@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@tamaina
Copy link
Contributor Author

tamaina commented May 17, 2023

mount前にbitmap作成はmount後にOffscreenCanvas直接描画より遅いことが判明した

HTMLCanvasElementではコンテキストを作るのがnative canvas?かOffscreenCanvasかの二択となっており…と思ったけどつまりImageBitmap転送式で最初に描画すればいいということか

createImageBitmap()を使っているからだった

@github-actions
Copy link
Contributor

Chromatic will skip testing but you may still have to review the changes on Chromatic.

@tamaina
Copy link
Contributor Author

tamaina commented May 18, 2023 •
edited
Loading

cqh周りのバグが解消される(Firefox 115)まで縦長デフォルト表示がバギーなんだけど、初期値を1:1にするとかもありうる

https://bugzilla.mozilla.org/show_bug.cgi?id=1832481

@tamaina
Copy link
Contributor Author

tamaina commented May 18, 2023

  • blurhash描画前にextractAvgColorFromBlurhashでfillRect
  • 複数のWorkerを生やすようにした（最大4つ）
  • MkAvatarのimgをMkImgWithBlurhashに

@syuilo
Copy link
Member

syuilo commented May 18, 2023

YOSASOU

@syuilo syuilo merged commit 59255e1 into develop May 19, 2023
@syuilo syuilo deleted the bh-worker branch May 19, 2023 00:44
@syuilo
Copy link
Member

syuilo commented May 19, 2023

👍🏻👍🏻👍🏻

sasagar pushed a commit to sasagar/misskey that referenced this pull request Jun 5, 2023
@tamaina @syuilo @sasagar
perf: MkImgWithBlurhashとMkMediaImageを最適化 (misskey-dev#10782)
ba78042 
* misskey-dev#10781

* fix tsconfig

* fetch image??

* Revert "fetch image??"

This reverts commit 0925c28.

* wip

* Revert "wip"

This reverts commit be97c6c.

* loading="eager"

* loading="eager" 2

* error

* wip

* wip

* wip

* wip

* clean up

* fix

* 生成するworkerを1つにする?

* clean up

* use buraha

* wip

* smaller width, height

* update buraha

* clean up

* fix

* Update MkMediaImage.vue

* Update MkImgWithBlurhash.vue

* Revert "fix(frontend): センシティブ設定された画像を開くとき一瞬レイアウトが崩れる問題を修正"

This reverts commit 41e9aa6.

* Update MkMediaList.vue

* Update MkMediaList.vue

* Update MkMediaList.vue

* Update CHANGELOG.md

* wait for decode

* fix

* ?

* (test) remove container-type: inline-size;

* Revert "(test) remove container-type: inline-size;"

This reverts commit 9448e64.

* container-name

* Revert "container-name"

This reverts commit 94385d3.

* width: 100%;

* improve performance

* refactor

* wip

* WIP

* wip

* Revert "wip"

This reverts commit 36e3b75.

* Revert "WIP"

This reverts commit 05b729e.

* Revert "wip"

This reverts commit 0801e79.

* misskey-dev#10860

* wip

* no worker

* Revert "no worker"

This reverts commit a9c49e4.

* ✌️

* workerNumber固定は不要

---------

Co-authored-by: syuilo <[email protected]>
@tamaina tamaina mentioned this pull request Jun 6, 2023
Closed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@syuilo syuilo syuilo left review comments

@kakkokari-gtyih kakkokari-gtyih kakkokari-gtyih left review comments

@acid-chicken acid-chicken Awaiting requested review from acid-chicken

Assignees
No one assigned
Labels
packages/frontend Client side specific issue/PR
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

MkAvatar も buraha を使う MkImgWithBlurhashのBlurhash描画の最適化
4 participants
@tamaina @acid-chicken @syuilo @kakkokari-gtyih