Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

追加: TTSEngine スナップショット #1013

Merged
merged 3 commits into from
Feb 4, 2024

Conversation

tarepan
Copy link
Contributor

@tarepan tarepan commented Jan 17, 2024

内容

TTSEngine 各メソッドのスナップショットを追加

関連 Issue

ref #978
ref #899

@tarepan tarepan requested a review from a team as a code owner January 17, 2024 11:01
@tarepan tarepan requested review from y-chan and removed request for a team January 17, 2024 11:01
@tarepan
Copy link
Contributor Author

tarepan commented Jan 17, 2024

test / test (ubuntu-20.04, 3.11.3) (pull_request) Successful in 1m
test / test (windows-latest, 3.11.3) (pull_request) Failing after 2m

=========================== short test summary info ===========================
FAILED test/tts_pipeline/test_tts_engine.py::test_mocked_synthesize_wave_output - assert [- snapshot] == [+ received]
    ......
      ...
        [
  -       1.30066,
  +       1.30067,
  -       1.30066
  +       1.30067
        ],
      ...
    ......
================== 1 failed, 131 passed, 5 warnings in 4.33s ==================

スナップショット値が Linux と Windows で一致しない...?

@Hiroshiba
数値計算周りで Linux/Windows 環境依存が問題になるケースって ENGINE で既出だったりしますか?

@Hiroshiba
Copy link
Member

Hiroshiba commented Feb 2, 2024

@tarepan

数値計算周りで Linux/Windows 環境依存が問題になるケースって ENGINE で既出だったりしますか?

onnxruntimeを使っている場合割と OS 依存で値が変わることはあったのですが、見たところモックを使っているので初めての現象な気がします!!!
こんなことありえるんですね!!!

問題はPythonかnumpyのどこかがfloat32だったりfloat64だったりすることな気がしています。
Pythonのfloatはデフォルトがdoubleで、numpyもdoubleなはずです。でもこれは一応実行環境のアーキテクチャーに依存しています。
もしかしたらGithub Actionsのubuntu20.04はfloat32で、windowsはflota64なのかもですが、そんなことある・・・?って気もしています。不明です。

とりあえずfloat32になってたり64になってたりしそうなところの第一候補は_gen_hello_hiho_queryで、↓みたいな感じでfloat32にすれば両方とも値が合うかもです。
https://github.com/tarepan/voicevox_engine/blob/c67bc2f26363f58d1de65279fb95b1cb3ac92ef9/test/tts_pipeline/test_tts_engine.py#L420-L421

あるいはもう雑に他と同様にround_floats(round_value=2)にしちゃう手もあるかもです。
upspeak辺りで += 0.3 する処理がありますが、0.3はfloat32とfloat64で微妙に値が違って計算誤差が出ると思います。
float32とfloat64が交じる場合は、round_value=5は難しいのかもしれません。

Copy link

github-actions bot commented Feb 3, 2024

Coverage Result

Resultを開く
Name Stmts Miss Cover
run.py 508 318 coverage-37%
voicevox_engine/init.py 1 0 coverage-100%
voicevox_engine/cancellable_engine.py 94 72 coverage-23%
voicevox_engine/core/core_adapter.py 81 29 coverage-64%
voicevox_engine/core/core_initializer.py 59 30 coverage-49%
voicevox_engine/core/core_wrapper.py 257 183 coverage-29%
voicevox_engine/dev/core/init.py 2 0 coverage-100%
voicevox_engine/dev/core/mock.py 36 4 coverage-89%
voicevox_engine/dev/tts_engine/init.py 2 0 coverage-100%
voicevox_engine/dev/tts_engine/mock.py 28 0 coverage-100%
voicevox_engine/engine_manifest/EngineManifest.py 36 0 coverage-100%
voicevox_engine/engine_manifest/EngineManifestLoader.py 12 0 coverage-100%
voicevox_engine/engine_manifest/init.py 3 0 coverage-100%
voicevox_engine/library_manager.py 92 4 coverage-96%
voicevox_engine/metas/Metas.py 36 0 coverage-100%
voicevox_engine/metas/MetasStore.py 28 6 coverage-79%
voicevox_engine/metas/init.py 2 0 coverage-100%
voicevox_engine/model.py 180 9 coverage-95%
voicevox_engine/morphing.py 71 46 coverage-35%
voicevox_engine/part_of_speech_data.py 5 0 coverage-100%
voicevox_engine/preset/Preset.py 13 0 coverage-100%
voicevox_engine/preset/PresetError.py 2 0 coverage-100%
voicevox_engine/preset/PresetManager.py 80 2 coverage-98%
voicevox_engine/preset/init.py 4 0 coverage-100%
voicevox_engine/setting/Setting.py 11 0 coverage-100%
voicevox_engine/setting/SettingLoader.py 17 0 coverage-100%
voicevox_engine/setting/init.py 3 0 coverage-100%
voicevox_engine/tts_pipeline/acoustic_feature_extractor.py 34 0 coverage-100%
voicevox_engine/tts_pipeline/kana_converter.py 88 1 coverage-99%
voicevox_engine/tts_pipeline/mora_list.py 7 0 coverage-100%
voicevox_engine/tts_pipeline/text_analyzer.py 146 6 coverage-96%
voicevox_engine/tts_pipeline/tts_engine.py 264 84 coverage-68%
voicevox_engine/user_dict.py 145 12 coverage-92%
voicevox_engine/utility/init.py 5 0 coverage-100%
voicevox_engine/utility/connect_base64_waves.py 37 0 coverage-100%
voicevox_engine/utility/core_version_utility.py 8 1 coverage-88%
voicevox_engine/utility/mutex_utility.py 13 0 coverage-100%
voicevox_engine/utility/path_utility.py 26 8 coverage-69%
voicevox_engine/utility/run_utility.py 10 7 coverage-30%
TOTAL 2446 822 coverage-66%

@tarepan
Copy link
Contributor Author

tarepan commented Feb 3, 2024

float32だったりfloat64だったり ... 一応実行環境のアーキテクチャーに依存

貴重な情報ありがとうございます。

あるいはもう雑に他と同様にround_floats(round_value=2)にしちゃう

👍
影響要素がありすぎるので、round_value=2 でラフに解決します。

テストパスを確認しました。Reviewよろしくお願いします。

@tarepan tarepan mentioned this pull request Feb 3, 2024
1 task
Copy link
Member

@Hiroshiba Hiroshiba left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!!!

test/tts_pipeline/test_tts_engine.py Show resolved Hide resolved
@Hiroshiba Hiroshiba merged commit 72da867 into VOICEVOX:master Feb 4, 2024
3 checks passed
@tarepan tarepan deleted the add/tts_snapshot branch February 6, 2024 12:06
@tarepan tarepan mentioned this pull request Feb 21, 2024
1 task
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants