Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: liberate VOICEVOX CORE #825

Draft
wants to merge 35 commits into
base: main
Choose a base branch
from
Draft
Show file tree
Hide file tree
Changes from 4 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
c6e0935
change: liberate VOICEVOX CORE
qryxip Aug 26, 2024
6e1cafb
ortをアップデート
qryxip Aug 31, 2024
a5f009c
`bin` → `vv-bin`
qryxip Aug 31, 2024
92fb4c9
ortをアップデート
qryxip Aug 31, 2024
38687d5
ortをアップデート
qryxip Sep 2, 2024
ff70980
`"type": "vv-bin"` → `"type": "vv_bin"`
qryxip Sep 2, 2024
b87afd8
`inputs.is_production`を残す
qryxip Sep 2, 2024
80345e5
嘘コメントをリバート
qryxip Sep 4, 2024
148fa5d
`install_name_tool`でのrpath変更をvoicevox_onnxruntime.framework宛てに
qryxip Sep 4, 2024
688c03f
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 5, 2024
89ea1f9
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 6, 2024
accb03f
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 12, 2024
026ff6f
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 19, 2024
c5db7b8
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 19, 2024
d5bce52
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 19, 2024
0b63929
Merge branch 'main' into change-liberate-voicevox-core
qryxip Sep 26, 2024
b1175b2
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 3, 2024
bb0c57f
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 4, 2024
012ac5a
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 8, 2024
5cf7fcb
Fix a test
qryxip Oct 8, 2024
929fec2
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 13, 2024
10a6111
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 13, 2024
c61141c
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 13, 2024
7780c34
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 14, 2024
17fedd5
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 21, 2024
2579467
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 23, 2024
5187f9e
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 25, 2024
f9e3cde
Merge branch 'main' into change-liberate-voicevox-core
qryxip Oct 29, 2024
09ffd63
Merge branch 'main' into change-liberate-voicevox-core
qryxip Nov 2, 2024
21e3e79
Merge branch 'main' into change-liberate-voicevox-core
qryxip Nov 27, 2024
3fa5a64
Merge branch 'main' into change-liberate-voicevox-core
qryxip Nov 29, 2024
ea4885e
Merge branch 'main' into change-liberate-voicevox-core
qryxip Dec 10, 2024
c5617c8
Merge branch 'main' into change-liberate-voicevox-core
qryxip Dec 12, 2024
9fde4e7
Merge branch 'main' into change-liberate-voicevox-core
qryxip Dec 12, 2024
a7599dd
Merge branch 'main' into change-liberate-voicevox-core
qryxip Dec 20, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
81 changes: 7 additions & 74 deletions .github/workflows/build_and_deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -16,11 +16,6 @@ on:
type: boolean
required: false
default: false
is_production:
description: "製品版をビルドする"
type: boolean
required: false
default: false
release:
types:
- published
Expand Down Expand Up @@ -158,30 +153,15 @@ jobs:

build_and_deploy:
needs: config
environment: ${{ inputs.is_production && 'production' || '' }} # 製品版のenvironment
environment: ${{ inputs.code_signing && 'production' || '' }} # コード署名用のenvironment
strategy:
matrix:
include: ${{ fromJson(needs.config.outputs.includes) }}
runs-on: ${{ matrix.os }}
env:
ASSET_NAME: voicevox_core-${{ matrix.artifact_name }}-${{ needs.config.outputs.version }}
steps:
- uses: actions/checkout@v4 # 製品版ではない場合
if: ${{ !inputs.is_production }}
- uses: actions/checkout@v4 # 製品版の場合
if: inputs.is_production
with:
fetch-depth: 0 # 全履歴取得
token: ${{ secrets.PRODUCTION_GITHUB_TOKEN }}
- name: Merge production branch
if: inputs.is_production
shell: bash
run: |
(
git remote add private ${{ secrets.PRODUCTION_REPOSITORY_URL }}
git fetch private refs/tags/${{ env.PRODUCTION_REPOSITORY_TAG }}
git -c user.name=dummy -c [email protected] merge FETCH_HEAD
) > /dev/null 2>&1
- uses: actions/checkout@v4
- name: Set up Python 3.8
if: matrix.python_whl
uses: actions/setup-python@v5
Expand All @@ -206,18 +186,6 @@ jobs:
run: |
echo "$ANDROID_NDK/toolchains/llvm/prebuilt/linux-x86_64/bin" >> "$GITHUB_PATH"
echo "AR_${{ matrix.target }}=llvm-ar" >> "$GITHUB_ENV"
- name: Checkout VOICEVOX RESOURCE
if: inputs.is_production
uses: actions/checkout@v4
with:
repository: VOICEVOX/voicevox_resource
ref: ${{ env.VOICEVOX_RESOURCE_VERSION }}
path: download/resource
- name: Raplace resource
if: inputs.is_production
shell: bash
run:
mv -f download/resource/core/README.md ./README.md
Hiroshiba marked this conversation as resolved.
Show resolved Hide resolved
- name: Install cargo-binstall
uses: taiki-e/install-action@cargo-binstall
- name: Install cargo-edit
Expand All @@ -228,22 +196,14 @@ jobs:
if ${{ matrix.python_whl }}; then cargo set-version "$VERSION" -p voicevox_core_python_api; fi
- name: cache target
uses: Swatinem/rust-cache@v2
if: ${{ !inputs.is_production }}
- name: build voicevox_core_c_api
shell: bash
run: |
case ${{ matrix.c_release_format }} in
plain-cdylib) linking=load-onnxruntime ;;
ios-xcframework) linking=link-onnxruntime ;;
esac
function build() {
cargo build -p voicevox_core_c_api -vv --features "$linking" --target ${{ matrix.target }} --release
}
if ${{ !inputs.is_production }}; then
build
else
build > /dev/null 2>&1
fi
cargo build -p voicevox_core_c_api -vv --features "$linking" --target ${{ matrix.target }} --release
env:
RUSTFLAGS: -C panic=abort
- name: build voicevox_core_python_api
Expand All @@ -254,26 +214,11 @@ jobs:
pip install --upgrade poetry
poetry config virtualenvs.create false
(cd crates/voicevox_core_python_api && poetry install --with dev)
function build() {
maturin build --manifest-path ./crates/voicevox_core_python_api/Cargo.toml --target ${{ matrix.target }} --release
}
if ${{ !inputs.is_production }}; then
build
else
build > /dev/null 2>&1
fi
maturin build --manifest-path ./crates/voicevox_core_python_api/Cargo.toml --target ${{ matrix.target }} --release
echo "whl=$(find ./target/wheels -type f)" >> "$GITHUB_OUTPUT"
- name: build voicevox_core_java_api
if: contains(matrix.target, 'android')
run: |
function build() {
cargo build -p voicevox_core_java_api -vv --target ${{ matrix.target }} --release
}
if ${{ !inputs.is_production }}; then
build
else
build > /dev/null 2>&1
fi
run: cargo build -p voicevox_core_java_api -vv --target ${{ matrix.target }} --release
- name: Organize artifact
run: |
mkdir -p "artifact/${{ env.ASSET_NAME }}"
Expand Down Expand Up @@ -377,25 +322,13 @@ jobs:
${{ env.ASSET_NAME }}.zip
target_commitish: ${{ github.sha }}

deploy_model:
deploy_sample_model:
runs-on: ubuntu-latest
needs: config
env:
ASSET_NAME: model-${{ needs.config.outputs.version }}
ASSET_NAME: sample-model-${{ needs.config.outputs.version }}
steps:
- uses: actions/checkout@v4
- name: Checkout VOICEVOX FAT RESOURCE
if: inputs.is_production
uses: actions/checkout@v4
with:
repository: VOICEVOX/voicevox_fat_resource
ref: ${{ env.VOICEVOX_FAT_RESOURCE_VERSION }}
path: download/fat_resource
- name: Raplace resource
if: inputs.is_production
shell: bash
run:
rm -r ./model; mv download/fat_resource/core/model ./model
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

この変更によってダウンローダーが製品版のVVMをダウンロードできなくなるので、最新版のmainブランチを見て製品版のVVMを使う方法がわからなくなったかもです。
ちょっとどういう運用にするか迷いますが、とりあえずダウンローダーがダウンロードできるものは製品版のままにしておくのはどうでしょう。

(将来的にはVVM置き場を別で作ったりするのもアリ。全部ダウンロードすると重いし。)

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

#825 (comment)と重複しますが、このPRでダウンローダから直接voicevox_fat_resouceのファイルをダウンロードするようにすることを考えています。どうでしょうか?

Copy link
Member

@Hiroshiba Hiroshiba Sep 2, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

今の範囲でレビューしちゃったので、可能なら分けていただけると助かる思いはあります!
他にも理由は2つほどありそうでした。

  • このPRは製品版コアのパッチ取得を辞めることが主目的だと思うので、VVMダウンロード方法とは独立してる
  • fat_resourceの運用が定まっておらず、あとからどうせコア側の変更が必要になるかもしれない(fat_resourceのバージョン管理とか)

まあでもどうしてもそうしてほしいというレベルじゃないです。must/should/want/canのshouldから少しwant寄りくらい!

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

わかりました。このPRでダウンローダは手を付けないことにします。

- name: Create artifact
run: |
mkdir "artifact"
Expand Down
7 changes: 1 addition & 6 deletions .github/workflows/build_and_deploy_downloader.yml
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,6 @@ on:
type: boolean
required: false
default: false
is_production:
description: "製品版をビルドする"
type: boolean
required: false
default: false
release:
types:
- published
Expand All @@ -42,7 +37,7 @@ defaults:

jobs:
deploy_and_deploy_downloader:
environment: ${{ inputs.is_production && 'production' || '' }} # コード署名用のenvironment
environment: ${{ inputs.code_signing && 'production' || '' }} # コード署名用のenvironment
strategy:
matrix:
include:
Expand Down
11 changes: 3 additions & 8 deletions .github/workflows/test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -284,17 +284,12 @@ jobs:
- run: cargo build -p test_util -vv # build scriptにより/crates/test_util/data/の生成
- run: poetry run maturin build --locked
- run: poetry run maturin develop --locked
- name: 必要なDLLをコピーしてpytestを実行
run: |
cp -v ../../target/debug/onnxruntime.dll . || true
cp -v ../../target/debug/libonnxruntime.so.* . || true
cp -v ../../target/debug/libonnxruntime.*.dylib . || true

poetry run pytest
- name: pytestを実行
run: poetry run pytest
- name: Exampleを実行
run: |
for file in ../../example/python/run{,-asyncio}.py; do
poetry run python "$file" ../test_util/data/model/sample.vvm --dict-dir ../test_util/data/open_jtalk_dic_utf_8-1.11
poetry run python "$file" ../test_util/data/model/sample.vvm --dict-dir ../test_util/data/open_jtalk_dic_utf_8-1.11 --onnxruntime ../test_util/data/lib/*onnxruntime*
done
build-and-test-java-api:
strategy:
Expand Down
4 changes: 2 additions & 2 deletions Cargo.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

4 changes: 2 additions & 2 deletions Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -91,8 +91,8 @@ windows = "0.43.0"
zip = "0.6.3"

[workspace.dependencies.voicevox-ort]
git = "https://github.com/VOICEVOX/ort.git"
rev = "8627833456a69e7841ae2a29fd184752df8de8d9"
git = "https://github.com/qryxip/ort.git"
rev = "5b5b22a135bd5257b31df9156d1cb6a51fbf92f0"
qryxip marked this conversation as resolved.
Show resolved Hide resolved

[workspace.dependencies.open_jtalk]
git = "https://github.com/VOICEVOX/open_jtalk-rs.git"
Expand Down
7 changes: 1 addition & 6 deletions crates/voicevox_core/src/devices.rs
Original file line number Diff line number Diff line change
Expand Up @@ -57,12 +57,7 @@ fn test_gpu(
/// use voicevox_core::{tokio::Onnxruntime, SupportedDevices};
///
/// # voicevox_core::blocking::Onnxruntime::load_once()
/// # .filename(if cfg!(windows) {
/// # // Windows\System32\onnxruntime.dllを回避
/// # test_util::ONNXRUNTIME_DYLIB_PATH
/// # } else {
/// # voicevox_core::blocking::Onnxruntime::LIB_VERSIONED_FILENAME
/// # })
/// # .filename(test_util::ONNXRUNTIME_DYLIB_PATH)
/// # .exec()?;
/// #
/// let onnxruntime = Onnxruntime::get().unwrap();
Expand Down
8 changes: 2 additions & 6 deletions crates/voicevox_core/src/infer.rs
Original file line number Diff line number Diff line change
@@ -1,5 +1,4 @@
pub(crate) mod domains;
mod model_file;
pub(crate) mod runtimes;
pub(crate) mod session_set;

Expand All @@ -13,6 +12,7 @@ use thiserror::Error;

use crate::{
devices::{DeviceSpec, GpuSpec},
voice_model::ModelBytes,
StyleType, SupportedDevices,
};

Expand All @@ -33,7 +33,7 @@ pub(crate) trait InferenceRuntime: 'static {
#[allow(clippy::type_complexity)]
fn new_session(
&self,
model: impl FnOnce() -> std::result::Result<Vec<u8>, DecryptModelError>,
model: &ModelBytes,
options: InferenceSessionOptions,
) -> anyhow::Result<(
Self::Session,
Expand Down Expand Up @@ -204,7 +204,3 @@ pub(crate) enum ExtractError {
#[error(transparent)]
Shape(#[from] ShapeError),
}

#[derive(Error, Debug)]
#[error("不正なモデルファイルです")]
pub(crate) struct DecryptModelError;
5 changes: 0 additions & 5 deletions crates/voicevox_core/src/infer/model_file.rs

This file was deleted.

37 changes: 18 additions & 19 deletions crates/voicevox_core/src/infer/runtimes/onnxruntime.rs
Original file line number Diff line number Diff line change
Expand Up @@ -11,11 +11,12 @@ use ort::{
use crate::{
devices::{DeviceSpec, GpuSpec, SupportedDevices},
error::ErrorRepr,
voice_model::ModelBytes,
};

use super::super::{
DecryptModelError, InferenceRuntime, InferenceSessionOptions, InputScalarKind,
OutputScalarKind, OutputTensor, ParamInfo, PushInputTensor,
InferenceRuntime, InferenceSessionOptions, InputScalarKind, OutputScalarKind, OutputTensor,
ParamInfo, PushInputTensor,
};

// TODO: `trait AsyncRuntime`みたいなものを作って抽象化しながら同期版と非同期版に別個の役割を
Expand Down Expand Up @@ -62,7 +63,7 @@ impl InferenceRuntime for self::blocking::Onnxruntime {

fn new_session(
&self,
model: impl FnOnce() -> std::result::Result<Vec<u8>, DecryptModelError>,
model: &ModelBytes,
options: InferenceSessionOptions,
) -> anyhow::Result<(
Self::Session,
Expand All @@ -86,8 +87,10 @@ impl InferenceRuntime for self::blocking::Onnxruntime {
}
};

let model = model()?;
let sess = builder.commit_from_memory(&{ model })?;
let sess = match model {
ModelBytes::Onnx(onnx) => builder.commit_from_memory(onnx),
ModelBytes::VvBin(bin) => builder.commit_from_vv_bin(bin),
}?;

let input_param_infos = sess
.inputs
Expand Down Expand Up @@ -261,12 +264,10 @@ pub(crate) mod blocking {
/// # use voicevox_core as another_lib;
/// #
/// # fn main() -> anyhow::Result<()> {
/// # if cfg!(windows) {
/// # // Windows\System32\onnxruntime.dllを回避
/// # voicevox_core::blocking::Onnxruntime::load_once()
/// # .filename(test_util::ONNXRUNTIME_DYLIB_PATH)
/// # .exec()?;
/// # }
/// # voicevox_core::blocking::Onnxruntime::load_once()
/// # .filename(test_util::ONNXRUNTIME_DYLIB_PATH)
/// # .exec()?;
/// #
/// let ort1 = voicevox_core::blocking::Onnxruntime::load_once().exec()?;
/// let ort2 = another_lib::tokio::Onnxruntime::get().expect("`ort1`と同一のはず");
/// assert_eq!(ptr_addr(ort1), ptr_addr(ort2));
Expand All @@ -289,7 +290,7 @@ pub(crate) mod blocking {
/// ONNX Runtimeのライブラリ名。
#[cfg(feature = "load-onnxruntime")]
#[cfg_attr(docsrs, doc(cfg(feature = "load-onnxruntime")))]
pub const LIB_NAME: &'static str = "onnxruntime";
pub const LIB_NAME: &'static str = "voicevox_onnxruntime";

/// 推奨されるONNX Runtimeのバージョン。
#[cfg(feature = "load-onnxruntime")]
Expand Down Expand Up @@ -450,12 +451,10 @@ pub(crate) mod tokio {
/// #
/// # #[tokio::main]
/// # async fn main() -> anyhow::Result<()> {
/// # if cfg!(windows) {
/// # // Windows\System32\onnxruntime.dllを回避
/// # voicevox_core::blocking::Onnxruntime::load_once()
/// # .filename(test_util::ONNXRUNTIME_DYLIB_PATH)
/// # .exec()?;
/// # }
/// # voicevox_core::blocking::Onnxruntime::load_once()
/// # .filename(test_util::ONNXRUNTIME_DYLIB_PATH)
/// # .exec()?;
/// #
/// let ort1 = voicevox_core::tokio::Onnxruntime::load_once().exec().await?;
/// let ort2 = another_lib::blocking::Onnxruntime::get().expect("`ort1`と同一のはず");
/// assert_eq!(ptr_addr(ort1), ptr_addr(ort2));
Expand All @@ -477,7 +476,7 @@ pub(crate) mod tokio {
#[cfg(feature = "load-onnxruntime")]
#[cfg_attr(docsrs, doc(cfg(feature = "load-onnxruntime")))]
// ブロッキング版と等しいことはテストで担保
pub const LIB_NAME: &'static str = "onnxruntime";
pub const LIB_NAME: &'static str = "voicevox_onnxruntime";

/// 推奨されるONNX Runtimeのバージョン。
#[cfg(feature = "load-onnxruntime")]
Expand Down
8 changes: 4 additions & 4 deletions crates/voicevox_core/src/infer/session_set.rs
Original file line number Diff line number Diff line change
Expand Up @@ -4,10 +4,10 @@ use anyhow::bail;
use enum_map::{Enum as _, EnumMap};
use itertools::Itertools as _;

use crate::error::ErrorRepr;
use crate::{error::ErrorRepr, voice_model::ModelBytes};

use super::{
model_file, InferenceDomain, InferenceInputSignature, InferenceOperation, InferenceRuntime,
InferenceDomain, InferenceInputSignature, InferenceOperation, InferenceRuntime,
InferenceSessionOptions, InferenceSignature, ParamInfo,
};

Expand All @@ -18,7 +18,7 @@ pub(crate) struct InferenceSessionSet<R: InferenceRuntime, D: InferenceDomain>(
impl<R: InferenceRuntime, D: InferenceDomain> InferenceSessionSet<R, D> {
pub(crate) fn new(
rt: &R,
model_bytes: &EnumMap<D::Operation, Vec<u8>>,
model_bytes: &EnumMap<D::Operation, ModelBytes>,
options: &EnumMap<D::Operation, InferenceSessionOptions>,
) -> anyhow::Result<Self> {
let mut sessions = model_bytes
Expand All @@ -28,7 +28,7 @@ impl<R: InferenceRuntime, D: InferenceDomain> InferenceSessionSet<R, D> {
<D::Operation as InferenceOperation>::PARAM_INFOS[op];

let (sess, actual_input_param_infos, actual_output_param_infos) =
rt.new_session(|| model_file::decrypt(model_bytes), options[op])?;
rt.new_session(model_bytes, options[op])?;

check_param_infos(expected_input_param_infos, &actual_input_param_infos)?;
check_param_infos(expected_output_param_infos, &actual_output_param_infos)?;
Expand Down
2 changes: 1 addition & 1 deletion crates/voicevox_core/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
//! 開きます。[CUDA]と[DirectML]が利用できます。
//! - **`link-onnxruntime`**: ONNX Runtimeをロード時動的リンクします。iOSのような`dlopen`の利用が
//! 困難な環境でのみこちらを利用するべきです。_Note_:
//! [動的リンク対象のライブラリ名]は`onnxruntime`で固定です。変更
//! [動的リンク対象のライブラリ名]は`voicevox_onnxruntime`で固定です。変更
//! は`patchelf(1)`や`install_name_tool(1)`で行ってください。また、[ONNX RuntimeのGPU機能]を使う
//! ことはできません。
//!
Expand Down
Loading
Loading