Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ソング:labファイルを書き出す機能を追加 #2383

Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
59 changes: 59 additions & 0 deletions src/components/Sing/AudioExportOverlay.vue
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

メッセージの部分だけ差し替えられるようにすれば共通化できそう?
Container/PresentationのContainerが音声用とlab用とで2個ある形でも、SingEditorから差し込む形でも。

Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
<template>
<div v-if="nowAudioExporting" class="exporting-dialog">
<div>
<QSpinner color="primary" size="2.5rem" />
<div class="q-mt-xs">
{{ nowRendering ? "レンダリング中・・・" : "音声を書き出し中・・・" }}
</div>
<QBtn
v-if="nowRendering"
padding="xs md"
label="音声の書き出しをキャンセル"
class="q-mt-sm"
outline
@click="cancelExport"
/>
</div>
</div>
</template>

<script setup lang="ts">
import { computed } from "vue";
import { useStore } from "@/store";

const store = useStore();

const nowRendering = computed(() => {
return store.state.nowRendering;
});
const nowAudioExporting = computed(() => {
return store.state.nowAudioExporting;
});

const cancelExport = () => {
void store.actions.CANCEL_AUDIO_EXPORT();
};
</script>

<style scoped lang="scss">
@use "@/styles/v2/variables" as vars;
@use "@/styles/colors" as colors;

.exporting-dialog {
background-color: rgba(colors.$display-rgb, 0.15);
position: absolute;
inset: 0;
z-index: 10;
display: flex;
text-align: center;
align-items: center;
justify-content: center;

> div {
color: colors.$display;
background: colors.$surface;
border-radius: 6px;
padding: 14px;
}
}
</style>
63 changes: 63 additions & 0 deletions src/components/Sing/LabelExportOverlay.vue
Original file line number Diff line number Diff line change
@@ -0,0 +1,63 @@
<template>
<div v-if="nowLabelExporting" class="exporting-dialog">
<div>
<QSpinner color="primary" size="2.5rem" />
<div class="q-mt-xs">
{{
nowRendering
? "レンダリング中・・・"
: "labファイルを書き出し中・・・"
}}
</div>
<QBtn
v-if="nowRendering"
padding="xs md"
label="labファイルの書き出しをキャンセル"
class="q-mt-sm"
outline
@click="cancelExport"
/>
</div>
</div>
</template>

<script setup lang="ts">
import { computed } from "vue";
import { useStore } from "@/store";

const store = useStore();

const nowRendering = computed(() => {
return store.state.nowRendering;
});
const nowLabelExporting = computed(() => {
return store.state.nowLabelExporting;
});

const cancelExport = () => {
void store.actions.CANCEL_LABEL_EXPORT();
};
</script>

<style scoped lang="scss">
@use "@/styles/v2/variables" as vars;
@use "@/styles/colors" as colors;

.exporting-dialog {
background-color: rgba(colors.$display-rgb, 0.15);
position: absolute;
inset: 0;
z-index: 10;
display: flex;
text-align: center;
align-items: center;
justify-content: center;

> div {
color: colors.$display;
background: colors.$surface;
border-radius: 6px;
padding: 14px;
}
}
</style>
31 changes: 4 additions & 27 deletions src/components/Sing/SingEditor.vue
Original file line number Diff line number Diff line change
Expand Up @@ -2,22 +2,8 @@
<ToolBar />
<div class="sing-main" :class="{ 'sidebar-open': isSidebarOpen }">
<EngineStartupOverlay :isCompletedInitialStartup />
<div v-if="nowAudioExporting" class="exporting-dialog">
<div>
<QSpinner color="primary" size="2.5rem" />
<div class="q-mt-xs">
{{ nowRendering ? "レンダリング中・・・" : "音声を書き出し中・・・" }}
</div>
<QBtn
v-if="nowRendering"
padding="xs md"
label="音声の書き出しをキャンセル"
class="q-mt-sm"
outline
@click="cancelExport"
/>
</div>
</div>
<AudioExportOverlay />
<LabelExportOverlay />

<QSplitter
:modelValue="isSidebarOpen ? sidebarWidth : 0"
Expand Down Expand Up @@ -46,6 +32,8 @@ import ToolBar from "./ToolBar/ToolBar.vue";
import ScoreSequencer from "./ScoreSequencer.vue";
import SideBar from "./SideBar/SideBar.vue";
import EngineStartupOverlay from "@/components/EngineStartupOverlay.vue";
import AudioExportOverlay from "@/components/Sing/AudioExportOverlay.vue";
import LabelExportOverlay from "@/components/Sing/LabelExportOverlay.vue";
import { useStore } from "@/store";
import onetimeWatch from "@/helpers/onetimeWatch";
import {
Expand Down Expand Up @@ -80,17 +68,6 @@ watch(
},
);

const nowRendering = computed(() => {
return store.state.nowRendering;
});
const nowAudioExporting = computed(() => {
return store.state.nowAudioExporting;
});

const cancelExport = () => {
void store.actions.CANCEL_AUDIO_EXPORT();
};

const isCompletedInitialStartup = ref(false);
// TODO: Vueっぽくないので解体する
onetimeWatch(
Expand Down
25 changes: 24 additions & 1 deletion src/components/Sing/menuBarData.ts
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@ import { computed } from "vue";
import { useStore } from "@/store";
import { MenuItemData } from "@/components/Menu/type";
import { useRootMiscSetting } from "@/composables/useRootMiscSetting";
import { notifyResult } from "@/components/Dialog/Dialog";

export const useMenuBarData = () => {
const store = useStore();
Expand All @@ -24,16 +25,38 @@ export const useMenuBarData = () => {
});
};

const exportLabelFile = async () => {
const results = await store.actions.EXPORT_LABEL_FILES({});

if (results.length === 0) {
throw new Error("results.length is 0.");
}
notifyResult(
results[0], // TODO: SaveResultObject[] に対応する
"text",
store.actions,
store.state.confirmedTips.notifyOnGenerate,
);
};

// 「ファイル」メニュー
const fileSubMenuData = computed<MenuItemData[]>(() => [
{
type: "button",
label: "音声を出力",
label: "音声書き出し",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

トーク側と合わせました。

onClick: () => {
void exportAudioFile();
},
disableWhenUiLocked: true,
},
{
type: "button",
label: "labファイルを書き出し",
onClick: () => {
void exportLabelFile();
},
disableWhenUiLocked: true,
},
{ type: "separator" },
{
type: "button",
Expand Down
23 changes: 17 additions & 6 deletions src/sing/domain.ts
Original file line number Diff line number Diff line change
Expand Up @@ -513,7 +513,7 @@ export type PhonemeTimingEditData = Map<NoteId, PhonemeTimingEdit[]>;
/**
* 音素列を音素タイミング列に変換する。
*/
function phonemesToPhonemeTimings(phonemes: FramePhoneme[]) {
export function phonemesToPhonemeTimings(phonemes: FramePhoneme[]) {
const phonemeTimings: PhonemeTiming[] = [];
let cumulativeFrame = 0;
for (const phoneme of phonemes) {
Expand All @@ -531,7 +531,7 @@ function phonemesToPhonemeTimings(phonemes: FramePhoneme[]) {
/**
* 音素タイミング列を音素列に変換する。
*/
function phonemeTimingsToPhonemes(phonemeTimings: PhonemeTiming[]) {
export function phonemeTimingsToPhonemes(phonemeTimings: PhonemeTiming[]) {
return phonemeTimings.map(
(value): FramePhoneme => ({
phoneme: value.phoneme,
Expand All @@ -544,7 +544,7 @@ function phonemeTimingsToPhonemes(phonemeTimings: PhonemeTiming[]) {
/**
* フレーズごとの音素列を全体の音素タイミング列に変換する。
*/
function toEntirePhonemeTimings(
export function toEntirePhonemeTimings(
phrasePhonemeSequences: FramePhoneme[][],
phraseStartFrames: number[],
) {
Expand Down Expand Up @@ -725,7 +725,7 @@ function applyPhonemeTimingEditToPhonemeTimings(
/**
* 音素が重ならないように音素タイミングとフレーズの終了フレームを調整する。
*/
function adjustPhonemeTimingsAndPhraseEndFrames(
export function adjustPhonemeTimingsAndPhraseEndFrames(
phonemeTimings: PhonemeTiming[],
phraseStartFrames: number[],
phraseEndFrames: number[],
Expand Down Expand Up @@ -816,13 +816,24 @@ function adjustPhonemeTimingsAndPhraseEndFrames(
}
}

function calcPhraseStartFrames(phraseStartTimes: number[], frameRate: number) {
/**
* フレーズの開始フレームを算出する。
* 開始フレームは整数。
*/
export function calcPhraseStartFrames(
phraseStartTimes: number[],
frameRate: number,
) {
return phraseStartTimes.map((value) =>
secondToRoundedFrame(value, frameRate),
);
}

function calcPhraseEndFrames(
/**
* フレーズの終了フレームを算出する。
* 終了フレームは整数。
*/
export function calcPhraseEndFrames(
phraseStartFrames: number[],
phraseQueries: EditorFrameAudioQuery[],
) {
Expand Down
Original file line number Diff line number Diff line change
@@ -1,4 +1,8 @@
export const convertToWavFileData = (audioBuffer: AudioBuffer) => {
import Encoding from "encoding-japanese";
import { Encoding as EncodingType } from "@/type/preload";
import { FramePhoneme } from "@/openapi";

export function generateWavFileData(audioBuffer: AudioBuffer) {
const bytesPerSample = 4; // Float32
const formatCode = 3; // WAVE_FORMAT_IEEE_FLOAT

Expand Down Expand Up @@ -53,4 +57,53 @@ export const convertToWavFileData = (audioBuffer: AudioBuffer) => {
}

return new Uint8Array(buffer);
};
}

export async function generateTextFileData(obj: {
text: string;
encoding?: EncodingType;
}) {
obj.encoding ??= "UTF-8";

const textBlob = {
"UTF-8": (text: string) => {
const bom = new Uint8Array([0xef, 0xbb, 0xbf]);
return new Blob([bom, text], {
type: "text/plain;charset=UTF-8",
});
},
Shift_JIS: (text: string) => {
const sjisArray = Encoding.convert(Encoding.stringToCode(text), {
to: "SJIS",
type: "arraybuffer",
});
return new Blob([new Uint8Array(sjisArray)], {
type: "text/plain;charset=Shift_JIS",
});
},
}[obj.encoding](obj.text);

return await textBlob.arrayBuffer();
}

export async function generateLabelFileData(
phonemes: FramePhoneme[],
frameRate: number,
) {
let labString = "";
let timestamp = 0;

const writeLine = (phonemeLengthSeconds: number, phoneme: string) => {
labString += timestamp.toFixed() + " ";
timestamp += phonemeLengthSeconds * 10e7; // 100ns単位に変換
labString += timestamp.toFixed() + " ";
labString += phoneme + "\n";
};

for (const phoneme of phonemes) {
// REVIEW: vowel != "N" のときに vowel.toLowerCase() する必要がある…?
writeLine(phoneme.frameLength / frameRate, phoneme.phoneme);
}
Comment on lines +104 to +105
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

トークはvowel != "N" のときに vowel.toLowerCase() する実装になっていました。
ソングは必要なさそう…?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

うーーーーーーん!!! 知らなかった・・・!!!

ユーザー的にはどちらでも良さそう。
ただまあ仕様は揃えてあげた方が良さそうなのと、大文字の時は無声というのは若干特殊な気がしないでもないので、母音のaiueoにしちゃって良さそうに感じました!

もしトーク側が未実装だったら、大文字のままにした方が良いかもですね!
(でももしかしたら既存ソフトに合わせてる可能性もありそう。)


return await generateTextFileData({ text: labString });
}
Loading
Loading