Add TorchScript-based SoX I/O backend #726
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
mthrok
force-pushed
the
sox_io_backend
branch
7 times, most recently
from
54fbcc2 to
cb85a45
Compare
mthrok
changed the title
mthrok
force-pushed
the
sox_io_backend
branch
12 times, most recently
from
f85d969 to
167cc55
Compare
This was referenced Jun 18, 2020
mthrok
added a commit
that referenced
this pull request
Jun 18, 2020
This is a part of PRs to add new "sox_io" backend. #726 This PR adds `SignalInfo` structure, which is data exchange interface between Python and C++ in coming TorchScript-based sox IO backend. For the case, where C++ extension is not available (i.e. Windows), this PR also adds dummy class and module that will be substituted. This logic is implemented in `torchaudio.extension` moduel.
mthrok
force-pushed
the
sox_io_backend
branch
2 times, most recently
from
4bf7016 to
bfee816
Compare
mthrok
force-pushed
the
sox_io_backend
branch
9 times, most recently
from
d33a6ff to
3ffc88e
Compare
mthrok
added a commit
that referenced
this pull request
Jun 25, 2020
This is a part of PRs to add new "sox_io" backend. #726 and depends on #718 and #728 . This PR adds `load` function to "sox_io" backend, which is tested on the following audio formats; - `wav` - `mp3` - `flac` - `ogg/vorbis` * By default, "sox_io" backend returns Tensor with `float32` dtype and the shape of `[channel, time]`. The samples are normalized to fit in the range of `[-1.0, 1.0]`. Unlike existing "sox" backend, the new `load` function can handle WAV file natively, when the input format is WAV with integer type, (such as 32-bit signed integer, 16-bit signed integer and 8-bit unsigned integer) by providing `normalize=False`, this function can return integer Tensor, where the samples are expressed within the whole range of the corresponding dtype, that is, `int32` tensor for `32-bit PCM`, `int16` for `16-bit PCM` and `uint8` for `8-bit PCM`. This behavior follows [scipy.io.wavfile.read](https://docs.scipy.org/doc/scipy/reference/generated/scipy.io.wavfile.read.html). `normalize` parameter has no effect for other formats and the load function always return normalized value with `float32` Tensor. __* Note__ The current binary distribution of torchaudio does not contain `ogg/vorbis` and `opus` codecs. To handle these files, one needs to build torchaudio from the source with proper codecs in the system. __Note 2__ Since this PR, `scipy` becomes required module for running test.
Codecov Report
@@ Coverage Diff @@
## master #726 +/- ##
==========================================
+ Coverage 89.14% 89.16% +0.02%
==========================================
Files 32 32
Lines 2561 2566 +5
==========================================
+ Hits 2283 2288 +5
Misses 278 278
Continue to review full report at Codecov.
|
mthrok
force-pushed
the
sox_io_backend
branch
3 times, most recently
from
8ae9ebd to
a46c34f
Compare
vincentqb
reviewed
Jul 1, 2020
Comment on lines
+33
to
+34
| if backend == 'sox_io': | ||
| continue |
There was a problem hiding this comment.
Why is sox_io made a special case and skipped here?
There was a problem hiding this comment.
The test cases in this class depends on the global state of the previous test ran, which brakes the principle of unit test and having sox_io breaks it.
|
thanks! |
mthrok
pushed a commit
to mthrok/audio
that referenced
this pull request
Dec 13, 2022
…ated against (pytorch#726) Co-authored-by: holly1238 <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR (and dependent PRs) adds a new backend "sox_io" backend.
(So that users can opt-in, but we do not intend to change the default yet.)
The new "sox_io" backend has the following advantages;
The data process pipeline written using the new backend can be dumped and used from C++.
The original "sox" backend had a number of issues,
loadfunction cannot handle WAV file correctly.I have added bunch of tests to make sure that the new backend does not have the same issue.
This includes read/write operaions of
wav,flac,mp3andogg/vorbisformats. *This backend can also read
opus,though it's not in unit test.Add opus support to binary distribution #755dtypeis picked depending on the internal representation of WAV format. This behavior is same as how SciPy handles WAV file.normalizeoption, which correctly maps integer value range to[-1.0, 1.0]withfloat32.sox_signalinfo_tandsox_encodinginfo_tstructs directly, but TorchScript does not allow this. Also setting the correct parameters for these structs is not easy. In the new backend, options related to sox-internal are handled in C++, and users only need to providecompressionoption that corresponds tosox's-Coption.Note The current binary distribution of torchaudio does not contain#750ogg/vorbiscodecs. To handle these files, you need to build torchaudio from the source. Refer to README for the instruction.