-
Notifications
You must be signed in to change notification settings - Fork 25
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Add opus custom support #18
Open
emlynmac
wants to merge
41
commits into
alta:main
Choose a base branch
from
emlynmac:jamulus-coders
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
+326
−57
Open
Changes from 32 commits
Commits
Show all changes
41 commits
Select commit
Hold shift + click to select a range
b96e0ad
Enable Opus CUSTOM_MODES
emlynmac 144063f
Add comma
emlynmac fb5c31c
Add CELT headers / source
emlynmac 04fa515
Include base header / source makefiles
emlynmac db32719
Wrap variadic ctl functions for swift to import
emlynmac d1be9a6
Wrap up some variadic functions
emlynmac 80f7e0f
rename to copuswrapper
emlynmac dc5f841
Move external header to export directory
emlynmac 4e452fd
Add custom mode extensions to support Jamulus
emlynmac 6104386
Add jamulus custom configuration
emlynmac 7273180
Enable custom encoder / decoders to create higher level wrapper class…
emlynmac 40be05d
Enable access outside the module
emlynmac 593766d
Add custom decode / encode functions and contain frameSize
emlynmac 605b9ec
Fix up encoder bug
emlynmac fefebd7
Set frame size in encode
emlynmac 3204c1a
Generalize the ioctl call
emlynmac b0c9e3b
make the framesize public
emlynmac c898e20
Enable sample count to be passed in as a multiplier of frameSize
emlynmac 61b6a4b
Rename method parameter to reflect correct usage
emlynmac 9fa606f
Ensure error code is passed back
emlynmac 21164e1
Add explicit return
emlynmac bada8cf
Allocate a buffer properly
emlynmac ed39aac
Handle the case of null data, aka packet loss
emlynmac 2c72a72
Add documentation; add some safety
emlynmac 3ff24bf
Merge branch 'alta:main' into main
emlynmac 566f056
Merge branch 'main' into jamulus-coders
emlynmac b41d414
Tidy some changes prior to PR back to fork source
emlynmac 40fb047
More whitespace formatting
emlynmac e3788e3
Update submodule to correct (master branch version)
emlynmac 8b656b3
Apply swiftformat rules
emlynmac e7cb649
Revert swiftformat changes that cause changes to master
emlynmac a7f8272
One more swiftformat change revert
emlynmac a5b29aa
Change return type on ctl wrapper method
emlynmac 712b0ff
Merge opus custom with existing en/de-coders
emlynmac 33c68f4
Apply swiftformat
emlynmac 033520f
Rename to customFrameSize
emlynmac aa3db6f
One more rename
emlynmac 42d044f
Ensure that the compressed size is actually passed down
emlynmac f5f8735
Always build optimized builds of Opus
emlynmac 00d42cf
Merge branch 'main_upstream'
emlynmac a48104d
Merge branch 'main' into jamulus-coders
emlynmac File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,2 +1,4 @@ | ||
.build | ||
.DS_Store | ||
DerivedData | ||
.swiftpm |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#ifndef __OPUS_VARIADIC_WRAPPER_H__ | ||
#define __OPUS_VARIADIC_WRAPPER_H__ | ||
|
||
//#include "" | ||
#include <opus_defines.h> | ||
#include <opus_custom.h> | ||
|
||
int opus_custom_encoder_ctl_wrapper(OpusCustomEncoder *OPUS_RESTRICT st, int request, opus_int32 val); | ||
int opus_custom_decoder_ctl_wrapper(OpusCustomDecoder *OPUS_RESTRICT st, int request, opus_int32 val); | ||
|
||
#endif |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,11 @@ | ||
#include "variadic-wrapper.h" | ||
|
||
int opus_custom_encoder_ctl_wrapper(OpusCustomEncoder *OPUS_RESTRICT st, int request, opus_int32 val) | ||
{ | ||
return opus_custom_encoder_ctl(st, request, val); | ||
} | ||
|
||
int opus_custom_decoder_ctl_wrapper(OpusCustomDecoder *OPUS_RESTRICT st, int request, opus_int32 val) | ||
{ | ||
return opus_custom_decoder_ctl(st, request, val); | ||
} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,266 @@ | ||
import AVFoundation | ||
import Copus | ||
import Copuswrapper | ||
import Foundation | ||
|
||
public extension Opus { | ||
/// | ||
/// Implements a custom opus encoder / decoder. | ||
/// Custom implementations can have non-standard frame sizes | ||
/// | ||
final class Custom { | ||
private let opusCustomMode: OpaquePointer | ||
let encoder: OpaquePointer | ||
let decoder: OpaquePointer | ||
private let format: AVAudioFormat | ||
public let frameSize: Int32 | ||
|
||
public init(format: AVAudioFormat, | ||
application _: Application = .audio, | ||
frameSize: UInt32 = 128) throws | ||
{ | ||
if !format.isValidOpusPCMFormat { | ||
throw Opus.Error.badArgument | ||
} | ||
self.format = format | ||
self.frameSize = Int32(frameSize) | ||
|
||
var error: Opus.Error = .ok | ||
|
||
// Create custom parameters | ||
guard let customMode = opus_custom_mode_create( | ||
Int32(format.sampleRate), | ||
Int32(frameSize), | ||
&error.rawValue | ||
) else { throw error } | ||
opusCustomMode = customMode | ||
|
||
// Create custom encoder | ||
guard let opusEncoder = opus_custom_encoder_create( | ||
customMode, | ||
Int32(format.channelCount), | ||
&error.rawValue | ||
) else { throw error } | ||
|
||
encoder = opusEncoder | ||
|
||
// Create custom decoder | ||
guard let opusDecoder = opus_custom_decoder_create( | ||
customMode, | ||
Int32(format.channelCount), | ||
&error.rawValue | ||
) else { throw error } | ||
decoder = opusDecoder | ||
} | ||
|
||
deinit { | ||
opus_encoder_destroy(encoder) | ||
opus_decoder_destroy(decoder) | ||
opus_custom_mode_destroy(opusCustomMode) | ||
} | ||
|
||
/// | ||
/// Wrapper onto the opus_custom_encoder_ctl function | ||
/// - Parameter request The Opus CTL to change | ||
/// - Parameter value The value to set it to | ||
/// | ||
/// - Returns Opus.Error code raw type | ||
public func encoderCtl(request: Int32, value: Int32) -> Opus.Error.RawValue { | ||
emlynmac marked this conversation as resolved.
Show resolved
Hide resolved
|
||
opus_custom_encoder_ctl_wrapper(encoder, request, value) | ||
} | ||
|
||
/// | ||
/// Encode a PCM buffer to data using the custom mode configuration and max size | ||
/// - parameter avData Audio data to compress | ||
/// - parameter compressedSize Opus packet size to compress to | ||
/// - Returns Data containing the Opus packet | ||
public func encode(_ avData: AVAudioPCMBuffer, | ||
compressedSize: Int) throws -> Data | ||
{ | ||
var compressed = Data(repeating: 0, count: compressedSize) | ||
compressed.count = try compressed.withUnsafeMutableBytes( | ||
{ try encode(avData, to: $0, compressedSize: compressedSize) } | ||
) | ||
return compressed | ||
} | ||
|
||
private func encode(_ input: AVAudioPCMBuffer, | ||
to output: inout [UInt8], | ||
compressedSize: Int) throws -> Int | ||
{ | ||
try output.withUnsafeMutableBufferPointer { | ||
try encode(input, to: $0, compressedSize: compressedSize) | ||
} | ||
} | ||
|
||
private func encode(_ input: AVAudioPCMBuffer, | ||
to output: UnsafeMutableRawBufferPointer, | ||
compressedSize: Int) throws -> Int | ||
{ | ||
let output = UnsafeMutableBufferPointer( | ||
start: output.baseAddress!.bindMemory( | ||
to: UInt8.self, capacity: output.count | ||
), | ||
count: output.count | ||
) | ||
return try encode(input, to: output, compressedSize: compressedSize) | ||
} | ||
|
||
private func encode(_ input: AVAudioPCMBuffer, | ||
to output: UnsafeMutableBufferPointer<UInt8>, | ||
compressedSize: Int) throws -> Int | ||
{ | ||
guard input.format.sampleRate == format.sampleRate, | ||
input.format.channelCount == format.channelCount | ||
else { | ||
throw Opus.Error.badArgument | ||
} | ||
|
||
switch format.commonFormat { | ||
case .pcmFormatInt16: | ||
let input = UnsafeBufferPointer( | ||
start: input.int16ChannelData![0], | ||
count: Int(input.frameLength * format.channelCount) | ||
) | ||
return try encode(input, to: output, compressedSize: compressedSize) | ||
|
||
case .pcmFormatFloat32: | ||
let input = UnsafeBufferPointer( | ||
start: input.floatChannelData![0], | ||
count: Int(input.frameLength * format.channelCount) | ||
) | ||
return try encode(input, to: output, compressedSize: compressedSize) | ||
|
||
default: | ||
throw Opus.Error.badArgument | ||
} | ||
} | ||
|
||
private func encode(_ input: UnsafeBufferPointer<Int16>, | ||
to output: UnsafeMutableBufferPointer<UInt8>, | ||
compressedSize: Int) throws -> Int | ||
{ | ||
let encodedSize = opus_custom_encode( | ||
encoder, | ||
input.baseAddress!, | ||
frameSize, | ||
output.baseAddress!, | ||
Int32(compressedSize) | ||
) | ||
|
||
if encodedSize < 0 { | ||
throw Opus.Error(encodedSize) | ||
} | ||
return Int(encodedSize) | ||
} | ||
|
||
private func encode(_ input: UnsafeBufferPointer<Float32>, | ||
to output: UnsafeMutableBufferPointer<UInt8>, | ||
compressedSize: Int) throws -> Int | ||
{ | ||
let encodedSize = opus_custom_encode_float( | ||
encoder, | ||
input.baseAddress!, | ||
frameSize, | ||
output.baseAddress!, | ||
Int32(compressedSize) | ||
) | ||
if encodedSize < 0 { | ||
throw Opus.Error(encodedSize) | ||
} | ||
return Int(encodedSize) | ||
} | ||
|
||
/// | ||
/// Decode an opus packet | ||
/// If the data is empty, it is treated as a dropped packet | ||
/// - Parameter data Compressed data | ||
/// - Parameter compressedPacketSize Number of bytes of data | ||
/// - Parameter sampleMultiplier Frame size multiplier if greater than one | ||
/// - Returns Uncompressed audio buffer | ||
public func decode(_ data: Data, | ||
compressedPacketSize: Int32, | ||
sampleMultiplier: Int32 = 1) throws -> AVAudioPCMBuffer | ||
{ | ||
guard data.isEmpty || data.count == compressedPacketSize else { | ||
throw Opus.Error.bufferTooSmall | ||
} | ||
|
||
return try data.withUnsafeBytes { | ||
let input = $0.bindMemory(to: UInt8.self) | ||
|
||
let output = AVAudioPCMBuffer( | ||
pcmFormat: format, | ||
frameCapacity: AVAudioFrameCount(frameSize * sampleMultiplier) | ||
)! | ||
try decode(input, to: output, packetSize: compressedPacketSize) | ||
|
||
return output | ||
} | ||
} | ||
|
||
private func decode(_ input: UnsafeBufferPointer<UInt8>, | ||
to output: AVAudioPCMBuffer, | ||
packetSize: Int32) throws | ||
{ | ||
let decodedCount: Int | ||
|
||
switch output.format.commonFormat { | ||
case .pcmFormatInt16: | ||
let output = UnsafeMutableBufferPointer( | ||
start: output.int16ChannelData![0], | ||
count: Int(output.frameCapacity) | ||
) | ||
decodedCount = try decode(input, to: output, packetSize: packetSize) | ||
|
||
case .pcmFormatFloat32: | ||
let output = UnsafeMutableBufferPointer( | ||
start: output.floatChannelData![0], | ||
count: Int(output.frameCapacity) | ||
) | ||
decodedCount = try decode(input, to: output, packetSize: packetSize) | ||
default: | ||
throw Opus.Error.badArgument | ||
} | ||
|
||
if decodedCount < 0 { | ||
throw Opus.Error(decodedCount) | ||
} | ||
output.frameLength = AVAudioFrameCount(decodedCount) | ||
} | ||
|
||
private func decode(_ input: UnsafeBufferPointer<UInt8>, | ||
to output: UnsafeMutableBufferPointer<Int16>, | ||
packetSize: Int32) throws -> Int | ||
{ | ||
let decodedCount = opus_custom_decode( | ||
decoder, | ||
input.isEmpty ? nil : input.baseAddress, | ||
packetSize, | ||
output.baseAddress!, | ||
Int32(output.count) | ||
) | ||
if decodedCount < 0 { | ||
throw Opus.Error(decodedCount) | ||
} | ||
return Int(decodedCount) | ||
} | ||
|
||
private func decode(_ input: UnsafeBufferPointer<UInt8>, | ||
to output: UnsafeMutableBufferPointer<Float32>, | ||
packetSize: Int32) throws -> Int | ||
{ | ||
let decodedCount = opus_custom_decode_float( | ||
decoder, | ||
input.isEmpty ? nil : input.baseAddress, | ||
packetSize, | ||
output.baseAddress!, | ||
Int32(output.count) | ||
) | ||
if decodedCount < 0 { | ||
throw Opus.Error(decodedCount) | ||
} | ||
return Int(decodedCount) | ||
} | ||
} | ||
} |
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
What’s the purpose of this file? Can it be deleted?
(I think it should be, if
Opus.Encoder
andOpus.Decoder
can support nonstandard frame sizes.