Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Implement Tokens in Swift and Kotlin #227

Merged
merged 16 commits into from
Aug 5, 2023
Merged

Implement Tokens in Swift and Kotlin #227

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
16 commits
Select commit Hold shift + click to select a range
240c04e
Implemented tokens and timestamps in Python API
w11wo Jul 10, 2023
313ffbd
Fixed Double Semicolon
w11wo Jul 10, 2023
171c9f2
Implemented tokens in Swift API
w11wo Jul 10, 2023
a0e0793
Implemented tokens in Jni
ductranminh Jul 10, 2023
3b39860
Add json & correct token in OnlineRecognizerResult
ductranminh Jul 13, 2023
92b0432
Merge branch 'master' into swift-tokens
ductranminh Jul 17, 2023
d1dfa87
Correct get tokens on android
ductranminh Jul 19, 2023
25b708c
Merge branch 'master' into swift-tokens
ductranminh Jul 19, 2023
24aa98d
Merge branch 'k2-fsa:master' into swift-tokens
w11wo Jul 28, 2023
3d7c6f5
Fixed Styling
w11wo Jul 28, 2023
28d43bc
Merge branch 'k2-fsa:master' into swift-tokens
w11wo Aug 4, 2023
2b9c69e
Add tokensArr into result
ductranminh Aug 4, 2023
48038a7
remove space
ductranminh Aug 5, 2023
0a8d370
Correct coding convention
ductranminh Aug 5, 2023
ccc3ba4
remove unused casting
ductranminh Aug 5, 2023
c313653
clean up code
ductranminh Aug 5, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions sherpa-onnx/c-api/c-api.cc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -154,18 +154,23 @@ SherpaOnnxOnlineRecognizerResult *GetOnlineStreamResult(
memset(reinterpret_cast<void *>(const_cast<char *>(r->tokens)), 0,
totalLength);
r->timestamps = new float[r->count];
char **tokensTemp = new char*[r->count];
int pos = 0;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
int pos = 0;
int32_t pos = 0;

for (int32_t i = 0; i < r->count; ++i) {
tokensTemp[i] = const_cast<char*>(r->tokens) + pos;
memcpy(reinterpret_cast<void *>(const_cast<char *>(r->tokens + pos)),
result.tokens[i].c_str(), result.tokens[i].size());
// +1 to move past the null character
pos += result.tokens[i].size() + 1;
r->timestamps[i] = result.timestamps[i];
}

r->tokensArr = const_cast<const char *const *>(tokensTemp);
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
r->tokensArr = const_cast<const char *const *>(tokensTemp);
r->tokensArr = tokensTemp;

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm done

} else {
r->count = 0;
r->timestamps = nullptr;
r->tokens = nullptr;
r->tokensArr = nullptr;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please also add code to free it to avoid memory leak.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@csukuangfj sorry, what do you want to free? This line is just to initialize null for pointer incase there is no token from SherpaOnnxOnlineRecognizerResult

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please change

sherpa-onnx/sherpa-onnx/c-api/c-api.cc

Lines 179 to 182 in 48038a7

void DestroyOnlineRecognizerResult(const SherpaOnnxOnlineRecognizerResult *r) {
delete[] r->text;
delete r;
}

to

void DestroyOnlineRecognizerResult(const SherpaOnnxOnlineRecognizerResult *r) {
  delete[] r->text;
  delete[] r->json;
  delete[] r->tokensArr;
  delete r;
}

By the way, please change the variable names:

  • totalLength -> total_length
  • tokensTemp -> tokens_temp
  • tokensArr -> tokens_arr
    to follow the code convention in sherpa-onnx.

}

return r;
Expand Down
3 changes: 3 additions & 0 deletions sherpa-onnx/c-api/c-api.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -108,6 +108,9 @@ SHERPA_ONNX_API typedef struct SherpaOnnxOnlineRecognizerResult {
// which are seperated by \0
const char *tokens;
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I suggest that we define

const char *_tokens;
const char *const*tokens;

where _tokens is your current tokens

And the new char **tokens is a pointer array.
tokens[0] contains the address of the first token in
_tokens. tokens[1] contains the address of the second
token in _tokens.

In this way, it simplifies users' life as they only need to iterate
char **tokens.

for (int32 i = 0; i != count; ++i) {
  const char*t = tokens[i];
  // process this token
}

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@csukuangfj I added const char *const *tokensArr; for easy to implement, and I also tested


// a pointer array contains the address of the first item in tokens
const char *const *tokensArr;

// Pointer to continuous memory which holds timestamps which
// are seperated by \0
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please update the comment. It is not separated by \0 for timestamps.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

fixed

float *timestamps;
Expand Down
14 changes: 7 additions & 7 deletions swift-api-examples/SherpaOnnx.swift
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -104,17 +104,17 @@ class SherpaOnnxOnlineRecongitionResult {
}

var tokens: [String] {
if let tokensPointer = result.pointee.tokens {
if let tokensPointer = result.pointee.tokensArr {
var tokens: [String] = []
var currentIndex = 0
for _ in 0..<count {
let token = String(cString: tokensPointer.advanced(by: currentIndex))
tokens.append(token)
currentIndex += token.utf8.count + 1
for index in 0..<count {
if let tokenPointer = tokensPointer[Int(index)] {
let token = String(cString: tokenPointer)
tokens.append(token)
}
}
return tokens
} else {
var tokens: [String] = []
let tokens: [String] = []
return tokens
}
}
Expand Down
Loading