[neon] Modify neon sgemv_fp16 #46
Closed
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
- Previously, sgemv_fp16 was dependent of two conditions: 1. should have 8-divisible column or row 2. fully work with fp16 digit (which might raise accuracy issue) - In this commit, we expect sgemv to work like: 1. support every column length (with adaptive-compute optimization) 2. use temporal fp32 array to secure cumulative digit error in large scale Tensor 3. accelerate fp32 to fp16 copy and vice versa with neon to enhance time performance - some trivial typo fix included **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
skykongkong8
force-pushed
the
soft_pr
branch
from
0fc814a
to
cf32826
Compare
- Instead of explicitly declaring float16x4_t and converting into float32x4_t, it is better to implement it in inline code considering the number of registers on device, and memory consumption. Resolves: **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
s-debadri
reviewed
Sep 27, 2023
nntrainer/tensor/blas_neon.cpp
Outdated
| float32x4_t b0_7_2_low = vcvt_f32_f16(vget_low_f16(b0_7_2)); | ||
| float32x4_t b0_7_2_high = vcvt_f32_f16(vget_high_f16(b0_7_2)); | ||
| float32x4_t b0_7_0_low = vcvt_f32_f16(vld1_f16(&B[k * N + n])); | ||
| float32x4_t b0_7_0_high = vcvt_f32_f16(vld1_f16(&B[k * N + n])); |
There was a problem hiding this comment.
This should be float32x4_t b0_7_0_high = vcvt_f32_f16(vld1_f16(&B[k * N + n + 4]));
Otherwise it will load lower bits.
- Instead of explicitly declaring float16x4_t and converting into float32x4_t, it is better to implement it in inline code considering the number of registers on device, and memory consumption. **Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
skykongkong8
force-pushed
the
soft_pr
branch
from
ca7ca69
to
4f55742
Compare
jijoongmoon
approved these changes
Oct 4, 2023
- Previously, we used full-fp16 variables for sgemv and sgemm loop code. - However, such practice might cause acummulation error that exceeds our expected epsilon. - Now, it uses inter-fp32 value to preseve accuracy and avoid precision loss. Self evaluation:** 1. Build test: [X]Passed [ ]Failed [ ]Skipped 2. Run test: [X]Passed [ ]Failed [ ]Skipped Signed-off-by: skykongkong8 <[email protected]>
|
Already merged to upstream. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Self evaluation:
Signed-off-by:ss.kong[email protected]