Make FP16 code more reusable. @open sesame 03/07 10:42 #2417
Conversation
This allows writing FP16-independent code without breaking the current code. This is a non-invasive subset of nnstreamer#2403 to address the concern of nnstreamer#2403 (comment) I believe nntrainer can use this during the refactoring so that we do not introduce additional code divergences. Signed-off-by: MyungJoo Ham <[email protected]>
myungjoo
requested review from
jijoongmoon,
again4you,
jaeyun-jung,
leemgs,
wooksong,
helloahn,
kparichay,
gichan-jang,
anyj0527,
zhoonit and
a team
as code owners
|
📝 TAOS-CI Version: 1.5.20200925. Thank you for submitting PR #2417. Please a submit 1commit/1PR (one commit per one PR) policy to get comments quickly from reviewers. Your PR must pass all verificiation processes of cibot before starting a review process from reviewers. If you are new member to join this project, please read manuals in documentation folder and wiki page. In order to monitor a progress status of your PR in more detail, visit http://ci.nnstreamer.ai/. |
| @@ -28,8 +30,56 @@ | |||
| #else | |||
| #define _FP16 _Float16 | |||
| #endif | |||
| #else /* !ENABLE_FP16 */ | |||
| /* Keep the FP16 programming interface, but don't allow using it in run-time */ | |||
| #define _FP16 uint16_t | |||
There was a problem hiding this comment.
This is great. It definitely can be used for refactoring the multi type handle in NNTrainer.
| @@ -54,6 +54,9 @@ class ActiFunc { | |||
| template <typename T = float> void setActiFunc(ActivationType acti_type) { | |||
| activation_type = acti_type; | |||
|
|
|||
| if (typeid(T) == typeid(_FP16)) | |||
| THROW_UNLESS_FP16_ENABLED; | |||
There was a problem hiding this comment.
this needs to be checked again. I don't think we need to throw an error here. These activation function type surely need to support FP16 datatype. To check, we have to enable the 'enable-fp16' option in meson_options.txt should be true.
There was a problem hiding this comment.
This ensures thorwing an exception when FP16 activation is called without FP16 support.
By calling (conditional) THROW at the point where FP16 op actually exists, we can skip writing FP16 conditions at the callers (where you update codes mostly). This further reduces dependencies on FP16 in the code, overall. (you don't need to write if-fp16-enable codes elsewhere)
Note that this is currently implemented as preprocessor macro, coming from meson's options.
There was a problem hiding this comment.
This is an exception for a situation where attempting to use FP16 even though FP16 is disabled in the Meson option, so it seems proper to handle the exception here.
DonghakPark
changed the title
jijoongmoon
requested review from
lhs8928,
songgot,
DonghakPark,
skykongkong8,
djeong20 and
EunjuYang
as code owners
This allows writing FP16-independent code without
breaking the current code.
This is a non-invasive subset of #2403 to
address the concern of #2403 (comment)
I believe nntrainer can use this during the refactoring so that we do not introduce additional code divergences.