Arch-dep refactoring plan after the next release

[myungjoo]

After the next release, we can try arch-dep refactoring.

Objectives:

• Remove #ifdef/#if conditions depending on architectures (e.g., NEON, ARMv9, AVX, RVV, ...) from general C++ source codes (*.cpp)

As-is:

To-be:

Files affected:

• (list to be populated)

How they will be reorganized:

• (plan to be written)

Timeline:

• after the next release.

Note:

• Don't be alarmed. This is to make a proposal of the refactored architecture. I'm not trying to refactor the code right away.

[myungjoo]

TACTICS proposal for NEON (ARCH-DEPs)

• nntrainer/utils/util_simd*: base virtual class "compute", derived class "compute_neon". / singleton.
  Make this as the ROOT of arch-dep decision makers.

  • Proposal code is ready: myungjoo@d146d6d

• nntrainer/tensor/blas_neon & blas_interface: make sure that blas_neon cannot be accessed out of blas_interface.
  classify blas_interface (singleton), rewrite blas_neon as a derived class of blas_interface, determined by "compute" class.

• nntrainer/tensor/neon_mathfun.h: just make sure that this cannot be out of blas_neon

• nntrainer/layers/multi_head_attention_layer: let it use "compute" class directly. Move fallback code to the compute base class.

• Applications/*: never mind.

Surprisingly, this is all you need.

TACTICS proposal for FP16 (strongly related to ARCH-DEPs)

• Define the behaviors of FP16 models in non-FP16 built binaries or non-FP16 systems.

• Applications/, test/: never mind.

• Type A: Use a macro w/ throw for fp16 required code

• Type B: Remove ENABLE_FP16 condition. Delegate FP16 handling/exception to other module by calling functions defined by other modules.

• nntrainer/models/neuralnet.cpp: Type A

• api/ccapi/include/layer.h: Type B

• api/ccapi/include/tensor_dim.h: Keep it here. Or create another header for "tensor data type"

• nntrainer/tensor/tensor_v2.cpp: Type B for if fp16; then call X; else throw; fi code

• nntrainer/tensor/blas_interface.cpp:

• nntrainer/tensor/blas_neon.cpp:

[myungjoo]

Recommendation before the refactoring & release:

For the new functions, please try to give consistent arguments and semantics across different arch-deps (NEON vs non-NEON).
I.e., when you make "function_for_neon", write "function_for_fallback" which has the same set of arguments.

Examples:
NOT GOOD

...
   if NEON
      function_neon_C(abc);
   else
      op a;
      op b;
      function_C2(c);
   ...

RECOMMENDED

   if NEON
      function_neon_C(abc);
   else
      function_fallback_C(abc);

[skykongkong8]

Thank you for clear direction. Great guide!