Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Wait for #2563] [ Mixed ] Create weight with var32 tensor #2566

Closed
wants to merge 2 commits into from
Closed

[Wait for #2563] [ Mixed ] Create weight with var32 tensor #2566

wants to merge 2 commits into from

Commits on May 7, 2024

  1. [ Weight ] Add Var32 Tensor in Weight. 

    We will add Var32 Tensor if the Variable Weight is not Full
precision (FP32). This eables the Weight Update with full precision
and only Apply Gradient Process ueses this Tensor. Therefore, the
lifespan of this tensor should be "ApplyGradient".

. Modify TensorPool to generate Weigth considering Mixed Precsion.

**Self evaluation:**
1. Build test:	 [X]Passed [ ]Failed [ ]Skipped
2. Run test:	 [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <[email protected]>
    @jijoongmoon
    jijoongmoon committed May 7, 2024
    Configuration menu
    Copy the full SHA
    8505065 View commit details
    Browse the repository at this point in the history

  2. [ Mixed ] Create weight with var32 tensor 

    This pr create the variable fp32 tensor when we create the Weight and
Optimizer Weight.

. update the manager to create Weight with  var32 tensor which
requested to weight pool.
. update the weight requests with Weight Spec and var, grad and var32
tensors which created already.
. add clone Tensor with specific type in tensor.h

Resolves:

**Self evaluation:**
1. Build test:	 [X]Passed [ ]Failed [ ]Skipped
2. Run test:	 [X]Passed [ ]Failed [ ]Skipped

Signed-off-by: jijoong.moon <[email protected]>
    @jijoongmoon
    jijoongmoon committed May 7, 2024
    Configuration menu
    Copy the full SHA
    e79d4df View commit details
    Browse the repository at this point in the history