Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Wait for #2796][Graph] add inplace direction setting through layer property #2797

Open
wants to merge 4 commits into
base: main
Choose a base branch
from

Commits on Nov 14, 2024

  1. [Graph] change of inplace-type setting method

    The method of setting the inplace-type has been redefined.
    
    The reason why inplace processing becomes complicated is that since a multi-out layer shares output variables, so it needs to be considered whether or not inplace can be performed.
    
    To simplify the problem, the layers that can perform inplace even after the multi-out layer are only no-operation layers(no-op layers). These no-op layers include identity, reshape, and flatten layers.
    
    For other layers, even if they support inplace, they cannot perform inplace when there is a multi-out layer in front of them.
    
    Note that because no-op layers connected with multi-out layer share memory with the multi-out layer, so they have the same properties as the multi-out layer. This is expressed as RESTRICTING in our script.
    
    Based on these definitions, I've redesigned the method of setting inplace type.
    
    1. By default, initialize the inplace type for each layer. If supportInPlace is true, it will be initialized as NON_RESTRICTING; otherwise, it will be initialized as NONE.
    2. However, not all layers are initialized like this. For multi-out layers or no-op layers, if supportInPlace is true, they will be initialized as RESTRICTING types(However, the no-op layer will be changed to a non-restricting type if that is not connected with the multi-out layer).
    3. After initialization, confirm the input connections from the network_graph.cpp to determine the final inplace type. It's clearer to see the source code for this part.
    
    **Self evaluation:**
    1. Build test:   [X]Passed [ ]Failed [ ]Skipped
    2. Run test:     [X]Passed [ ]Failed [ ]Skipped
    
    Signed-off-by: Seungbaek Hong <[email protected]>
    baek2sm committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    4aade30 View commit details
    Browse the repository at this point in the history
  2. [Graph] rename variable and remove non-working codes

    - The variables with the same meaning are written as "in_place" or "is_inplace" by the script, so I unified it to use the term "is_inplace".
    
    - Some layer's finalize function includes code that determines whether or not to support in-place depending on the tensor type. However, this code does not work. The reason it seems like this code is working is because there is a similar purpose of code at the top of the `canExecuteInPlace` function within the `network_graph.cpp` and that code works.
    
    It is meaningless to determine whether or not to support in-place within the `finalize` function because the `canExecuteInPlace` function, which decides how InPlace will behave, is called before the `finalize` function.
    
     The canExecuteInPlace function is called during `compile` while the finalize function is called during `initialize` after `compile`. Therefore, setting supportInplace inside the finalize function does not work.
    
    **Self evaluation:**
    1. Build test:   [X]Passed [ ]Failed [ ]Skipped
    2. Run test:     [X]Passed [ ]Failed [ ]Skipped
    
    Signed-off-by: Seungbaek Hong <[email protected]>
    baek2sm committed Nov 14, 2024
    Configuration menu
    Copy the full SHA
    9d43feb View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. [Graph] add inplace setting through layer property

    - now you can set in-place flag through layer property of tensor
    operation layers.
    - rename "initializeInPlaceType" func to "initializeInPlace". now
    "is_inplace" property is set in that function, too.
    - in some layers, support_backwarding flag may be changed by the
    in-place setting.
    
    **Self evaluation:**
    1. Build test:   [X]Passed [ ]Failed [ ]Skipped
    2. Run test:     [X]Passed [ ]Failed [ ]Skipped
    
    Signed-off-by: Seungbaek Hong <[email protected]>
    baek2sm committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    ac949ad View commit details
    Browse the repository at this point in the history
  2. [Graph] add inplace direction setting through layer property

    add inplace direction setting for binary tensor operation layer.
    
    **Self evaluation:**
    1. Build test:   [X]Passed [ ]Failed [ ]Skipped
    2. Run test:     [X]Passed [ ]Failed [ ]Skipped
    
    Signed-off-by: Seungbaek Hong <[email protected]>
    baek2sm committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    3802e65 View commit details
    Browse the repository at this point in the history