[Snippets] Dynamic SplitDimensionM via runtime configurator

[v-Golubev]

Details:

• Introduced ParallelWAOptimizer class, which is used in RuntimeConfigurator to split M dimension of LIR with brgemms in order to optimize parallel work amount

Tickets:

• 147834

[a-sidorova]

First part

[a-sidorova]

Status: need to validate performance, will do soon

[IvanNovoselov]

This is a rather complicated interface for a transformation, it should be simplified in the scope of 148805.

[IvanNovoselov]

Why use deque when all push/pop are performed on front? Why not to use vector instead?
AFAIK deque is needed when we need mixed front/end access.

[v-Golubev]

From my perspective, in this use case (all push/pops are performed on front) std::deque is still better than vector: it has the corresponding API, and the complexity of Insertion or removal of elements at the beginning is O(1)

[IvanNovoselov]

Right, no objection from the performance perspective.
My question is rather why do we need to use smth exotic like deque, where a simple vector would do just as good (just use push_bask and pop_back).
To be clear, I don't suggest to change anything, it's just sincere curiosity 🙂

[v-Golubev]

I finally understood your question :) Actually you are right, we can easily use vector here, the logic doesn't require elements pop/push exactly from the beginning

[IvanNovoselov]

Again, with signatures like this it's unclear whether we want to get the initialized_info_map as a result of this function call or we want to allow users to specify some loop_infos. In other words, is it valid to pass non-empty initialized_info_map?

[v-Golubev]

Please take a look at the description:

    /**
     * @brief Update Loop informations in LinearIR: Unified and ExpandedLoopInfo
     * @param linear_ir LinearIR
     * @param initializated_info_map Reference on a map [LoopInfo->RuntimeParams].
     * Can be used to pass in the method loop infos which were already initialized, e.g. by parallel domain optimization
     */
    void update_loop_info(const lowered::LinearIRCPtr& linear_ir, LoopInfoRuntimeParamsMap& initializated_info_map) const;

I tried to point out there that user can pass a map with already initialized infos => it is valid to pass non-empty initializated_info_map. Please let me know if the description is not clear enough -- I will try to reformulate it

[IvanNovoselov]

After reviewing this one and a few subsequent PRs, I realized that probably these kind of interfaces with a map reference as an argument is an inevitable and a lesser evil at this point. So let's leave it as is.
I think that the real problem is how the loop_infos' are stored, especially in the case splitted loops (but for Expanded loop info as well). To make these kind of updates more transparent, we need to reorganize loop_infos' connectivity on the design level.
Please remind me about this on the meeting, I'll share some more details about this. It would be interesting to know your opinion.

[IvanNovoselov]

Why do we need to pass shapes and layouts to the optimizer explicitly here, when we already passes him m_io_desc during the initialization?
The optimizer can extract shapes and layouts internally. It should also be responsible for tensor_rank and data_offsets update. This would allow to simplify signatures and reduce code connectivity.
As a user, I would expect something like:

if (!m_optimizer.optimize(...)) // if m_optimizer.optimize() is true => then the optimizer handled everything correctly
    update_data_offsets(extract_shapes(), extract_layouts());        // if m_optimizer.optimize() is false => optimizer is disabled, and we need to init offsets manually

[v-Golubev]

I like your suggestion, but currently, update_data_offsets is RuntimeConfigurator's protected API, so we can't just call it inside optimizer. I think it will be easier to implement this idea within 148891, when all the needed logic is performed by lowered passes.

[IvanNovoselov]

so we can't just call it inside optimizer

I'm pretty sure we can, because ParallelWAOptimizer is an inner class of RuntimeConfigurator: it has access even to the configurators' private members.
So the only thing we need to do to call update_data_offsets from the optimizer is to pass it a pointer to the configurator: m_optimizer.optimize(this)

[v-Golubev]

I applied your suggestion in the follow-up PR. Thanks for the proposal!

[IvanNovoselov]

Can we make the optimizer responsible for calling update_loop_info for affected loops?
For example:

1. Implement update_loop_info(loop_id) that updates 1 loop
2. modify update_loop_info(linear_ir) so that it just calls update_loop_info(loop_id) for all loops in loop_map
3. Update loops before M dim optimization, and make the optimizer to update the necessary loops if the dimension was splitted.
   This would enchance modularity and would simplify the logic. Otherwise we have to complicate signatures, and carry some opaque states from optimizer downwards (shapes/layouts/loop_info), and we don't even know if the states were updated or not.

[IvanNovoselov]

In general, we should think of the optimizer as a pass: it takes lir and updates everything that needs to be updated, so all other passes/functions could work completely independently (not semantically, since passes may depend on each other, but functionally, i.e. the only state that is passes between the passes (sorry 🙃) is lir itself

[v-Golubev]

Again, update_loop_info is RuntimeConfigurator's API, so it should be extracted to apply your suggestion. It seems like the best way here is to extract it as a separate lowered pass. I believe this should be done within work on 148891 ticket (where all RuntimeConfigurator::update work should be done by passes) in order to avoid a mess of 2 approaches in one place

[IvanNovoselov]

The optimizer is currently implemented as a part of RuntimeConfigurator, so as a quick solution we can pass the configurator to the optimizer to make the necessary updates. The idea is to isolate all the optimization-related logic in one class.
Yes, the pass solution is better, and it's a long-term fix, but it's not clear when it will be addressed. So I would still suggest to update the optimizer a bit now, if it won't take too much time.

[v-Golubev]

The refactoring is done in the follow-up PR

[IvanNovoselov]

@v-Golubev, please address the remaining comments in a separate PR