GScan improvements, propagate state

[kinow]

These changes close #721 and supersede #733 (not yet, but will in a few more commits 🤞 )

Requirements check-list

• I have read CONTRIBUTING.md and added my name as a Code Contributor.
• Contains logically grouped changes (else tidy your branch by rebase).
• Does not contain off-topic changes (use other PRs for other changes).
• Appropriate tests are included (unit and/or functional).
• No change log entry required (why? e.g. invisible to users).
• No documentation update required.

[kinow]

While working on #733 for #721, I noticed the computed variable that would re-trigger more elements in the UI to be updated than desired.

In this pull request I will move the GScan data out of the component, into Vuex, and update it incrementally based on deltas. The goal is to have the least amount of elements being updated as possible.

Here's a screenshot from this morning where I hacked the code to print whenever a TreeItem was updated.

Even though five was being updated, the TreeItem's for jh1, rose/run1, etc, were being updated as well 😬 which results in a major performance issue.

[kinow]

I've split the workflows.module since it had data used by GScan, and also by the Tree view. The tree view data is relatively more complex at this point, so moving it out simplifies the workflows.module.

[kinow]

The code added here was in workflows/deltas.js, but was used only by the Tree view. Instead of moving to the tree view, I've moved it here to common since if we ever need to use a delta that creates a lookup, anyone can simply call this function ☝️

I think I won't need a lookup for GScan, at least for now. Or at least I am avoiding adding one unless really needed. But if I need to add one, then I'll use this function.

[kinow]

I've implemented most of what I had in mind for GScan. But this afternoon just before a meeting I decided to look if the TreeItem's were still being updated, and strangely everything was still updating.

I created a smaller test scenario, with yarn run serve and only 1 workflow one. Then I push a delta manually to have another worklfow a. This workflow is sibling of one in the GScan tree, but one is being updated too, which is not really desirable.

I'm investigating now why that's happening, and so far these are the two best leads: vuejs/vue#7257 & vuejs/vue#10115 (comment) since they match what why-did-you-render is reporting in the browser console... more to come 👍 🛏️

[kinow]

All right, coming back to this one after lunch, and strong ☕ to help me fixing conflicts (no conflicts yet 🎉 ), and remember where was I.

[kinow]

Hmm, I think I figured how to store the data in each node in a way that I can update with added/updated/pruned deltas.

The leaf nodes in the GScan tree are workflows. They will keep the same structure (latestStates + stateTotals). The intermediary nodes (workflow-name-part nodes, as I called them 🧐 ) will have a computed variable with same name. This way the component's templates won't need to be changed.

However, the real data will hold:

• maps/dictionaries backing the latestStates and stateTotals computed variables, where
  • the key is the leaf node ID
  • the value is the leaf node latestStates or stateTotals

Then the computed variables will display the summarized numbers of latestStates and stateTotals for all entries in the maps/dictionaries.

This is necessary so that we can:

1. add the values to the dictionary/map (deltas.added),
2. update the existing values (deltas.updated), and
3. remove the values when the node appears in a deltas.pruned

The approach of keeping a pointer to the leaf-node stateTotals/latestStates is necessary so that when a leaf-node is pruned, we can safely delete it from the map/dictionary, causing the computed variables to be calculated again.

An alternative would be to store the ID's only. And then use the lookup to locate the node, and finally retrieve its latestStates and stateTotals. Will think on the pros & cons of each alternative, and try to come up with some code.

At the moment, the code in this pull request correctly produces a GScan tree with added nodes, with no stateTotals/latestStates. It's pending:

• refactor GScan using tree lookup
• performance and code-review improvements (found a few things that were unnecessary, or could be improved)
• add nodes with deltas.added to the GScan tree using a tree/lookup approach as in the tree component
• update nodes with deltas.updated
• prune nodes with deltas.pruned
• create a WorkflowStateSummary component to isolate the functionality for task summary state in a workflow
• sort the nodes again after an update (e.g. you have workflows a/run1 and a/run2, both stopped; you start a/run2, now it must move to the top)
• propagate states

[kinow]

Done for the day, next week will start sorting the parent.children of a node when it was updated. Noticed this error when I started a random run and it didn't move up the tree.

Previously, we were always re-generating the whole tree 😨 so it looked like the tree element was correctly re-positioned… but it was actually due to the new three being created from scratch.

[kinow]

Think I found how to improve GScan a bit more, preventing it from re-rendering all the nodes in the GScan tree, refs:

• https://dev.to/veebuv/5-extremely-easy-ways-to-drastically-improve-your-vuejs-app-s-speed-5k0
• https://betterprogramming.pub/6-ways-to-speed-up-your-vue-js-application-2673a6f1cde4

[kinow]

Will edit the last commit tomorrow, but wanted to save what I wrote today. Pending more manual tests with working workflows. Here's a preview of what it's looking like in the offline mode right now (modified one workflow a little, will undo tomorrow since it's used by e2e tests).

[kinow]

This PR can probably be closed.

I started working on the GScan here to propagate states. Then realized we had some improvements that could be applied to GScan, and started changing that here.

Then another PR came that changed the callbacks and deltas in GScan 😬

I moved the improvements to #802 , mimicking with we have in the Tree view.

There's one commit in this PR that tries to propagate the states in GScan. I think someone could either drop the other commits and fix conflicts after #802 has been merged, and then finish the current work.

Or just close this and use as reference for future implementations.

🖖

[hjoliver]

Thanks for the advice @kinow, good to know 👍

[wxtim]

Stale.