Aggregated optimization changes #2744

monrostar · 2024-03-16T22:18:32Z

Related issue: #2738

…h big datasets

…rate call and even move with many relations

…eat/aggregated-prs

…aggregated-prs

monrostar · 2024-03-16T22:21:57Z

@michaelbromley
I created this PR to pull all the changes together and refactor them to avoid conflicts.

In this PR I will optimize queries inside TransactionalConnection, it turned out to be difficult because there are assets inside collections that don't have reverse relation. We need to figure something out, maybe this relation can be found through metadata

The problem arises at this point

vendure/packages/core/src/entity/asset/orderable-asset.entity.ts

Line 27 in 7dd258c

@ManyToOne(type => Asset, { eager: true, onDelete: 'CASCADE' })

…lations without any eager relations

…rseSide relation for list-query-builder

monrostar · 2024-03-17T09:57:53Z

@michaelbromley We are just a little bit away from the final optimization of the most important methods

In overview:

ListQueryBuilder.build was refactored and covered more edgecases
softDelete from ProductService was optimized
Get existing translations for TranslatorSaver.update was optimized
assignToChannel from ChannelService was optimized
findOneInChannel and findByIdsInChannel were optimized in TransactionalConnection

Also, this PR includes fixes that address this issue #2738

tianyingchun · 2024-03-19T09:09:11Z

how progress of this PR :)

monrostar · 2024-03-19T10:09:19Z

When i will be available

# Conflicts: # packages/core/src/service/helpers/entity-hydrator/entity-hydrator.service.ts # packages/core/src/service/helpers/list-query-builder/list-query-builder.ts # packages/core/src/service/services/channel.service.ts

…th of eager relations

monrostar · 2024-03-19T18:00:03Z

PR is ready 🚀

michaelbromley

Excellent! Just a couple of small "tidy up" points. Thanks again for this incredible effort.

michaelbromley · 2024-03-20T08:30:21Z

packages/core/src/service/helpers/utils/tree-relations-qb-joiner.ts

+ *                       has already been joined to avoid adding duplicate join statements.
+ * @returns boolean Returns true if the relation has already been joined (based on the alias), false otherwise.
+ * @template T extends VendureEntity The entity type for which the query builder is configured.
+ */


Looks like this doc block is out of place?

Yeah, I don't know how to do it here. If you have recommendations, I'd be glad to change it and maybe I need to reduce the amount of text...

Looks like this doc block is out of place?

I deleted the documentation that was accidentally copied and updated the documentation I was adding for the method. Check it out, please.

michaelbromley · 2024-03-20T08:32:55Z

packages/core/src/entity/asset/orderable-asset.entity.ts

@@ -24,7 +24,7 @@ export abstract class OrderableAsset extends VendureEntity implements Orderable
    assetId: ID;

    @Index()
-    @ManyToOne(type => Asset, { eager: true, onDelete: 'CASCADE' })
+    @ManyToOne(type => Asset,  { eager: true, onDelete: 'CASCADE' })


extra space

ok, I will fix it

extra space

done

michaelbromley · 2024-03-20T08:33:59Z

packages/core/src/connection/transactional-connection.ts

 } from 'typeorm';
+import { DataSource } from 'typeorm/data-source/DataSource';


should this (and the InjectDataSource above) really be deep imports?

ok, I will fix it. this is from my IDE, I didn't noticed

done

should this (and the InjectDataSource above) really be deep imports?

monrostar added 12 commits March 15, 2024 16:56

fix(core): self-referencing relations Not unique table/alias

4034445

perf(core): upgrade sql requests for more performant memory usage wit…

16cfc07

…h big datasets

perf(core): Upgrade EntityHydrator to add more performance to any hyd…

469a02b

…rate call and even move with many relations

fix(core): update self-related manual joins with deep eager relations

10c0798

chore(core): clean up code

4824d86

fix(core): update test cases for new TestEntities order

5eb6391

feat(core): optimization for assignToChannels method

0750065

feat(core): fix changes for new getAssignedEntityChannels method

dc463ba

Merge branch 'perf/performance-improvement-in-entity-hydrator' into f…

4486cd2

…eat/aggregated-prs

Merge branch 'perf/optimize-assign-to-channels' into feat/aggregated-prs

949dc54

Merge branch 'perf/product-soft-delete-performance-update' into feat/…

85821b7

…aggregated-prs

feat(core): add optimization into findOneInChannel and more refactoring

c376eef

monrostar mentioned this pull request Mar 17, 2024

fix(core): self-referencing relations Not unique table/alias #2740

Merged

5 tasks

monrostar changed the title ~~Feat/aggregated prs~~ Aggregated optimization changes Mar 17, 2024

monrostar added 3 commits March 17, 2024 09:53

perf(core): optimize translatable-saver.ts query to get existingTrans…

661c2e9

…lations without any eager relations

fix(core): Added processing of deeply nested relations that lack Reve…

eaca1c9

…rseSide relation for list-query-builder

fix(core): fix condition for tree type of metadata in list-query-builder

11e3904

monrostar added 2 commits March 19, 2024 12:23

Merge branch 'minor' into feat/aggregated-prs

d1494bd

# Conflicts: # packages/core/src/service/helpers/entity-hydrator/entity-hydrator.service.ts # packages/core/src/service/helpers/list-query-builder/list-query-builder.ts # packages/core/src/service/services/channel.service.ts

feat(core): Added edge case handling with customFields as well as dep…

c92bf6d

…th of eager relations

monrostar marked this pull request as ready for review March 19, 2024 17:22

monrostar added 3 commits March 19, 2024 18:25

feat(core): Clean up code and remove joinEagerRelations helper

b54f0f0

feat(core): Fix import in the unit test case

7af0d4a

feat(core): Fix import in the unit test case

5f836b9

feat(core): Fix condition for tree type of all relation entities

0dfb4f6

michaelbromley reviewed Mar 20, 2024

View reviewed changes

monrostar added 2 commits March 20, 2024 09:42

chore(core): Code style fixes

8a15a26

docs(core): Update docs for joinTreeRelationsDynamically

d6c50d5

monrostar requested a review from michaelbromley March 20, 2024 08:48

chore(core): Remove unused import

a444894

michaelbromley merged commit 48b239b into vendure-ecommerce:minor Mar 20, 2024
10 of 12 checks passed

monrostar deleted the feat/aggregated-prs branch March 20, 2024 10:01

michaelbromley mentioned this pull request Oct 14, 2024

missing FROM-clause entry for table "collection__translations" #3107

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Aggregated optimization changes #2744

Aggregated optimization changes #2744

monrostar commented Mar 16, 2024 •

edited

Loading

monrostar commented Mar 16, 2024 •

edited

Loading

monrostar commented Mar 17, 2024 •

edited

Loading

tianyingchun commented Mar 19, 2024

monrostar commented Mar 19, 2024 via email •

edited

Loading

monrostar commented Mar 19, 2024

michaelbromley left a comment

michaelbromley Mar 20, 2024

monrostar Mar 20, 2024

monrostar Mar 20, 2024

michaelbromley Mar 20, 2024

monrostar Mar 20, 2024

monrostar Mar 20, 2024

michaelbromley Mar 20, 2024

monrostar Mar 20, 2024

monrostar Mar 20, 2024

		} from 'typeorm';
		import { DataSource } from 'typeorm/data-source/DataSource';

Aggregated optimization changes #2744

Aggregated optimization changes #2744

Conversation

monrostar commented Mar 16, 2024 • edited Loading

monrostar commented Mar 16, 2024 • edited Loading

monrostar commented Mar 17, 2024 • edited Loading

In overview:

tianyingchun commented Mar 19, 2024

monrostar commented Mar 19, 2024 via email • edited Loading

monrostar commented Mar 19, 2024

michaelbromley left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

monrostar commented Mar 16, 2024 •

edited

Loading

monrostar commented Mar 16, 2024 •

edited

Loading

monrostar commented Mar 17, 2024 •

edited

Loading

monrostar commented Mar 19, 2024 via email •

edited

Loading