Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove all instances of cloneDeep() #323

Closed
tokebe opened this issue Oct 18, 2021 · 3 comments
Closed

Remove all instances of cloneDeep() #323

tokebe opened this issue Oct 18, 2021 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@tokebe
Copy link
Member

tokebe commented Oct 18, 2021

After investigation of the below query, it's become apparent that the use of cloneDeep in any function which is expected to be called many times in quick succession, or is expected to perform transformations on large objects, i.e. _createBTEEdges(), is more-or-less bound to cause performance issues (both in responsiveness and overall resource use).

It would probably be worthwhile to figure out how best to change the internal flow of data such that any current uses can be removed. Currently all uses are in qedge2bteedge.

The following query causes server crash due to out-of-memory exception while cloning:

{
    "message": {
        "query_graph": {
            "nodes": {
                "n0": {
                     "ids": ["MONDO:0005359", "SNOMEDCT:197354009"],
                     "categories": ["biolink:DiseaseOrPhenotypicFeature"]
                },
                "n1": {
                    "categories": ["biolink:DiseaseOrPhenotypicFeature"]
                },
                "n2": {
                    "categories": ["biolink:Gene"]
                },
                "n3": {
                    "categories": ["biolink:Drug", "biolink:SmallMolecule"]
                }
            },
            "edges": {
                "e01": {
                    "subject": "n0",
                    "object": "n1",
                    "predicates": ["biolink:has_real_world_evidence_of_association_with"]
                },
                "e02": {
                    "subject": "n2",
                    "object": "n1",
                    "predicates": ["biolink:gene_associated_with_condition"]
                },
                "e03": {
                    "subject": "n3",
                    "object": "n2",
                    "predicates": ["biolink:affects", "biolink:interacts_with"]
                }
            }
        }
    },
    "submitter": "WorkflowBNotebook",
    "stream_progress": false,
    "query_options": {
        "kp_timeout": 1800
    }
}
@tokebe
Copy link
Member Author

tokebe commented Oct 20, 2021

Tests show that removing cloneDeep improves memory usage, but does not prevent the above query from crashing BTE.

@colleenXu
Copy link
Collaborator

@tokebe @newgene Can this be closed or is this an ongoing task?

@tokebe
Copy link
Member Author

tokebe commented Oct 26, 2021

This can be closed.

@tokebe tokebe closed this as completed Oct 26, 2021
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants