Automatic cleanup jobs

[achantavy]

Uses the schema in #1038 to automatically create cartography cleanup jobs.

Refactors AWS EMR sync to use this new method and adds tests.

Before

{
    "statements": [
        {
            "query": "MATCH (:AWSAccount{id: $AWS_ID})-[:RESOURCE]->(n:EMRCluster) WHERE n.lastupdated <> $UPDATE_TAG WITH n LIMIT $LIMIT_SIZE DETACH DELETE (n)",
            "iterative": true,
            "iterationsize": 100
        },
        {
            "query": "MATCH (:AWSAccount{id: $AWS_ID})-[r:RESOURCE]->(:EMRCluster) WHERE r.lastupdated <> $UPDATE_TAG WITH r LIMIT $LIMIT_SIZE DELETE r",
            "iterative": true,
            "iterationsize": 100
        }
    ],
    "name": "cleanup EMRCluster"
}

def cleanup(neo4j_session: neo4j.Session, common_job_parameters: Dict) -> None:
    run_cleanup_job('aws_import_emr_cleanup.json', neo4j_session, common_job_parameters)

Now

def cleanup(neo4j_session: neo4j.Session, common_job_parameters: Dict) -> None:
    cleanup_job = GraphJob.from_node_schema(EMRClusterSchema(), common_job_parameters)
    cleanup_job.run(neo4j_session)

[ramonpetgrave64]

rename to cleanup_job_from_node_schema

[achantavy]

There is also a GraphJob.from_json_file(), so I think we should follow the naming convention

[ramonpetgrave64]

I love this

[ramonpetgrave64]

Looks good, but I really would like to see _build_cleanup_node_query() and _build_cleanup_rel_query() combined into one function.

[ramonpetgrave64]

No need for this conditional, since we would raise an exception first.

[ramonpetgrave64]

I think we should be explicit about cleaning up the sub_resource_rel, since we will always require the node to have it. And it would simplify the logic a bit in _build_cleanup_node_query()

[achantavy]

TODO: sketch out unifying this function with build_cleanup_node_query

[ramonpetgrave64]

Last thing, I think about the example path

[ramonpetgrave64]

👏