Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

#1302: Refactor EC2 LaunchTemplate to use data model #1312

Merged
merged 2 commits into from
Jun 17, 2024
Merged

Conversation

achantavy
Copy link
Contributor

Fixes #1302: refactors EC2 launch template sync to use data model. This way, writes to the graph are automatically batched and write failures are retried.

@achantavy
Copy link
Contributor Author

cc: @heryxpc @chandanchowdhury


# Reformat some fields
current['Id'] = f"{version['LaunchTemplateId']}-{version['VersionNumber']}"
current['CreateTime'] = str(int(version['CreateTime'].timestamp()))
Copy link
Contributor

@heryxpc heryxpc Jun 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same as above comment about the expected value of CreateTime.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Always available.

Side note: neo4j datetime support has improved a lot since this module was first written so we don't even need this str(int()) thing anymore, but for the sake of not breaking any interfaces and keeping the existing behavior I will defer that change to a future PR.

result: list[dict[str, Any]] = []
for template in templates:
current = template.copy()
current['CreateTime'] = str(int(current['CreateTime'].timestamp()))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is it safe to consider CreateTime a always present ISO 8601 format date or is it safer to use a .get and return a null value in case it's missing?

Copy link
Contributor

@heryxpc heryxpc left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, left a nit comment, no rush to apply it.

@achantavy achantavy merged commit 96ae24e into master Jun 17, 2024
5 checks passed
@achantavy achantavy deleted the launchtemplate branch June 17, 2024 15:47
achantavy added a commit that referenced this pull request Jun 17, 2024
Forgot to remove some manual index creations in #1312.
chandanchowdhury pushed a commit to juju4/cartography that referenced this pull request Jun 26, 2024
…artography-cncf#1312)

Fixes cartography-cncf#1302: refactors EC2 launch template sync to use data model. This
way, writes to the graph are automatically batched and write failures
are retried.
chandanchowdhury pushed a commit to juju4/cartography that referenced this pull request Jun 26, 2024
SecPrez pushed a commit to SecPrez/cartography that referenced this pull request Nov 10, 2024
…artography-cncf#1312)

Fixes cartography-cncf#1302: refactors EC2 launch template sync to use data model. This
way, writes to the graph are automatically batched and write failures
are retried.
SecPrez pushed a commit to SecPrez/cartography that referenced this pull request Nov 10, 2024
chandanchowdhury pushed a commit to chandanchowdhury/cartography that referenced this pull request Nov 27, 2024
…artography-cncf#1312)

Fixes cartography-cncf#1302: refactors EC2 launch template sync to use data model. This
way, writes to the graph are automatically batched and write failures
are retried.

Signed-off-by: chandanchowdhury <[email protected]>
chandanchowdhury pushed a commit to chandanchowdhury/cartography that referenced this pull request Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

EC2 launch template sync TransactionOutOfMemoryError
2 participants