Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Added MWAA Module #32

Merged
merged 35 commits into from
Apr 13, 2024
Merged

Added MWAA Module #32

merged 35 commits into from
Apr 13, 2024

Conversation

mahesh-balumuri
Copy link
Contributor

No description provided.

@dgraeber
Copy link

@kukushking we should touch base on this. I don't see a requirements.txt reference and I am thinking that this will need a matching requirements?

@kukushking
Copy link
Contributor

Thanks @dgraeber yes, this will need matching requirements file - @mahesh-balumuri is on top of this, working on a DAG and will add the requirements for it.

@mahesh-balumuri mahesh-balumuri marked this pull request as draft March 28, 2024 14:06
@kukushking
Copy link
Contributor

Can you please add the example DAG module in README.MD table and update CHANGELOG.MD

@mahesh-balumuri mahesh-balumuri marked this pull request as ready for review April 12, 2024 06:22
Copy link
Contributor

@kukushking kukushking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you please run the below for the formatting checks to pass:

scripts/fix.sh --language python --path modules/examples/airflow-dags/

README.md Outdated Show resolved Hide resolved
@@ -0,0 +1,44 @@
deploy:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is a topic of advanced Seed-Farmer usage: to make this module generic we can leverage publishGenericEnvVariables. This will need a change of the naming conventions of the module parameters but will allow to use the module with other projects/deployments.

@kukushking
Copy link
Contributor

@mahesh-balumuri I just ran an E2E test and the DAG has succeeded. Great work!

kukushking
kukushking previously approved these changes Apr 12, 2024
Copy link
Contributor

@kukushking kukushking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fantastic work! Thank you!

@@ -0,0 +1,22 @@
--constraint https://raw.githubusercontent.com/awslabs/aiops-modules/main/data/mwaa/requirements/constraint-requirements.txt
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not familiar with how constraints work but are we unable to reference the local file? I'm just worried that if we make a change to this requirements file and the constraints file in a dev branch, it will still be pulling the constraints file from main.

@kukushking kukushking self-requested a review April 12, 2024 16:37
kukushking
kukushking previously approved these changes Apr 13, 2024
Copy link
Contributor

@kukushking kukushking left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Great work, thank you so much!

Signed-off-by: Anton Kukushkin <[email protected]>
@kukushking kukushking merged commit aed4ca0 into awslabs:main Apr 13, 2024
11 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants