-
Notifications
You must be signed in to change notification settings - Fork 23
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Added MWAA Module #32
Conversation
@kukushking we should touch base on this. I don't see a requirements.txt reference and I am thinking that this will need a matching requirements? |
Thanks @dgraeber yes, this will need matching requirements file - @mahesh-balumuri is on top of this, working on a DAG and will add the requirements for it. |
Can you please add the example DAG module in README.MD table and update CHANGELOG.MD |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you please run the below for the formatting checks to pass:
scripts/fix.sh --language python --path modules/examples/airflow-dags/
@@ -0,0 +1,44 @@ | |||
deploy: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
This is a topic of advanced Seed-Farmer usage: to make this module generic we can leverage publishGenericEnvVariables. This will need a change of the naming conventions of the module parameters but will allow to use the module with other projects/deployments.
@mahesh-balumuri I just ran an E2E test and the DAG has succeeded. Great work! |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fantastic work! Thank you!
@@ -0,0 +1,22 @@ | |||
--constraint https://raw.githubusercontent.com/awslabs/aiops-modules/main/data/mwaa/requirements/constraint-requirements.txt |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I'm not familiar with how constraints work but are we unable to reference the local file? I'm just worried that if we make a change to this requirements file and the constraints file in a dev branch, it will still be pulling the constraints file from main
.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Great work, thank you so much!
Signed-off-by: Anton Kukushkin <[email protected]>
Signed-off-by: Anton Kukushkin <[email protected]>
No description provided.