-
Notifications
You must be signed in to change notification settings - Fork 1.4k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Self-managed Apache Airflow on EKS #878
Conversation
Agree. We should we move the MWAA example under Analytics folder and then rename the folders to The main difference is that MWAA has nothing to do with EKS unlike self-managed airflow hosted on EKS. MWAA example is only to trigger the job on EKS I have added additional information to Docs(apache-airflow.md) and Readme as well. |
Check Markdown links / markdown-link-check error is expected as the link is not available in MAIN until this PR is merged |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Looks great - few minor details to correct on the CI checks and then good to go !
Plan error seems to be related to RAY example. Let me see if that needs fixing
|
* Airflow helm addon example * Airflow helm addon example * Airflow helm addon example * Airflow deployment docs added * Airflow deployment docs added * Airflow doc added * Postgres db version updated * removed commented code * Updates to ariflow values * added region data source * Added versions file * Added PVC for EFS * Update modules/irsa/outputs.tf Co-authored-by: Bryant Biggs <[email protected]> * Update modules/irsa/outputs.tf Co-authored-by: Bryant Biggs <[email protected]> * s3 module added and updated with comments * resolved conflicts with variables * Docs updated * removed irsa outputs to avoid count issues * Updates to values and readme * Updates to readme and values * fix to tfsec warnings * Terraform helm release issue fixed with airflow * removed static password * removed special characters for postgres password Co-authored-by: Bryant Biggs <[email protected]>
What does this PR do?
Closes the issue: #810
🛑 Please open an issue first to discuss any significant work and flesh out details/direction - we would hate for your time to be wasted.
Consult the CONTRIBUTING guide for submitting pull-requests.
Motivation
Running Apache Airflow on Kubernetes is the common pattern for Data teams. This example will help customers to build production ready Apache Airflow on EKS.
This is also a request from few AWS customers
More
pre-commit run -a
with this PRNote: Not all the PRs required examples and docs except a new pattern or add-on added.
For Moderators
Additional Notes