Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Build Airflow Slack connections from path variables #3813

Open
stacimc opened this issue Feb 20, 2024 · 0 comments
Open

Build Airflow Slack connections from path variables #3813

stacimc opened this issue Feb 20, 2024 · 0 comments
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs

Comments

@stacimc
Copy link
Collaborator

stacimc commented Feb 20, 2024

Current Situation

Currently, the Airflow connections for the Slack hook are configured through the Admin UI in production and by environment variable locally. The connection URI looks is formatted like: https://hooks.slack.com/services/foo/bar/baz. In order for the host to be parsed correctly, the string must be url encoded when set via an environment variable (ie, https://hooks.slack.com%2Fservices%2Ffoo%2Fbar%2Fbaz. This is somewhat tedious when switching between local testing set ups, so for convenience we modify the Slack environment variables in the entrypoint script to encode them properly (#3805).

Suggested Improvement

Modifying the environment variables in the entrypoint script is not ideal and has led to confusion in the past (#3805). We could instead use an Airflow variable to define the Slack webhook URL and then use it to create the Connection in the SlackMessage class.

Benefit

  • avoids confusing modification of environment variables
  • allows the webhook url to be obfuscated in the Airflow UI (as part of a Variable, rather than the host component of a Connection)
@stacimc stacimc added 🟩 priority: low Low priority and doesn't need to be rushed 🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🧱 stack: catalog Related to the catalog and Airflow DAGs labels Feb 20, 2024
@openverse-bot openverse-bot moved this to 📋 Backlog in Openverse Backlog Feb 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🤖 aspect: dx Concerns developers' experience with the codebase 🧰 goal: internal improvement Improvement that benefits maintainers, not users 🟩 priority: low Low priority and doesn't need to be rushed 🧱 stack: catalog Related to the catalog and Airflow DAGs
Projects
Status: 📋 Backlog
Development

No branches or pull requests

1 participant