How do you feel about putting hook configuration in airflow.cfg?

[jbalogh]

We distribute our configs via text files managed by puppet so it would be awkward to put that into the db for airflow. Would you accept a patch that looks for conn_id in settings.conf before checking the database?

[mistercrunch]

Are you concerned about the security aspect of storing connection information in a database, or is it more about deployment and how you get the information in there?

If it's the later you could easily setup a simple script that populates the database. It would look like what's in init_db here https://github.com/mistercrunch/airflow/blob/master/airflow/utils.py#L79, but with session.merge instead of session.add.

It's true that I'd also rather have connection information in source control rather than in the db. Having it in the db is an artifact of early development. But it does work. I believe all of the connection info fetching is done through here: https://github.com/mistercrunch/airflow/blob/master/airflow/utils.py#L79

[jbalogh]

I'm more interested in the deployment / config management aspect of it. Writing a script to manage the db config would definitely be possible, but it's another moving part to write and test and make sure it handles config changes properly.

[mistercrunch]

For now your best bet is either a small python script or a one time manual entry.

Ideally eventually running airflow initdb should have a hook into some sort of airflow_settings that you'd define on your side.