You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Adding support for SSH tunnels for any database destination will require adding support in two places:
in the database connector itself
in normalization
this issue is split from #312 to capture the fact that it needs to be added to normalization.
Describe the solution you’d like
Today DBT is invoked from this line in this bash script. The simplest way to add ssh tunneling functionality to normalization is by adding a function/block of code after configuredbt and before dbt run which initializes the SSH tunnel and removes it afterwards (potentially using trap).
I don't love this implementation mainly because I don't love the idea of making an already non trivial bash script even less trivial. However, invoking DBT directly from Python is not recommended. So our options are either:
run a python script which starts the SSH tunnel and forks another process to run dbt run
keep using the bash script
If we go with option 2 the bash addition we'd add would probably:
read the config.json (bound to the $CONFIG_FILE var) input into the connector and detects if SSH configuration is present. The currently proposed JsonSchema for configuring SSH tunnel functionality can be found here in the spec.json file.
Depending on the kind of SSH configuration (ssh key vs. username/password based) run the right ssh command. See the SSH Tunnel section in this RFC for an example of the command we'd need to run to get an ssh tunnel using private keys.
The text was updated successfully, but these errors were encountered:
Tell us about the problem you're trying to solve
Adding support for SSH tunnels for any database destination will require adding support in two places:
this issue is split from #312 to capture the fact that it needs to be added to normalization.
Describe the solution you’d like
Today DBT is invoked from this line in this bash script. The simplest way to add ssh tunneling functionality to normalization is by adding a function/block of code after
configuredbt
and beforedbt run
which initializes the SSH tunnel and removes it afterwards (potentially usingtrap
).I don't love this implementation mainly because I don't love the idea of making an already non trivial bash script even less trivial. However, invoking DBT directly from Python is not recommended. So our options are either:
dbt run
If we go with option 2 the bash addition we'd add would probably:
config.json
(bound to the$CONFIG_FILE
var) input into the connector and detects if SSH configuration is present. The currently proposed JsonSchema for configuring SSH tunnel functionality can be found here in thespec.json
file.ssh
command. See theSSH Tunnel
section in this RFC for an example of the command we'd need to run to get an ssh tunnel using private keys.The text was updated successfully, but these errors were encountered: