Skip to content

Commit

Permalink
Fix instantiating Vault Secret Backend during configuration
Browse files Browse the repository at this point in the history
When Secrets Backend are instantiated during configuration, not
all Airlfow packages are yet imported, because they need Secret
Backends. We have a weird cyclical relation between models,
configuration and settins which forces us to be extra careful
around configuration, settings and backends.

In this case top-level import of Connections by the Vault Secret
Backend triggered cyclic import problem (importing airflow models
require configuration to be fully loaded and initialized) but then
it could not be initialized because models needed to be imported
first.

The fix is to move Connections to be locally imported.
  • Loading branch information
potiuk committed Aug 31, 2021
1 parent 2aa443f commit dfa6c73
Showing 1 changed file with 11 additions and 3 deletions.
14 changes: 11 additions & 3 deletions airflow/providers/hashicorp/secrets/vault.py
Original file line number Diff line number Diff line change
Expand Up @@ -16,9 +16,8 @@
# specific language governing permissions and limitations
# under the License.
"""Objects relating to sourcing connections & variables from Hashicorp Vault"""
from typing import Optional
from typing import TYPE_CHECKING, Optional

from airflow.models.connection import Connection
from airflow.providers.hashicorp._internal_client.vault_client import _VaultClient
from airflow.secrets import BaseSecretsBackend
from airflow.utils.log.logging_mixin import LoggingMixin
Expand Down Expand Up @@ -206,7 +205,12 @@ def get_conn_uri(self, conn_id: str) -> Optional[str]:

return response.get("conn_uri") if response else None

def get_connection(self, conn_id: str) -> Optional[Connection]:
# Make sure connection is imported this way for type checking, otherwise when importing
# the backend it will get a circular dependency and fail
if TYPE_CHECKING:
from airflow.models.connection import Connection

def get_connection(self, conn_id: str) -> 'Optional[Connection]':
"""
Get connection from Vault as secret. Prioritize conn_uri if exists,
if not fall back to normal Connection creation.
Expand All @@ -215,6 +219,10 @@ def get_connection(self, conn_id: str) -> Optional[Connection]:
:rtype: Connection
:return: A Connection object constructed from Vault data
"""
# The Connection needs to be locally imported because otherwise we get into cyclic import
# problems when instantiating the backend during configuration
from airflow.models.connection import Connection

response = self.get_response(conn_id)
if response is None:
return None
Expand Down

0 comments on commit dfa6c73

Please sign in to comment.