Fix CLI connections import and migrate logic from secrets to Connection model

[natanweinberger]

This is a bug fix for the CLI command connections_import. I originally added this CLI functionality in #15177. The bug was reported by @mudravrik here.

Bug summary

Running airflow connections import <filepath> results in an AttributeError.

Traceback (most recent call last):
...
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 244, in connections_import
    _import_helper(args.file)
  File "/home/airflow/.local/lib/python3.8/site-packages/airflow/cli/commands/connection_command.py", line 272, in _import_helper
    key: value for key, value in conn_values.items() if key in allowed_fields
AttributeError: 'Connection' object has no attribute 'items'

Root cause

I called the existing function load_connections_dict to get the contents of the target file, thinking it returned a list of dictionaries. When I later looped through the list, I called the dictionary method .items() on each connection. However, each connection was actually stored into a Connection model instance rather than a dictionary, so the call I made to .items() resulted in an AttributeError.

Solution

In taking a closer look at this issue, I realized the code block that contained the bug can be removed altogether. Having a list of Connection models to begin with (rather than dictionaries) eliminated the need to even examine the keys and values of the dictionaries. I had only done that to safely load the data into Connection models in the first place.

Preventing future issues

To avoid future issues, I also changed how the tests work for providing sample data. Before, a dictionary of sample data was set as the return value of load_connections_dict, which actually returns an object of type List[Connection]. Now, that dictionary of sample data is set to be the return value of a lower level function, _parse_secrets_file, which does no heavy lifting outside of parsing a JSON, YAML or env file and returns a dictionary.

Bug originally introduced in #15177, which closed #9855.

[natanweinberger]

I've refactored the functionality that reads data from a file and creates connections from it.

• the helper functions to parse the contents of JSON, YAML and env files are moved from airflow/secrets/local_filesystem.py to airflow/utils/parse.py, as suggested by @mik-laj above
• there is a new method Connection.from_dict
• the connections import CLI function uses the parse utils and Connection.from_dict to import connections from a file, which eliminates dependency on secrets

I was as conservative as possible in refactoring the secrets backend in this PR, but it seems like there's not much that uses it. I'd be open to doing some broader clean up there as a follow-up PR.

[natanweinberger]

@mik-laj I've incorporated your feedback, can you please re-review when you have a chance? 🙏