[CT-2062] [Bug] Dbt doesn't run when FIPS mode is enabled

[ZorinAnton]

Is this a new bug in dbt-core?

• I believe this is a new bug in dbt-core
• I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

Execution failure with

ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

Expected Behavior

normal execution

Steps To Reproduce

Execute dbt run on a RHEL>=8.6 with FIPS mode enabled

Relevant log output

12:36:42.801111 [info ] [MainThread]: Running with dbt=1.4.1
12:36:42.802962 [debug] [MainThread]: running dbt with arguments {'write_json': True, 'use_colors': False, 'printer_width': 80, 'version_check': True, 'partial_parse': True, 'static_parser': True, 'profiles_dir': '/home/cloud-user/dbt-test/TestFilterPushdown-PostgreSQL-0', 'send_anonymous_usage_stats': False, 'quiet': False, 'no_print': False, 'cache_selected_only': False, 'project_dir': '/home/cloud-user/dbt-test/TestFilterPushdown-PostgreSQL-0', 'which': 'run', 'rpc_method': 'run', 'indirect_selection': 'eager'}
12:36:42.803135 [debug] [MainThread]: Tracking: do not track
12:36:42.803470 [error] [MainThread]: Encountered an error:
[digital envelope routines: EVP_DigestInit_ex] disabled for FIPS
12:36:42.804416 [error] [MainThread]: Traceback (most recent call last):
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/main.py", line 135, in main
    results, succeeded = handle_and_check(args)
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/main.py", line 198, in handle_and_check
    task, res = run_from_args(parsed)
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/main.py", line 244, in run_from_args
    with track_run(task):
  File "/usr/lib64/python3.9/contextlib.py", line 119, in __enter__
    return next(self.gen)
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/main.py", line 206, in track_run
    dbt.tracking.track_invocation_start(config=task.config, args=task.args)
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/tracking.py", line 273, in track_invocation_start
    get_invocation_start_context(active_user, config, args),
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/tracking.py", line 210, in get_invocation_start_context
    data = get_invocation_context(user, config, args)
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/tracking.py", line 197, in get_invocation_context
    "project_id": None if config is None else config.hashed_name(),
  File "/home/cloud-user/.local/lib/python3.9/site-packages/dbt/config/project.py", line 666, in hashed_name
    return hashlib.md5(self.project_name.encode("utf-8")).hexdigest()
ValueError: [digital envelope routines: EVP_DigestInit_ex] disabled for FIPS

Environment

- OS: RHEL8.6 with FIPS mode enabled
- Python: 3.9.13
- dbt: 1.41

Which database adapter are you using with dbt?

postgres

Additional Context

No response

[dbeatty10]

Thanks for reporting this @ZorinAnton !

The situation

Here are a couple similar issues that other open-source projects have run into related to FIPS mode and MD5 hashing:

• MD5 not supported without usedforsecurity=False in FIPS 140-2 environment dask/dask#8608
• ValueError: error:060800A3:digital envelope routines:EVP_DigestInit_ex:disabled for fips s3tools/s3cmd#1005

Basically, it sounds like you got this error because your system wanted to make extra certain that the md5 hash we are performing isn't related to cryptographic security.

The fix

We aren't using this for cryptographic security, so I think the fix would just be to update this:

dbt-core/core/dbt/config/project.py

Lines 668 to 669 in df64511

	def hashed_name(self):
	return hashlib.md5(self.project_name.encode("utf-8")).hexdigest()

To be this instead:

    def hashed_name(self):
        return hashlib.md5(self.project_name.encode("utf-8"), usedforsecurity=False).hexdigest()

[ZorinAnton]

possible way to solve it: https://github.com/pytest-dev/pytest-randomly/pull/415/files

[jtcohen6]

Sounds like we should do this everywhere we're calling hashlib.md5.

And by that, I actually mean: We should swap out all those direct calls to hashlib.md5 with a consistent call to dbt.utils.md5, so that we can make this change in exactly one place.

[nielspardon]

Any chance this could be included in the next patch release?

[jtcohen6]

@nielspardon Sure, it feels like a precise-enough change, and unblocks your ability to use dbt on a FIPS-enabled machine running Python 3.9+.

We just cut v1.4.2rc1 yesterday, so this wouldn't be included until v1.4.3.

[nielspardon]

I guess that's unlucky timing then. How often do you perform patch releases?

[nielspardon]

@jtcohen6 I see 1.4.5 was released in the meantime and the fix for this was not backported. What's up?

[jtcohen6]

@nielspardon Sorry about that! We'll include in the next v1.4 patch