Not able to link with pyo3 in debug mode on Windows

[dlaugt]

Bug Description

In my debug builds, for years, I used to compile my third-parties from the source code in debug mode.

For example, I compile python 3.10.8 in debug mode and it produces:
python_d.exe
python3_d.dll
python310_d.dll
python3_d.lib
python310_d.lib

On my build chain, I would like to add the python cryptography third-party. When I compile it from the source code in debug mode, I have the following error:

python_d -m build --wheel --no-isolation -C--build-option="build --debug"
...
     Running `rustc --crate-name cryptography_rust --edition=2018 src\lib.rs --error-format=json --json=diagnostic-rendered-ansi,artifacts,future-incompat --crate-type cdylib --emit=dep-info,link -C embed-bitcode=no -C debuginfo=2 --crate-type cdylib --cfg "feature=\"default\"" --cfg "feature=\"extension-module\"" -C metadata=d36a220355622476 --out-dir C:\python-modules\cryptography\src\rust\target\debug\deps -C incremental=C:\python-modules\cryptography\src\rust\target\debug\incremental -L dependency=C:\python-modules\cryptography\src\rust\target\debug\deps --extern asn1=C:\python-modules\cryptography\src\rust\target\debug\deps\libasn1-89a4a2858d227583.rlib --extern chrono=C:\python-modules\cryptography\src\rust\target\debug\deps\libchrono-9c0c4c35fe6d5120.rlib --extern once_cell=C:\python-modules\cryptography\src\rust\target\debug\deps\libonce_cell-d359c42b90186127.rlib --extern ouroboros=C:\python-modules\cryptography\src\rust\target\debug\deps\libouroboros-818767ba07ca8884.rlib --extern pem=C:\python-modules\cryptography\src\rust\target\debug\deps\libpem-9d903dcf6b15ed52.rlib --extern pyo3=C:\python-modules\cryptography\src\rust\target\debug\deps\libpyo3-78c5487a10cfe0e9.rlib -L native=C:\usrlib\python\libs`
...
   = note: LINK : fatal error LNK1181: cannot open input file 'python3.lib'

This error is coming from the following line:

pyo3/pyo3-build-config/src/impl_.rs

Line 1536 in 56f11c2

const WINDOWS_ABI3_LIB_NAME: &str = "python3";

If I change WINDOWS_ABI3_LIB_NAME to "python310_d", it is linking fine and the cryptography third-party is working.

However, if I change WINDOWS_ABI3_LIB_NAME to "python3_d", it is linking fine but I have an error on the runtime:
ImportError: Module use of python3_d.dll conflicts with this version of Python.

This error is coming the following line in cpython repo:
https://github.com/python/cpython/blob/5f4ae86a639fb84260d622e31468da21dc468265/Python/dynload_win.c#L309-L330

The variable buffer is equals to python310_d.dll and the variable import_python is equals to python3_d.dll.

Could the linking of PyO3 in debug mode on Windows be fixed?

Steps to Reproduce

1. Compile python in debug mode
2. Compile python cryptography in debug mode

Backtrace

Here is the backtrace of the runtime error when I change WINDOWS_ABI3_LIB_NAME to "python3_d":
python_d.exe -vvvvvvvvv -c "from cryptography.fernet import Fernet"
...
# trying C:\usrlib\python\Lib\site-packages\cryptography\hazmat\bindings\_rust_d.cp310-win_amd64.pyd
Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\usrlib\python\Lib\site-packages\cryptography\fernet.py", line 14, in <module>
    from cryptography.hazmat.primitives import hashes, padding
  File "<frozen importlib._bootstrap>", line 1078, in _handle_fromlist
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 688, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 883, in exec_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
  File "C:\usrlib\python\Lib\site-packages\cryptography\hazmat\primitives\padding.py", line 11, in <module>
    from cryptography.hazmat.bindings._rust import (
  File "<frozen importlib._bootstrap>", line 1027, in _find_and_load
  File "<frozen importlib._bootstrap>", line 1006, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 674, in _load_unlocked
  File "<frozen importlib._bootstrap>", line 571, in module_from_spec
  File "<frozen importlib._bootstrap_external>", line 1176, in create_module
  File "<frozen importlib._bootstrap>", line 241, in _call_with_frames_removed
ImportError: Module use of python3_d.dll conflicts with this version of Python.

Your operating system and version

Windows 10 21H2 (OS Build 19044.2130)

Your Python version (python --version)

Python 3.10.8 (in debug mode)

Your Rust version (rustc --version)

rustc 1.65.0 (897e37553 2022-11-02)

Your PyO3 version

0.15.2

How did you install python? Did you use a virtualenv?

Compilation from the source code in debug mode.

Additional Info

No response

[davidhewitt]

Hi @dlaugt, sorry it has taken me so long to look at this.

I confirmed what you have found here. It looks like you've identified two separate issues:

• Python has an issue when linking against python3_d.dll, which I've reported upstream at Cannot import extensions linked against python3_d.dll debug build on Windows python/cpython#101614
• We don't detect debug builds on Windows. Turns out this isn't very clean, I have a solution at link against pythonXY_d.dll for debug Python on Windows #2937