test_sysconfig test_library fails on macOS framework installs, like the python.org installer

[ned-deily]

Bug report

With the python.org 3.13.0a1 installer for macOS, the new test_library of test_sysconfig fails:

$ python3.13
Python 3.13.0a1 (v3.13.0a1:ad056f03ae, Oct 13 2023, 06:35:05) [Clang 13.0.0 (clang-1300.0.29.30)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
$ python3.13 -m test -w test_sysconfig
Using random seed 1311857515
Raised RLIMIT_NOFILE: 256 -> 1024
0:00:00 load avg: 1.72 Run 1 test sequentially
0:00:00 load avg: 1.72 [1/1] test_sysconfig
test test_sysconfig failed -- multiple errors occurred; run in verbose mode for details
test_sysconfig failed (2 failures)

== Tests result: FAILURE ==

1 test failed:
    test_sysconfig

0:00:00 load avg: 1.72 Re-running 1 failed tests in verbose mode in subprocesses
0:00:00 load avg: 1.72 Run 1 test in parallel using 1 worker process
0:00:00 load avg: 1.72 [1/1/1] test_sysconfig failed (2 failures)
Re-running test_sysconfig in verbose mode (matching: test_library, test_user_similar)
test_library (test.test_sysconfig.TestSysConfig.test_library) ... FAIL
test_user_similar (test.test_sysconfig.TestSysConfig.test_user_similar) ... FAIL

======================================================================
FAIL: test_library (test.test_sysconfig.TestSysConfig.test_library)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13/test/test_sysconfig.py", line 416, in test_library
    self.assertTrue(ldlibrary.startswith(f'libpython{major}.{minor}'))
AssertionError: False is not true

======================================================================
FAIL: test_user_similar (test.test_sysconfig.TestSysConfig.test_user_similar)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13/test/test_sysconfig.py", line 385, in test_user_similar
    self.assertEqual(user_path, expected)
AssertionError: '/Users/nad/Library/Python/3.13/lib/python3.13' != '/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13'
- /Users/nad/Library/Python/3.13/lib/python3.13
+ /Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13


----------------------------------------------------------------------
Ran 2 tests in 0.001s

FAILED (failures=2)

Linked PRs

• gh-110824 Fix test_sysconfig.test_library on framework builds. #113298

[FFY00]

@ned-deily could you post the output of python -m sysconfig? Thanks!

[ned-deily]

Platform: "macosx-10.9-universal2"
Python version: "3.13"
Current installation scheme: "posix_prefix"

Paths:
	data = "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11"
	include = "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/include/python3.13"
	platinclude = "/Library/Frameworks/Python.framework/Versions/3.13/include/python3.13"
	platlib = "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/site-packages"
	platstdlib = "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13"
	purelib = "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13/site-packages"
	scripts = "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/bin"
	stdlib = "/Library/Frameworks/Python.framework/Versions/3.13.0a1_11/lib/python3.13"

[...]

full_sysconfig_output.txt

[ned-deily]

@FFY00, I'm sorry that this is kind of late in the game, but, while reviewing @smontanaro's proposed fix in #113298, I realized I didn't know exactly what this is supposed to be testing, in particular, what is or should be the purpose on non-Windows builds of the variables mentioned in #88402. Are they intended to help with building extensions? With embedding Python? How do they fit in with the existing Embedding and Embedding the Python Interpreter manual in the Documentation Set and the sections on building extensions on non-Windows and Windows platforms and the existing python3.x-config script?

As it stands, it's not at all clear to me why and how I would use the LIBRARY variable. For example, on macOS Framework builds:

	LIBDIR = "/Library/Frameworks/Python.framework/Versions/3.13/lib"
	LIBPC = "/Library/Frameworks/Python.framework/Versions/3.13/lib/pkgconfig"
	LIBPL = "/Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/config-3.13-darwin"
	LIBPYTHON = ""
	LIBRARY = "libpython3.13.a"

# $LIBDIR
$ ls /Library/Frameworks/Python.framework/Versions/3.13/lib/libpython*
 /Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.dylib -> ../Python

# $LIBDIR/$LIBRARY
$ ls -l /Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.a
ls: /Library/Frameworks/Python.framework/Versions/3.13/lib/libpython3.13.a: No such file or directory

# $LIBPL
$ ls -l /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/config-3.13-darwin
 Makefile
 Setup
 Setup.bootstrap
 Setup.local
 Setup.stdlib
 __pycache__
 config.c
 config.c.in
 install-sh
 libpython3.13.a -> ../../../Python
 libpython3.13.dylib -> ../../../Python
 makesetup
 python-config.py
 python.o

# $LIBPL/$LIBRARY
$ ls -l /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/config-3.13-darwin/libpython3.13.a
 /Library/Frameworks/Python.framework/Versions/3.13/lib/python3.13/config-3.13-darwin/libpython3.13.a -> ../../../Python

[ronaldoussoren]

@FFY00, @ned-deily . How do we move forward with this? This is one of the failing tests for a framework build and I'd like to get us back at having no failing tests.

The PR helps with that, but as Ned notices it is not clear what the semantics of these sysconfig variables is meant to be. The current test expectations don't match the variables usage in the Makefile for a framework build, and that usage cannot be changed.

It would IMHO be better to specify a new interface for getting information needed for building/linking and implement that on top of the information we already have. That's something for a different issue though.

[FFY00]

Hey, sorry for the delay getting back to you, I have been taking a break for mental health reasons. I also moved house in the meantime, so things have been a bit chaotic, I am only now getting back to normal.

The PR helps with that, but as Ned notices it is not clear what the semantics of these sysconfig variables is meant to be. The current test expectations don't match the variables usage in the Makefile for a framework build, and that usage cannot be changed.

Yep, that is exactly the problem, there aren't really any properly defined semantics. The value of the variables here should be the value of the variables on the Makefile, but since that appears to not be the case from what you say, we should try to do whatever breaks less user code.

It would IMHO be better to specify a new interface for getting information needed for building/linking and implement that on top of the information we already have. That's something for a different issue though.

Yep, that is my plan (GH-103480). sysconfig.get_config_vars is a good "escape-hatch" mechanism, but shouldn't be the primary source of information. As it stands, if someone makes a change to the Makefile, that might very possibly break a lot of user code, which is kinda nuts 😅

[FFY00]

Looking into this deeper, I don't think we should change any of the variable values. Users that want to use LDLIBRARY as the dynamic library file will have to special case framework builds, which is probably what the users that have run into this issue have been doing anyway.

From configure.ac:

MacOSX framework builds need more magic. LDLIBRARY is the dynamic
library that we build, but we do not want to link against it (we
will find it with a -framework option). For this reason there is an
extra variable BLDLIBRARY against which Python and the extension
modules are linked, BLDLIBRARY. This is normally the same as
LDLIBRARY, but empty for MacOSX framework builds.

[ned-deily]

To move this forward, @smontanaro's PR has added a check to test_library for macOS framework builds and, for now, the test is skipped on framework builds. I would like to continue the discussion about the sysconfig values for framework builds elsewhere in the hopes of coming to a resolution prior to 3.13.0b1 and then updating this test. So for now, we can mark this issue as closed. Thanks everyone!