docs: add build info (#878)

Adding a few things, mostly a docs update with more help debugging
builds.

Signed-off-by: Henry Schreiner <[email protected]>

.pre-commit-config.yaml

@@ -25,7 +25,7 @@ repos:
         exclude: "^tests"
 
   - repo: https://github.com/astral-sh/ruff-pre-commit
-    rev: v0.5.6
+    rev: v0.6.2
     hooks:
       - id: ruff
         args: ["--fix", "--show-fixes"]
@@ -50,8 +50,8 @@ repos:
       - id: cmake-format
         exclude: ^src/scikit_build_core/resources/find_python
 
-  - repo: https://github.com/pre-commit/mirrors-prettier
-    rev: "v4.0.0-alpha.8"
+  - repo: https://github.com/rbubley/mirrors-prettier
+    rev: "v3.3.3"
     hooks:
       - id: prettier
         types_or: [yaml, markdown, html, css, scss, javascript, json]
@@ -60,7 +60,7 @@ repos:
         stages: [manual]
 
   - repo: https://github.com/pre-commit/mirrors-mypy
-    rev: v1.11.1
+    rev: v1.11.2
     hooks:
       - id: mypy
         exclude: |
@@ -130,15 +130,20 @@ repos:
         additional_dependencies: [cogapp]
 
   - repo: https://github.com/henryiii/validate-pyproject-schema-store
-    rev: 2024.07.29
+    rev: 2024.08.26
     hooks:
       - id: validate-pyproject
 
   - repo: https://github.com/python-jsonschema/check-jsonschema
-    rev: 0.29.1
+    rev: 0.29.2
     hooks:
       - id: check-dependabot
       - id: check-github-workflows
       - id: check-readthedocs
       - id: check-metaschema
         files: \.schema\.json
+
+  - repo: https://github.com/scientific-python/cookie
+    rev: 2024.08.19
+    hooks:
+      - id: sp-repo-review

docs/build.md

@@ -1,8 +1,53 @@
 # Build procedure
 
+## Quickstart
+
+For any backend, you can make a SDist and then build a wheel from it with one
+command (choose your favorite way to run apps):
+
+````{tab} pipx
+
+```bash
+pipx run build
+```
+
+````
+
+````{tab} uv
+
+```bash
+uvx --from build pyproject-build --installer=uv
+```
+
+````
+
+````{tab} pip
+
+```bash
+pip install build
+python -m build
+```
+
+````
+
+You can then check the file contents:
+
+```bash
+tar -tf dist/*.tar.gz
+unzip -l dist/*.whl
+```
+
+The SDist should contain a copy of the repo with all the files you'll need (CI
+files and such are not required). And the wheel should look like the installed
+project with a few helper files.
+
+You can inspect any SDist or wheel on PyPI at <https://inspector.pypi.io>.
+
+## In-depth
+
 Modern Python build procedure is as follows:
 
-## SDist
+### SDist
 
 The SDist is a tarfile with all the code required to build the project, along
 with a little bit of metadata. To build an SDist, you use the `build` tool with
@@ -29,7 +74,16 @@ Without build isolation, you can build an SDist manually with
 This will produce an SDist in the `dist` directory. For any other backend,
 substitute the backend above.
 
-## Wheel
+#### File structure in the SDist
+
+Since you can build a wheel from the source or from the SDist, the structure
+should be identical to the source, though some files (like CI files) may be
+omitted. Files from git submodules should be included. It is best if the SDist
+can be installed without internet connection, but that's not always the case.
+
+There also is a `PKG-INFO` file with metadata in SDists.
+
+### Wheel
 
 The wheel is a zip file (ending in `.whl`) with the built code of the project,
 along with required metadata. There is no code that executes on install; it is a
@@ -69,7 +123,36 @@ without building a wheel, and editable versions of the wheel build. Editable
 "wheels" are temporary wheels that are only produced to immediately install and
 discard, and are expected to provide mechanisms to link back to the source code.
 
-## Installing
+#### File structure in the wheel
+
+The basic structure of the wheel is what will be extracted to site-packages.
+This means most of the files are usually in `<package-name>/...`, though if a
+top-level extension is present, then that could be something like
+`<package-name>.<platform-tag>.so`. There's also a
+`<package-name>-<package-version>.dist-info/` directory with various metadata
+files in it (`METADATA`, `WHEEL`, and `RECORD`), along with license files. There
+are a few other metadata files that could be here too, like `entry_points.txt`.
+
+There are also several directories that installers can extract to different locations,
+namely:
+
+* `<package-name>.data/scripts`: Goes to the `/bin` or `/Scripts` directory in
+  the environment. Any file starting with `#!python` will get the correct path
+  injected by the installer. Most build-backends (like setuptools and
+  scikit-build-core) will convert normal Python shabang lines like
+  `#!/usr/bin/env python` into `#!python` for you. Though if you are writing Python
+  and placing them here, it's usually better to use entry points and let the installer
+  generate the entire file.
+* `<package-name>.data/headers`: Goes to the include directory for the current
+  version of Python in the environment.
+* `<package-name>.data/data`: Goes to the root of the environment.
+
+Note that if a user is not in a virtual environment, these folders install
+directly to the Python install's location, which could be `/` or `/usr`!  In
+general, it's best to put data inside the package's folder in site-packages and
+then use `importlib.resources` to access it.
+
+### Installing
 
 Installing simply unpacks a wheel into the target filesystem. No code is run, no
 configuration files are present. If pip tries to install a repo or an SDist, it

pyproject.toml

@@ -217,7 +217,6 @@ ignore = ["W002"]  # Triggers on __init__.py's
 
 
 [tool.ruff]
-src = ["src"]
 exclude = ["src/scikit_build_core/_vendor"]  # Required due to "build" module
 
 [tool.ruff.lint]

tests/conftest.py

@@ -157,13 +157,13 @@ def install(self, *args: str, isolated: bool = True) -> None:
         self.module("pip", "install", *isolated_flags, *args)
 
 
-@pytest.fixture()
+@pytest.fixture
 def isolated(tmp_path: Path, pep518_wheelhouse: Path) -> VEnv:
     path = tmp_path / "venv"
     return VEnv(path, wheelhouse=pep518_wheelhouse)
 
 
-@pytest.fixture()
+@pytest.fixture
 def virtualenv(tmp_path: Path) -> VEnv:
     path = tmp_path / "venv"
     return VEnv(path)
@@ -205,7 +205,7 @@ def process_package(
         shutil.rmtree("dist")
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simple_pyproject_ext(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -220,7 +220,7 @@ def package_simple_pyproject_ext(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simple_pyproject_script_with_flags(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -231,7 +231,7 @@ def package_simple_pyproject_script_with_flags(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simple_pyproject_source_dir(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -242,7 +242,7 @@ def package_simple_pyproject_source_dir(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simple_setuptools_ext(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -251,7 +251,7 @@ def package_simple_setuptools_ext(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_mixed_setuptools(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -260,7 +260,7 @@ def package_mixed_setuptools(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_filepath_pure(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -269,7 +269,7 @@ def package_filepath_pure(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_dynamic_metadata(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -278,14 +278,14 @@ def package_dynamic_metadata(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_hatchling(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> PackageInfo:
     package = PackageInfo("hatchling")
     process_package(package, tmp_path, monkeypatch)
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simplest_c(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> PackageInfo:
     package = PackageInfo(
         "simplest_c",
@@ -294,7 +294,7 @@ def package_simplest_c(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Packa
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def navigate_editable(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> PackageInfo:
     package = PackageInfo(
         "navigate_editable",
@@ -303,7 +303,7 @@ def navigate_editable(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> Packag
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def broken_fallback(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> PackageInfo:
     package = PackageInfo(
         "broken_fallback",
@@ -312,7 +312,7 @@ def broken_fallback(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> PackageI
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_sdist_config(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -323,7 +323,7 @@ def package_sdist_config(
     return package
 
 
-@pytest.fixture()
+@pytest.fixture
 def package_simple_purelib_package(
     tmp_path: Path, monkeypatch: pytest.MonkeyPatch
 ) -> PackageInfo:
@@ -342,7 +342,7 @@ def which_mock(name: str) -> str | None:
     return None
 
 
-@pytest.fixture()
+@pytest.fixture
 def protect_get_requires(fp, monkeypatch):
     """
     Protect get_requires from actually calling anything variable during tests.

tests/test_broken_fallback.py

@@ -9,8 +9,8 @@
 )
 
 
-@pytest.mark.compile()
-@pytest.mark.configure()
+@pytest.mark.compile
+@pytest.mark.configure
 @pytest.mark.usefixtures("broken_fallback")
 @pytest.mark.parametrize("broken_define", ["BROKEN_CMAKE", "BROKEN_CODE"])
 def test_broken_code(broken_define: str, capfd: pytest.CaptureFixture[str]):

tests/test_cmake_config.py

@@ -71,7 +71,7 @@ def configure_args(
         yield f"-C{cmake_init}"
 
 
-@pytest.mark.configure()
+@pytest.mark.configure
 def test_init_cache(
     generator: str,
     tmp_path: Path,
@@ -116,7 +116,7 @@ def test_init_cache(
     )
 
 
-@pytest.mark.configure()
+@pytest.mark.configure
 def test_too_old(fp, monkeypatch):
     monkeypatch.setattr(shutil, "which", lambda _: None)
     fp.register(
@@ -133,7 +133,7 @@ def test_too_old(fp, monkeypatch):
     assert "Could not find CMake with version >=3.15" in excinfo.value.args[0]
 
 
-@pytest.mark.configure()
+@pytest.mark.configure
 def test_cmake_args(
     generator: str,
     tmp_path: Path,
@@ -167,7 +167,7 @@ def test_cmake_args(
     assert len(fp.calls) == 2
 
 
-@pytest.mark.configure()
+@pytest.mark.configure
 def test_cmake_paths(
     generator: str,
     tmp_path: Path,

tests/test_dynamic_metadata.py

@@ -91,7 +91,7 @@ def special_loader(name: str, *args: Any, **kwargs: Any) -> Any:
     return original_loader(name, *args, **kwargs)
 
 
-@pytest.fixture()
+@pytest.fixture
 def mock_entry_points(monkeypatch):
     monkeypatch.setattr(importlib, "import_module", special_loader)
 
@@ -229,8 +229,8 @@ def test_dual_metadata():
         get_standard_metadata(pyproject, settings)
 
 
-@pytest.mark.compile()
-@pytest.mark.configure()
+@pytest.mark.compile
+@pytest.mark.configure
 @pytest.mark.usefixtures("mock_entry_points", "package_dynamic_metadata")
 def test_pep517_wheel(virtualenv):
     dist = Path("dist")