Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

regression: uv build silently produces empty wheels with hatchling if .gitignore is missing #8200

Closed
jinnatar opened this issue Oct 15, 2024 · 9 comments · Fixed by #8220
Closed
Assignees
Labels
bug Something isn't working great writeup A wonderful example of a quality contribution 💜

Comments

@jinnatar
Copy link

jinnatar commented Oct 15, 2024

Summary:

Newer versions of uv build "empty" wheels under the following conditions with hatchling.build when depending on hatch default logic of finding the sources at src/$project/__init__.py.

  • Last version where it works: 0.4.17
  • First version that fails: 0.4.18
  • The trigger seems to be whether the project root has a file .gitignore or not, where it being missing triggers this issue!

Minimal pyproject.toml:

[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

[project]
name = "demo"
version = "0.1.1"
description = ""
requires-python = ">=3.11"
dependencies = []

[project.scripts]
demo = 'demo:run'

Source contents is only one file at src/demo/__init__.py:

def run():
    print("Running like the wind!")

Dockerfile for repro:

# Fails with 0.4.18
FROM ghcr.io/astral-sh/uv:0.4.18-python3.12-alpine AS builder
# Succeeds with 0.4.17
#FROM ghcr.io/astral-sh/uv:0.4.17-python3.12-alpine AS builder

RUN mkdir /build
WORKDIR /build

COPY pyproject.toml uv.lock ./
COPY src ./src
RUN uv build

CMD ["unzip", "-l", "dist/demo-0.1.1-py3-none-any.whl"]

Expected output:

Archive:  dist/demo-0.1.1-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
       47  02-02-2020 00:00   demo/__init__.py
       72  02-02-2020 00:00   demo-0.1.1.dist-info/METADATA
       87  02-02-2020 00:00   demo-0.1.1.dist-info/WHEEL
       34  02-02-2020 00:00   demo-0.1.1.dist-info/entry_points.txt
      358  02-02-2020 00:00   demo-0.1.1.dist-info/RECORD 
 --------                     -------                  
      598                     5 files

With newer versions instead we get:

Archive:  dist/demo-0.1.1-py3-none-any.whl
  Length      Date    Time    Name
---------  ---------- -----   ----
       72  02-02-2020 00:00   demo-0.1.1.dist-info/METADATA
       87  02-02-2020 00:00   demo-0.1.1.dist-info/WHEEL
       34  02-02-2020 00:00   demo-0.1.1.dist-info/entry_points.txt
      287  02-02-2020 00:00   demo-0.1.1.dist-info/RECORD
 --------                     -------
      480                     4 files
  • The key diff is that the demo package itself is missing, we only get the dist-info, which while technically a valid wheel, doesn't do anything useful either.
@jinnatar
Copy link
Author

What's confusing here is that in my local dev env, uv build with 0.4.21 produces a valid wheel with src contents. So not sure what bit in the Docker env minimalism makes it trip.

@jinnatar
Copy link
Author

No one will believe me, BUT, the presence of a .gitignore file in the build context, even if empty, makes it work. If it's not there, as it's not in my minimal Dockerfile, it fails.

@jinnatar jinnatar changed the title regression: uv build silently produces empty wheels with hatchling regression: uv build silently produces empty wheels with hatchling if .gitignore is missing Oct 15, 2024
@jinnatar
Copy link
Author

No useful verbose logging diff between the failing and working builds:

32d31
< DEBUG Found fresh response for: https://files.pythonhosted.org/packages/88/5f/e351af9a41f866ac3f1fac4ca0613908d9a41741cfcf2228f4ad853b697d/pluggy-1.5.0-py3-none-any.whl.metadata
33a33
> DEBUG Found fresh response for: https://files.pythonhosted.org/packages/88/5f/e351af9a41f866ac3f1fac4ca0613908d9a41741cfcf2228f4ad853b697d/pluggy-1.5.0-py3-none-any.whl.metadata
41c41
< DEBUG Installing in hatchling==1.25.0, packaging==24.1, pathspec==0.12.1, pluggy==1.5.0, trove-classifiers==2024.10.13 in /home/user/.cache/uv/builds-v0/.tmpnuqDDO
---
> DEBUG Installing in hatchling==1.25.0, packaging==24.1, pathspec==0.12.1, pluggy==1.5.0, trove-classifiers==2024.10.13 in /home/user/.cache/uv/builds-v0/.tmpE02Jyf
50c50
< DEBUG Calling `hatchling.build.build_sdist("/vol/home/src/uv-repro.git/dist/.tmpigoeaQ", {})`
---
> DEBUG Calling `hatchling.build.build_sdist("/vol/home/src/uv-repro.git/dist/.tmpmpoSRe", {})`
72,73c72,73
< DEBUG Split specific environment resolution took 0.000s
< DEBUG Installing in hatchling==1.25.0, packaging==24.1, pathspec==0.12.1, pluggy==1.5.0, trove-classifiers==2024.10.13 in /home/user/.cache/uv/builds-v0/.tmpsS489a
---
> DEBUG Split specific environment resolution took 0.001s
> DEBUG Installing in hatchling==1.25.0, packaging==24.1, pathspec==0.12.1, pluggy==1.5.0, trove-classifiers==2024.10.13 in /home/user/.cache/uv/builds-v0/.tmptTLjxm
82c82
< DEBUG Calling `hatchling.build.build_wheel("/vol/home/src/uv-repro.git/dist/.tmpUJojVo", {}, None)`
---
> DEBUG Calling `hatchling.build.build_wheel("/vol/home/src/uv-repro.git/dist/.tmp3We1rG", {}, None)`

@jinnatar
Copy link
Author

My best guess is, it's something to do with #7835 @konstin does this make any sense to you as the source of the regression?

@charliermarsh
Copy link
Member

Thanks @jinnatar -- we're taking a look!

@charliermarsh
Copy link
Member

(@konstin has been debugging to try and understand why hatchling is behaving this way.)

@charliermarsh charliermarsh added the bug Something isn't working label Oct 15, 2024
@zanieb
Copy link
Member

zanieb commented Oct 15, 2024

Related pypa/hatch#1273

@jinnatar
Copy link
Author

Tried a uvx hatch build with and without a project level .gitignore, the issue doesn't repro there. So possible hatch does some workarounding that invoking hatchling directly does not?

konstin added a commit that referenced this issue Oct 15, 2024
When building a source distribution to a wheels, we perform the build inside a temporary directory inside the output directory. By default, the output directory is `dist/` in the repository root. This temp dir placement allows us to move the final wheel to the output directory instead of copying it (a temp dir might be on another device, which means we need to copy instead of moving).

Some build backends such as hatchling traverse upwards from the current directory (the source dist build location) looking for gitignore files to consider. By adding a gitignore in `dist/` with `*`, we caused hatchling to ignore all files in our temporary build directory below it, causing empty wheels. To prevent this, we add a `.git` file as a phony git root. We are already using this trick successfully in the cache. Hatchling sees this `.git` file, considers it a boundary and does not traverse up to `dist/.gitignore`.

I'd appreciate an eye on potential side effects of adding a `.git` here that I've missed.

Fixes #8200
konstin added a commit that referenced this issue Oct 15, 2024
When building a source distribution to a wheels, we perform the build inside a temporary directory inside the output directory. By default, the output directory is `dist/` in the repository root. This temp dir placement allows us to move the final wheel to the output directory instead of copying it (a temp dir might be on another device, which means we need to copy instead of moving).

Some build backends such as hatchling traverse upwards from the current directory (the source dist build location) looking for gitignore files to consider. By adding a gitignore in `dist/` with `*`, we caused hatchling to ignore all files in our temporary build directory below it, causing empty wheels. To prevent this, we add a `.git` file as a phony git root. We are already using this trick successfully in the cache. Hatchling sees this `.git` file, considers it a boundary and does not traverse up to `dist/.gitignore`.

I'd appreciate an eye on potential side effects of adding a `.git` here that I've missed.

Fixes #8200
konstin added a commit that referenced this issue Oct 15, 2024
When building a source distribution to a wheels, we perform the build inside a temporary directory inside the output directory. By default, the output directory is `dist/` in the repository root. This temp dir placement allows us to move the final wheel to the output directory instead of copying it (a temp dir might be on another device, which means we need to copy instead of moving).

Some build backends such as hatchling traverse upwards from the current directory (the source dist build location) looking for gitignore files to consider. By adding a gitignore in `dist/` with `*`, we caused hatchling to ignore all files in our temporary build directory below it, causing empty wheels. To prevent this, we add a `.git` file as a phony git root. We are already using this trick successfully in the cache. Hatchling sees this `.git` file, considers it a boundary and does not traverse up to `dist/.gitignore`.

I'd appreciate an eye on potential side effects of adding a `.git` here that I've missed.

Fixes #8200
@konstin konstin added the great writeup A wonderful example of a quality contribution 💜 label Oct 15, 2024
@konstin
Copy link
Member

konstin commented Oct 15, 2024

I've added a fix and an explanation in #8220, it's a chain of circumstances adding together.

Thanks for the great write-up and reproducer, they were very helpful for debugging.

charliermarsh added a commit that referenced this issue Oct 15, 2024
When building a source distribution to a wheels, we perform the build
inside a temporary directory inside the output directory. By default,
the output directory is `dist/` in the repository root. This temp dir
placement allows us to move the final wheel to the output directory
instead of copying it (a temp dir might be on another device, which
means we need to copy instead of moving).

Some build backends such as hatchling traverse upwards from the current
directory (the source dist build location) looking for gitignore files
to consider. By adding a gitignore in `dist/` with `*`, we caused
hatchling to ignore all files in our temporary build directory below it,
causing empty wheels. To prevent this, we add a `.git` file as a phony
git root. We are already using this trick successfully in the cache.
Hatchling sees this `.git` file, considers it a boundary and does not
traverse up to `dist/.gitignore`.

Fixes #8200

---------

Co-authored-by: Charlie Marsh <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working great writeup A wonderful example of a quality contribution 💜
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants