Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ad-hoc version/local scheme substitution #455

Open
webknjaz opened this issue Jun 8, 2020 · 8 comments
Open

Ad-hoc version/local scheme substitution #455

webknjaz opened this issue Jun 8, 2020 · 8 comments

Comments

@webknjaz
Copy link
Member

webknjaz commented Jun 8, 2020

It is known that PyPI doesn't accept local version identifiers (#322)
so when uploading non-tagged versions to TestPyPI, I need to do something
like echo 'local_scheme = "no-local-version"' >> pyproject.toml which
effectively marks the tree dirty influencing the generated version.
I also want to keep the defaults in all other cases (everything except
for non-tagged commits in CI/CD).
Would it be reasonable to support some sort of an env var based solution
for overriding a local version scheme w/o touching the file system?

@RonnyPfannschmidt
Copy link
Contributor

this one needs some thought wrt howto integrate it sanely

@webknjaz
Copy link
Member Author

I guess the immediate workaround would be git update-index --assume-unchanged pyproject.toml in CI.

webknjaz added a commit to ansible/pylibssh that referenced this issue Jun 10, 2020
@RonnyPfannschmidt
Copy link
Contributor

@webknjaz im considering adding a SETUPTOOLS_SCM_CONFIG_OVERRIDES_FOR_{package-name.capitalize().replace("-","_")] env var containing json

any opinion?

@webknjaz
Copy link
Member Author

That's exactly what I was thinking about.

@bilke
Copy link

bilke commented Oct 5, 2022

By using the (deprecated) setup.py-approach you can implement this via environment variables, e.g.:

# setuptools_scm config local_scheme via env var SETUPTOOLS_SCM_LOCAL_SCHEME:
scm_local_scheme = "node-and-date"
if "SETUPTOOLS_SCM_LOCAL_SCHEME" in os.environ:
    local_scheme_values = [
        "node-and-date",
        "node-and-timestamp",
        "dirty-tag",
        "no-local-version",
    ]
    if os.environ["SETUPTOOLS_SCM_LOCAL_SCHEME"] in local_scheme_values:
        scm_local_scheme = os.environ["SETUPTOOLS_SCM_LOCAL_SCHEME"]
    #else:
         # print some warning

setup(
    ...
    use_scm_version={
        "local_scheme": scm_local_scheme,
    },
)

@webknjaz

This comment was marked as resolved.

webknjaz added a commit to webknjaz/pylibssh that referenced this issue Nov 8, 2022
pypa/setuptools-scm#455 (comment)
turned out to be a bad idea and didn't work as expected.
webknjaz added a commit to webknjaz/pylibssh that referenced this issue Nov 8, 2022
pypa/setuptools-scm#455 (comment)
turned out to be a bad idea and didn't work as expected.
wmayner added a commit to wmayner/pyemd that referenced this issue Feb 20, 2023
Currently it seems there's no way to disable local versions without
modifying tracked files; see
pypa/setuptools-scm#455. For now we just forgo
them, but in the future it'd be better to use them to avoid accidental
uploading of non-release code.
gmgunter added a commit to gmgunter/snaphu-py that referenced this issue Jan 18, 2024
By default, setuptools_scm applies "local version identifiers" to
non-tag versions [^1]. In practice, this means appending the short git
hash as a suffix to the version string.

However, TestPyPI doesn't allow publishing distributions that include
local version labels [^2][^3][^4]. This is intended to prevent
accidental upload of non-release versions, but it makes it a bit more
complicated to test our CD pipeline.

We could try to work around this by spoofing the version using
`SETUPTOOLS_SCM_PRETEND_VERSION`[^5]. Alternatively, we could simply
disable local version identifiers in the setuptools_scm configuration.

Including the git hash in the version string of non-tag builds seems
very useful, but handling the version spoofing in CI seems much more
complicated than I'm willing to tolerate.

So for now we'll forego the local version labels, but in the future it'd
be nice to reenable them and adopt a better workaround.

[^1]: https://packaging.python.org/en/latest/specifications/version-specifiers/#local-version-identifiers
[^2]: pypa/setuptools-scm#322
[^3]: pypa/setuptools-scm#455
[^4]: pypa/setuptools-scm#478
[^5]: https://setuptools-scm.readthedocs.io/en/latest/overrides/
gmgunter added a commit to isce-framework/snaphu-py that referenced this issue Jan 18, 2024
* Build wheels & sdists in CI

Add a CI job that runs on pull requests and pushes to main and executes
the following steps:

* Build wheels for Linux & macOS x86_64, as well as Linux i686 and macOS
  ARM64 using cibuildwheel
* Run the test suite using generated wheels (except for macOS ARM64,
  which is cross-compiled)
* Lint wheels using check-wheel-contents
* Build an sdist and lint it using `twine check`
* Publish wheels & sdist to TestPyPI

* Fix TestPyPI URL

The publisher name & URL must match the project name in pyproject.toml.
I was hoping to name the Python module "snaphu" but name the PyPI
package "snaphu-py", but it looks like that's not possible (at least not
possible with this method of publishing distributions).

https://docs.pypi.org/trusted-publishers/troubleshooting/#upload-errors

* Remove local version identifiers

By default, setuptools_scm applies "local version identifiers" to
non-tag versions [^1]. In practice, this means appending the short git
hash as a suffix to the version string.

However, TestPyPI doesn't allow publishing distributions that include
local version labels [^2][^3][^4]. This is intended to prevent
accidental upload of non-release versions, but it makes it a bit more
complicated to test our CD pipeline.

We could try to work around this by spoofing the version using
`SETUPTOOLS_SCM_PRETEND_VERSION`[^5]. Alternatively, we could simply
disable local version identifiers in the setuptools_scm configuration.

Including the git hash in the version string of non-tag builds seems
very useful, but handling the version spoofing in CI seems much more
complicated than I'm willing to tolerate.

So for now we'll forego the local version labels, but in the future it'd
be nice to reenable them and adopt a better workaround.

[^1]: https://packaging.python.org/en/latest/specifications/version-specifiers/#local-version-identifiers
[^2]: pypa/setuptools-scm#322
[^3]: pypa/setuptools-scm#455
[^4]: pypa/setuptools-scm#478
[^5]: https://setuptools-scm.readthedocs.io/en/latest/overrides/

* Don't build wheels or publish to TestPyPI for PRs

Instead, these CI jobs should only be triggered by pushes to upstream
branches.

Building wheels is very slow compared to the other CI jobs (up to an
hour for 32-bit Linux wheels vs 1-2 minutes for the basic build&test CI
pipeline).

Furthermore, publishing to TestPyPI via OIDC seems to be impossible in
pull requests from public forks. The maximum allowed `id-token`
permissions for public forks is `read` [^1], whereas publishing via OIDC
requires `id-token: write` permissions [^2].

[^1]:
https://docs.github.com/en/actions/security-guides/automatic-token-authentication#permissions-for-the-github_token
[^2]:
https://docs.github.com/en/actions/deployment/security-hardening-your-deployments/about-security-hardening-with-openid-connect#adding-permissions-settings
@webknjaz
Copy link
Member Author

By using the (deprecated) setup.py

FTR setup.py is not deprecated. Calling it directly is.

@webknjaz
Copy link
Member Author

webknjaz commented Apr 19, 2024

UPD:

echo 'local_scheme = "no-local-version"' >> pyproject.toml

Looks like this stopped working with modern setuptools versions :(

Exactly what #686 (comment) says. I wonder why nobody filed an issue for this...

UPD 2: Nevermind, it was a combination of newer setuptools (v69) and older setuptools-scm (v7). Upgrading it to v8 fixed the problem.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

3 participants