Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tree-wide: Replace unsupported dontCheck with doCheck attribute #307838

Merged
merged 3 commits into from
Jun 2, 2024

Conversation

afh
Copy link
Member

@afh afh commented Apr 29, 2024

Description of changes

Remove unsupported dontCheck attribute (for context see this comment on #293498).

ℹ️ Since python3Module.gpt-2-simple is broken on aarch64-darwin I was not able to test it locally; yet the GitHub tests indicate that the changes proposed in this PR do not affect it negatively.

Things done

  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
    • sandbox = relaxed
    • sandbox = true
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
  • Fits CONTRIBUTING.md.

Add a 👍 reaction to pull requests you find important.

@AndersonTorres
Copy link
Member

I believe this is a serious inconsistency in Nixpkgs.
Remembering if we should set doPhase instead of dontPhase is a bit silly.

@afh
Copy link
Member Author

afh commented Apr 29, 2024

In your mind should both be supported, e.g. one being a "negating alias" of the other?

@AndersonTorres
Copy link
Member

In my mind only the doPhase should be supported, in order to avoid (more) confusion.

But, in the name of backwards compatibility, your idea of "negating alias" is good.

@afh
Copy link
Member Author

afh commented Apr 29, 2024

If doCheck and dontCheck are both supported and dontCheck is a "negating alias" of doCheck, should doCheck take precedence over dontCheck if both are present in a derivation?

@afh
Copy link
Member Author

afh commented Apr 29, 2024

Reading through make-derivation.nix I see this TODO-comment that hints at doCheck being old semantics. Do you have any context on this you could share, @AndersonTorres? I'm wondering what the "new semantics" might be…

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 30, 2024

I have few to nil idea.
It was introduced at this commit below:

b4cc2a2

Maybe @Ericson2314 should know.

@AndersonTorres
Copy link
Member

AndersonTorres commented Apr 30, 2024

If doCheck and dontCheck are both supported and dontCheck is a "negating alias" of doCheck, should doCheck take precedence over dontCheck if both are present in a derivation?

The most typical variant is dontPhase. So, we should have something like

  • dontCheck to control if checkPhase will be executed
    • default true, since checks are not enabled by default
  • if doCheck is set either as true or false, emit a warning and set dontCheck = (!doCheck)

The same for all the other doPhase variables.

@afh afh marked this pull request as ready for review April 30, 2024 15:37
@afh
Copy link
Member Author

afh commented Apr 30, 2024

Kindly requesting official review from @ckiee and @TomaSajt for the changes in the packages they maintain.
Unfortunately kaitastruct is unmaintained; if any of you would be willing to review the changes too, that would be very much appreciated 🙏

Copy link
Contributor

@TomaSajt TomaSajt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@afh
Copy link
Member Author

afh commented Apr 30, 2024

I found another interesting tidbit in relation to doPhase and dontPhase, @AndersonTorres: the runPhase shell function in pkgs/stdenv/generic/setup.sh:1551 provides a good overview of what is used and how.

I'd like to look into creating a "negating alias" for all the do* variables to allow for the possibility phase them out at some point in the (far) future. Any objections?

@AndersonTorres
Copy link
Member

No objections, only recommendations!

@afh afh force-pushed the remove-dontCheck branch from 96a9a91 to 3611c45 Compare May 1, 2024 05:44
@ofborg ofborg bot requested review from ckiee and TomaSajt May 1, 2024 06:16
Copy link
Contributor

@TomaSajt TomaSajt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like my wording was not clear, here are the changes I've described.

diff --git a/pkgs/development/python-modules/gpt-2-simple/default.nix b/pkgs/development/python-modules/gpt-2-simple/default.nix
index aae6dfbfbc40..ef19eb0ba1bb 100644
--- a/pkgs/development/python-modules/gpt-2-simple/default.nix
+++ b/pkgs/development/python-modules/gpt-2-simple/default.nix
@@ -1,10 +1,10 @@
-{ lib, buildPythonPackage, fetchFromGitHub, regex, requests, tqdm, numpy
+{ lib, buildPythonPackage, fetchFromGitHub, setuptools, regex, requests, tqdm, numpy
 , toposort, tensorflow }:
 
 buildPythonPackage rec {
   pname = "gpt-2-simple";
   version = "0.8.1";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchFromGitHub {
     owner = "minimaxir";
@@ -13,9 +13,9 @@ buildPythonPackage rec {
     hash = "sha256-WwD4sDcc28zXEOISJsq8e+rgaNrrgIy79Wa4J3E7Ovc=";
   };
 
-  propagatedBuildInputs = [ regex requests tqdm numpy toposort tensorflow ];
+  build-system = [ setuptools ];
 
-  dontCheck = true; # no tests in upstream
+  dependencies = [ regex requests tqdm numpy toposort tensorflow ];
 
   meta = with lib; {
     description =
diff --git a/pkgs/development/python-modules/kaitaistruct/default.nix b/pkgs/development/python-modules/kaitaistruct/default.nix
index 71ad13fadb55..6d30a4e03dd4 100644
--- a/pkgs/development/python-modules/kaitaistruct/default.nix
+++ b/pkgs/development/python-modules/kaitaistruct/default.nix
@@ -2,6 +2,7 @@
 , buildPythonPackage
 , fetchPypi
 , fetchFromGitHub
+, setuptools
 , brotli
 , lz4
 }:
@@ -17,7 +18,7 @@ in
 buildPythonPackage rec {
   pname = "kaitaistruct";
   version = "0.10";
-  format = "setuptools";
+  pyproject = true;
 
   src = fetchPypi {
     inherit pname version;
@@ -29,14 +30,13 @@ buildPythonPackage rec {
     sed '32ipackages = kaitai/compress' -i setup.cfg
   '';
 
-  propagatedBuildInputs = [
+  build-system = [ setuptools ];
+
+  dependencies = [
     brotli
     lz4
   ];
 
-  # no tests
-  dontCheck = true;
-
   pythonImportsCheck = [
     "kaitaistruct"
     "kaitai.compress"

This works, because when using pyproject = true setuptoolsCheckHook is not automatically added, so testing isn't enabled.
I also recommend changing the naming of the dependency lists to the newer ones, like shown in the diff.

@ofborg ofborg bot requested a review from TomaSajt May 4, 2024 17:39
@afh afh force-pushed the remove-dontCheck branch from 42847f7 to d81ae4a Compare May 4, 2024 19:42
@afh
Copy link
Member Author

afh commented May 4, 2024

Thank you for your feedback, @TomaSajt, very much appreciated. I may not have read your previous comment as thorough as I should have before making more changes to this PR. Hopefully I've now paid closer attention—although it is getting a bit late in the evening for me 🥱

@afh afh requested a review from ShamrockLee May 16, 2024 17:18
@afh
Copy link
Member Author

afh commented May 16, 2024

Thanks for reaching out to the python matrix chat, @TomaSajt, hopefully I've understood you correctly (see latest changes)

Copy link
Contributor

@TomaSajt TomaSajt left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@ofborg ofborg bot requested a review from TomaSajt May 16, 2024 19:34
@afh afh added the 12.approvals: 2 This PR was reviewed and approved by two reputable people label May 16, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1643

@afh afh force-pushed the remove-dontCheck branch from 0b79ad2 to 9ff8470 Compare May 16, 2024 21:01
@wegank wegank removed the 12.approvals: 2 This PR was reviewed and approved by two reputable people label May 17, 2024
@AndersonTorres
Copy link
Member

Personal opinion:
Since bisect is a thing, the packages that received changes besides removing dontCheck should be separated in their own commits (not necessarily PRs).

@ShamrockLee
Copy link
Contributor

I have too much on my plate and cannot review these changes this week.

@afh afh force-pushed the remove-dontCheck branch from 9ff8470 to 7eccf82 Compare May 18, 2024 17:11
@afh
Copy link
Member Author

afh commented May 18, 2024

I've created separate commits as preferred, @AndersonTorres.

Thanks for the heads up, @ShamrockLee.

@afh afh force-pushed the remove-dontCheck branch from 7eccf82 to ff2e2d3 Compare May 19, 2024 02:25
@afh afh changed the title tree-wide: Remove unsupported dontCheck attribute tree-wide: Replace unsupported dontCheck with doCheck attribute May 19, 2024
@wegank wegank added 12.approvals: 2 This PR was reviewed and approved by two reputable people 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in the package 2.status: merge conflict This PR has merge conflicts with the target branch labels May 19, 2024
afh added 3 commits May 29, 2024 08:23
and replace unsupported dontCheck with doCheck attribute
and replace unsupported dontCheck with doCheck attribute
@afh afh force-pushed the remove-dontCheck branch from ff2e2d3 to a3deb72 Compare May 29, 2024 06:28
@ofborg ofborg bot removed the 2.status: merge conflict This PR has merge conflicts with the target branch label May 29, 2024
@nixos-discourse
Copy link

This pull request has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/prs-already-reviewed/2617/1701

@lelgenio
Copy link
Contributor

Result of nixpkgs-review pr 307838 run on x86_64-linux 1

28 packages built:
  • mitmproxy (python311Packages.mitmproxy)
  • mitmproxy.dist (python311Packages.mitmproxy.dist)
  • mitmproxy2swagger
  • mitmproxy2swagger.dist
  • python311Packages.gpt-2-simple
  • python311Packages.gpt-2-simple.dist
  • python311Packages.kaitaistruct
  • python311Packages.kaitaistruct.dist
  • python311Packages.roadtools
  • python311Packages.roadtools.dist
  • python311Packages.roadtx
  • python311Packages.roadtx.dist
  • python311Packages.selenium-wire
  • python311Packages.selenium-wire.dist
  • python312Packages.kaitaistruct
  • python312Packages.kaitaistruct.dist
  • python312Packages.mitmproxy
  • python312Packages.mitmproxy.dist
  • python312Packages.roadtools
  • python312Packages.roadtools.dist
  • python312Packages.roadtx
  • python312Packages.roadtx.dist
  • python312Packages.selenium-wire
  • python312Packages.selenium-wire.dist
  • pyxel
  • pyxel.dist
  • wapiti
  • wapiti.dist

@SuperSandro2000 SuperSandro2000 merged commit b2a8225 into NixOS:master Jun 2, 2024
33 checks passed
@afh afh deleted the remove-dontCheck branch June 2, 2024 16:51
@afh
Copy link
Member Author

afh commented Jun 2, 2024

Thanks everyone for your helpful feedback on this and helping getting it merged, very much appreciated 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
6.topic: python 10.rebuild-darwin: 11-100 10.rebuild-linux: 11-100 12.approvals: 2 This PR was reviewed and approved by two reputable people 12.approved-by: package-maintainer This PR was reviewed and approved by a maintainer listed in the package
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants