Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

terminal-emulators/kitty: Fix ShellCheck lints #148947

Merged
merged 1 commit into from
Jul 28, 2023

Conversation

l0b0
Copy link
Contributor

@l0b0 l0b0 commented Dec 6, 2021

Motivation for this change

#133088, @Artturin.

Things done
  • Built on platform(s)
    • x86_64-linux
    • aarch64-linux
    • x86_64-darwin
    • aarch64-darwin
  • For non-Linux: Is sandbox = true set in nix.conf? (See Nix manual)
  • Tested, as applicable:
  • Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
  • Tested basic functionality of all binary files (usually in ./result/bin/)
  • 22.05 Release Notes (or backporting 21.11 Release notes)
    • (Package updates) Added a release notes entry if the change is major or breaking
    • (Module updates) Added a release notes entry if the change is significant
    • (Module addition) Added a release notes entry if adding a new NixOS module
    • (Release notes changes) Ran nixos/doc/manual/md-to-db.sh to update generated release notes
  • Fits CONTRIBUTING.md.

@l0b0 l0b0 force-pushed the tests/verify-embedded-shell-scripts branch 2 times, most recently from 5902432 to 1fdf6fe Compare December 6, 2021 09:51
@l0b0 l0b0 marked this pull request as ready for review December 6, 2021 10:20
@l0b0 l0b0 requested a review from Ericson2314 as a code owner December 6, 2021 10:20
@ofborg ofborg bot added the 10.rebuild-darwin-stdenv This PR causes stdenv to rebuild label Dec 6, 2021
@ofborg ofborg bot requested review from tex, rvolosatovs and Luflosi December 6, 2021 11:14
@l0b0
Copy link
Contributor Author

l0b0 commented Dec 9, 2021

@Artturin I've created #149934 and others to introduce ShellCheck in a bunch of places. Hopefully at least some of them can be merged without fixing a bajillion build scripts.

@zowoq
Copy link
Contributor

zowoq commented Dec 9, 2021

This would be better as a hook
https://nixos.org/manual/nixpkgs/stable/#ssec-setup-hooks

Agreed.

shellcheck also needs to be pinned somehow (and consistently treewide, not per package), see #148845 where the package broke after we bumped shellcheck.

@l0b0
Copy link
Contributor Author

l0b0 commented Dec 10, 2021

shellcheck also needs to be pinned somehow (and consistently treewide, not per package), see #148845 where the package broke after we bumped shellcheck.

Shouldn't whoever bumps ShellCheck also fix the lints as part of the PR? That way the code is simpler. It just couples the ShellCheck update to the linting, which seems natural.

@l0b0
Copy link
Contributor Author

l0b0 commented Dec 10, 2021

This would be better as a hook https://nixos.org/manual/nixpkgs/stable/#ssec-setup-hooks

Some examples https://github.com/NixOS/nixpkgs/blob/master/pkgs/build-support/install-shell-files/default.nix

https://github.com/NixOS/nixpkgs/tree/master/pkgs/build-support/setup-hooks

https://github.com/NixOS/nixpkgs/search?q=makeSetupHook&type=

I don't really understand that documentation. I'm only up to part 33 of How to Learn Nix, and sentences like "Packages adding a hook should not hard code a specific hook, but rather choose a variable relative to how they are included." are still just gobbledygook to me.

Based on the example code I'm guessing I need something like this (thrown together):

pkgs/build-support/setup-hooks/shell-check.bash:

shellCheck() {
    shellcheck -- "$@"
}

pkgs/build-support/shell-check/default.nix:

{ makeSetupHook, tests }:

let
  setupHook = makeSetupHook { name = "shell-check"; } ../setup-hooks/shell-check.sh;
in

setupHook.overrideAttrs (oldAttrs: {
  passthru = (oldAttrs.passthru or {}) // {
    tests = tests.shell-check;
  };
})

and then I need to use shellCheck "$out/bin/foo" in checkPhase? Is that better than just shellcheck "$out/bin/foo" in checkPhase? Or, more likely, am I missing something important?

@zowoq
Copy link
Contributor

zowoq commented Dec 10, 2021

Shouldn't whoever bumps ShellCheck also fix the lints as part of the PR? That way the code is simpler. It just couples the ShellCheck update to the linting, which seems natural.

In theory, sure. In practice, no. Depends entirely on how many fixes (and rebuilds) are needed. Likely this check will just end up being disabled selectively or outright unless it can be introduced in way that isn't going to cause potential problems in the future.

Also shellcheck isn't a isolated package, it's part of a package set (haskellPackages) that gets bumped every week or two as a separate workflow on the haskell-updates branch. Adding shellcheck as a dependency everywhere will eventually disrupt that workflow due to to many rebuilds, etc.

@Ma27
Copy link
Member

Ma27 commented Dec 12, 2021

If we start doing this everywhere, we'll effectively require the Haskell toolchain including GHC (which is btw the largest package in e.g. my system closure with ~1.5G) in the build-closure for each package. Also, I fully agree with @zowoq, this will cause even more rebuilds:

Adding shellcheck as a dependency everywhere will eventually disrupt that workflow due to to many rebuilds, etc.

I don't really understand why this shouldn't be part of a review/CI process. In theory, we'll only need to check this once when a PR is filed with the build scripts, right?

@l0b0
Copy link
Contributor Author

l0b0 commented Dec 12, 2021

Adding shellcheck as a dependency everywhere will eventually disrupt that workflow due to to many rebuilds, etc.

I don't really understand why this shouldn't be part of a review/CI process. In theory, we'll only need to check this once when a PR is filed with the build scripts, right?

That sounds like an excellent compromise. The vast majority of variable substitutions in embedded shell scripts are just paths or very simple strings, so there's probably a low chance of any reasonable configuration change to introduce a lint to a dependent script.

How would you get to the various phase scripts from the outside, though? Are they all part of the output?

@stale stale bot added the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 12, 2022
@l0b0 l0b0 force-pushed the tests/verify-embedded-shell-scripts branch from 1fdf6fe to 9feffb4 Compare June 13, 2023 21:32
@stale stale bot removed the 2.status: stale https://github.com/NixOS/nixpkgs/blob/master/.github/STALE-BOT.md label Jun 13, 2023
@l0b0 l0b0 changed the title terminal-emulators/kitty: ShellCheck embedded scripts terminal-emulators/kitty: Fix ShellCheck lints Jun 13, 2023
@l0b0
Copy link
Contributor Author

l0b0 commented Jun 13, 2023

@Artturin @Ma27 @zowoq Thank you for the feedback! I've reduced the scope of this PR to just fix some ShellCheck lints.

@ofborg ofborg bot added the 10.rebuild-linux-stdenv This PR causes stdenv to rebuild label Jun 13, 2023
@ofborg ofborg bot requested a review from adamcstephens June 13, 2023 22:56
@l0b0 l0b0 force-pushed the tests/verify-embedded-shell-scripts branch from 9feffb4 to a866e51 Compare June 14, 2023 20:36
Copy link
Contributor

@adamcstephens adamcstephens left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

These changes are ok with me. I'm a bit concerned about tooling and editor support to maintain this moving forward, but otherwise should be safe to merge.

@l0b0
Copy link
Contributor Author

l0b0 commented Jul 25, 2023

@Artturin Would you be OK to review this?

@@ -180,9 +180,10 @@ buildPythonApplication rec {
'';

installPhase = ''
set -o nounset
Copy link
Member

@Artturin Artturin Jul 26, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/nix/store/zx2jzw5j0lxigaq34wmyfkcia3n0cvxi-install-shell-files/nix-support/setup-hook: line 100: $1: unbound variable

I don't think this is worth setting like this because if someone changes one of the hooks but doesn't build kitty then the breakage will go undiscovered. Until the channels are updates (like what seems to have happened here)

a better place to start adding these would be pkgs/test/

example

diff --git a/pkgs/test/install-shell-files/default.nix b/pkgs/test/install-shell-files/default.nix
index aef5acc1de6b..27363175cfc3 100644
--- a/pkgs/test/install-shell-files/default.nix
+++ b/pkgs/test/install-shell-files/default.nix
@@ -5,7 +5,7 @@ let
     runCommandLocal "install-shell-files--${name}" ({
       nativeBuildInputs = [ installShellFiles ];
       meta.platforms = lib.platforms.all;
-    } // env) buildCommand;
+    } // env) ("set -o nounset\n" + buildCommand);
 in
 
 recurseIntoAttrs {
$ nix build ".#tests.install-shell-files.install-completion"
this derivation will be built:
  /nix/store/bblpr8v64b153v0p842hv7fp2dvppy59-install-shell-files--install-completion.drv
building '/nix/store/bblpr8v64b153v0p842hv7fp2dvppy59-install-shell-files--install-completion.drv'...
install-shell-files> /nix/store/zx2jzw5j0lxigaq34wmyfkcia3n0cvxi-install-shell-files/nix-support/setup-hook: line 100: $1: unbound variable

Copy link
Contributor Author

@l0b0 l0b0 Jul 27, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've removed the set -o nounset line - adding it to a more central test file should probably go into a separate PR. Relatedly:

  • Isn't changing that line going to cause thousands of rebuilds?

  • There are several other useful strictness directives; should we try to introduce several/all of the following?

    set -o errexit -o noclobber -o nounset -o pipefail
    shopt -s failglob inherit_errexit

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Changing the tests won't cause many rebuilds but fixing the issue will of course.

set -eu,set -o pipefail, and shopt -s inherit_errexit are already set setup.sh but for them to be set during the build i guess they should be set in default-builder.sh

@l0b0 l0b0 force-pushed the tests/verify-embedded-shell-scripts branch from a866e51 to a9a032f Compare July 27, 2023 02:10
@l0b0 l0b0 requested a review from Artturin July 27, 2023 02:10
@ofborg ofborg bot requested a review from adamcstephens July 27, 2023 02:27
@Artturin Artturin merged commit 1b6b147 into NixOS:master Jul 28, 2023
@l0b0 l0b0 deleted the tests/verify-embedded-shell-scripts branch July 28, 2023 01:29
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants