Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

🌱 Add Dockerfile linter #6886

Merged
merged 1 commit into from
Jul 26, 2022

Conversation

oscr
Copy link
Contributor

@oscr oscr commented Jul 10, 2022

What this PR does / why we need it:

Adds a Dockerfile linter to the make lint target. This is done via a script which finds all files named "Dockerfile" and runs Hadolint on them. I've configured it to only fail on the highest severity.

The new output is:

 make lint
(...)
./scripts/ci-lint-dockerfiles.sh
Linting: cmd/clusterctl/Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly
Linting: Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly
Linting: docs/Dockerfile
-:34 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
-:35 DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`.Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
Linting: scripts/Dockerfile
-:1 DL3006 warning: Always tag the version of an image explicitly
Linting: test/extension/Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly
Linting: test/infrastructure/docker/Dockerfile
-:19 DL3006 warning: Always tag the version of an image explicitly
-:28 SC2086 info: Double quote to prevent globbing and word splitting.
-:61 DL3007 warning: Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag

If you want to verify a failure, here is a failing Dockerfile that you can add:

FROM ubuntu

RUN sudo ls

And verify exitcode with echo $?

@k8s-ci-robot k8s-ci-robot added the cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. label Jul 10, 2022
@k8s-ci-robot k8s-ci-robot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Jul 10, 2022
Copy link
Contributor

@killianmuldoon killianmuldoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall I like Hadolint, and I like this script - but I don't know how useful it is to introduce into CI because our docker files are more dynamic than Hadolint's static analysis is properly able to deal with.

For example Hadolint marks as invalid, with a warning, our lines like:

FROM ${builder_image} as builder

as invalid as we don't specify an image version. In reality we do set an image version, but in the Makefile.

That said there's currently no results of error type in the repo - just warnings and info - so this linter does work and pass, and I'd expect it to continue to do so.

Overall hadolint is not as useful as it could be, but should help us catch major problems with the Dockerfiles so I'm +1 to the change.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 11, 2022
@killianmuldoon
Copy link
Contributor

Looking at this again I think it would be possible to fail on warnings and get a more useful linting signal from this byadding the nolint ignore directive (# hadolint ignore=DL3006) above all the 3006 warnings that occur where we're using FROM ${builder_image} as builder

I'm currently getting:

Linting: cmd/clusterctl/Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly
Linting: Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly
Linting: docs/Dockerfile
-:34 DL3008 warning: Pin versions in apt get install. Instead of `apt-get install <package>` use `apt-get install <package>=<version>`
-:35 DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`.Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
Linting: test/infrastructure/docker/Dockerfile
-:19 DL3006 warning: Always tag the version of an image explicitly
-:28 SC2086 info: Double quote to prevent globbing and word splitting.
-:61 DL3007 warning: Using latest is prone to errors if the image will ever update. Pin the version explicitly to a release tag
Linting: test/extension/Dockerfile
-:20 DL3006 warning: Always tag the version of an image explicitly

So once the build-args 3006 issues are no-linted there's only a couple of small changes needed to bring out Dockerfiles in line with what hadolint expects - @oscr WDYT?

@oscr oscr force-pushed the add-dockerfile-lint branch from 5277aa6 to ec80136 Compare July 11, 2022 11:28
@k8s-ci-robot k8s-ci-robot added size/M Denotes a PR that changes 30-99 lines, ignoring generated files. and removed lgtm "Looks good to me", indicates that a PR is ready to be merged. size/S Denotes a PR that changes 10-29 lines, ignoring generated files. labels Jul 11, 2022
@oscr
Copy link
Contributor Author

oscr commented Jul 11, 2022

@killianmuldoon Excellent suggestions! Thank you for the feedback. It's added now 😄

scripts/ci-lint-dockerfiles.sh Outdated Show resolved Hide resolved
test/infrastructure/docker/Dockerfile Show resolved Hide resolved
scripts/ci-lint-dockerfiles.sh Outdated Show resolved Hide resolved
@oscr oscr force-pushed the add-dockerfile-lint branch from ec80136 to e862609 Compare July 11, 2022 11:58
@oscr
Copy link
Contributor Author

oscr commented Jul 11, 2022

/test pull-cluster-api-e2e-informing-main

@oscr oscr force-pushed the add-dockerfile-lint branch from e862609 to 9cca44b Compare July 11, 2022 13:10
Copy link
Contributor

@killianmuldoon killianmuldoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Awesome work!

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 11, 2022
@oscr
Copy link
Contributor Author

oscr commented Jul 11, 2022

Thank you @killianmuldoon for the great feedback!

@oscr
Copy link
Contributor Author

oscr commented Jul 11, 2022

/test pull-cluster-api-e2e-informing-main

@oscr oscr force-pushed the add-dockerfile-lint branch from 9cca44b to 0c66bcf Compare July 13, 2022 14:54
@k8s-ci-robot k8s-ci-robot removed the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 13, 2022
@oscr
Copy link
Contributor Author

oscr commented Jul 13, 2022

I have updated the linting script to take optional arguments HADOLINT_VER and HADOLINT_FAILURE_THRESHOLD. If these are unspecified (e g running the script directly) they will have default values latest and warning.

make lint
(...)
./scripts/ci-lint-dockerfiles.sh v2.10.0 warning
Linting: cmd/clusterctl/Dockerfile
Linting: Dockerfile
Linting: docs/Dockerfile
-:35 DL3047 info: Avoid use of wget without progress bar. Use `wget --progress=dot:giga <url>`.Or consider using `-q` or `-nv` (shorthands for `--quiet` or `--no-verbose`).
Linting: test/infrastructure/docker/Dockerfile
-:33 SC2086 info: Double quote to prevent globbing and word splitting.
Linting: test/extension/Dockerfile

@oscr oscr requested review from vincepri and killianmuldoon July 15, 2022 08:28
Copy link
Contributor

@killianmuldoon killianmuldoon left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

/lgtm

@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Jul 21, 2022
Comment on lines +21 to +24
# Ignore Hadolint rule "Always tag the version of an image explicitly."
# It's an invalid finding since the image is explicitly set in the Makefile.
# https://github.com/hadolint/hadolint/wiki/DL3006
# hadolint ignore=DL3006
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can this be ignored globally instead of having it in each Dockerfile?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@vincepri Yes, it is possible to have global ignore rules. In an earlier review it was requested per file: #6886 (comment) I have no strong opinion here so whatever the community thinks is best I can implement.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I still think it's more explicit to do it where we need so we catch this if it's changed in e.g. a new Dockerfile.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can see that, that's completely fine, it was just noisy when reviewing the PR

@vincepri
Copy link
Member

/approve

@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: vincepri

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Jul 26, 2022
@k8s-ci-robot k8s-ci-robot merged commit ffee0a4 into kubernetes-sigs:main Jul 26, 2022
@k8s-ci-robot k8s-ci-robot added this to the v1.3 milestone Jul 26, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. size/M Denotes a PR that changes 30-99 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

5 participants