Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: replace super-linter with custom linter list #97

Merged
merged 35 commits into from
May 24, 2023
Merged
Show file tree
Hide file tree
Changes from 29 commits
Commits
Show all changes
35 commits
Select commit Hold shift + click to select a range
5c9ae41
removed super-linter and replaced with individual linters
ChiefHolland May 12, 2023
acc4ea2
Updating Dockerfile
ChiefHolland May 12, 2023
f1d0c93
added ability to use custom configs for linters
ChiefHolland May 17, 2023
1fa34a2
heading off merge conflicts
ChiefHolland May 17, 2023
fd4bf67
Merge branch 'main' into replace-super-linter
JonZeolla May 18, 2023
3a06296
Update Dockerfile
ChiefHolland May 18, 2023
6302e67
Update Dockerfile
ChiefHolland May 18, 2023
35ccbf1
post-review corrections
ChiefHolland May 19, 2023
587a6fa
adding git package
ChiefHolland May 19, 2023
35c02b7
cleaning up bool bashisms
ChiefHolland May 19, 2023
29a2df6
fixing logic of get_files_matching_filetype
ChiefHolland May 19, 2023
0e6925b
fixing logic
ChiefHolland May 19, 2023
2f7e4a7
fixing logic
ChiefHolland May 19, 2023
7d542c3
fixing logic
ChiefHolland May 19, 2023
6eb83c3
fixing logic
ChiefHolland May 19, 2023
3fb2811
fixing logic
ChiefHolland May 19, 2023
feafb1b
fixing logic
ChiefHolland May 19, 2023
84dea36
fixing logic
ChiefHolland May 19, 2023
b634dc2
fixing logic
ChiefHolland May 19, 2023
756c997
fixing logic
ChiefHolland May 20, 2023
5ba508c
updating README
ChiefHolland May 20, 2023
105eadc
fixing logic
ChiefHolland May 20, 2023
ccf83ac
fixing logic
ChiefHolland May 20, 2023
a023958
fixing logic
ChiefHolland May 20, 2023
385870b
correcting reviewed items
ChiefHolland May 21, 2023
c230b42
Merge branch 'main' into replace-super-linter
ChiefHolland May 22, 2023
8e6cba5
Log how long it took
JonZeolla May 23, 2023
11a89ab
Update docs
JonZeolla May 23, 2023
9f216ff
Apply suggestions from code review
JonZeolla May 23, 2023
63331f4
Turn off cfn-lint debug; incredibly noisy
JonZeolla May 24, 2023
5f92079
Add linter skipping log at the end
JonZeolla May 24, 2023
a705e4a
temporarily removing shfmt
ChiefHolland May 24, 2023
d9f606d
Finish shfmt cleanup
JonZeolla May 24, 2023
62de378
chore: updating docs
ChiefHolland May 24, 2023
93db7f1
Rearrange
JonZeolla May 24, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 3 additions & 0 deletions .github/etc/dictionary.txt
Original file line number Diff line number Diff line change
@@ -1,2 +1,5 @@
capricornis
crispus
hadolint
kubeconform
textlint
2 changes: 1 addition & 1 deletion .github/workflows/commit.yml
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
name: "CI"

on:
on: # yamllint disable-line rule:truthy
ChiefHolland marked this conversation as resolved.
Show resolved Hide resolved
push:
branches:
- main
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/security.yml
Original file line number Diff line number Diff line change
@@ -1,15 +1,15 @@
---
name: "Security"

on:
on: # yamllint disable-line rule:truthy
JonZeolla marked this conversation as resolved.
Show resolved Hide resolved
push:
branches:
- main
pull_request:
branches:
- main
schedule:
- cron: '2 3 * * *'
- cron: "2 3 * * *"

env:
python_version: "3.10"
Expand Down
53 changes: 0 additions & 53 deletions .pre-commit-config.yaml

This file was deleted.

64 changes: 59 additions & 5 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
# GOAT Development Notes

## Environmental Setup
## Environment Setup

Ensure you have `docker` and `pipenv` installed locally, and the `docker` daemon is running.
Then run the following command to install the dependencies onto your local system:
Ensure you have `docker` and `pipenv` installed locally, and the `docker` daemon is running.
Then run the following command to install the dependencies onto your local system:

```bash
pipenv install --deploy --ignore-pipfile --dev
pipenv install --deploy --ignore-pipfile --dev
```

## Running the goat against the goat project locally
Expand All @@ -25,9 +25,63 @@ There are two ways of running the `goat` locally:

```bash
docker build .
docker run -e RUN_LOCAL=true -v $PWD:/goat/ --rm <first several character of the hash output from the build step>
docker run -v $PWD:/goat/ --rm <first several character of the hash output from the build step>
```

3. To pass in custom configs for individual linters or to exclude files with regular expressions, set environment variables:

```bash
docker run -e INPUT_EXCLUDE='.*\.json$' -e BLACK_CONFIG='--required-version 21.9b0' -v $PWD:/goat/ --rm <hash>

Running Seiso Linter
--------------------------

Running linter: ruff
Running linter: hadolint
Running linter: kubeconform
Running linter: dockerfile_lint
Running linter: markdown-link-check
Running linter: cspell
Running linter: cfn-lint
Running linter: jscpd
Running linter: actionlint
Running linter: markdownlint
Running linter: textlint
Running linter: black
Running linter: mypy
Running linter: shfmt
Running linter: shellcheck
Running linter: yamllint
===============================
BLACK
-------------------------------
Oh no! 💥 💔 💥 The required version `21.9b0` does not match the running version `23.3.0`!

Scanned 45 files in 6 seconds
Excluded 575 files

INFO: hadolint completed successfully
INFO: dockerfile_lint completed successfully
INFO: cfn-lint completed successfully
INFO: textlint completed successfully
INFO: markdownlint completed successfully
INFO: mypy completed successfully
INFO: jscpd completed successfully
INFO: kubeconform completed successfully
INFO: shellcheck completed successfully
INFO: shfmt completed successfully
INFO: ruff completed successfully
INFO: cspell completed successfully
INFO: actionlint completed successfully
INFO: markdown-link-check completed successfully
INFO: yamllint completed successfully
ERROR: black found errors
ERROR: Linting failed
```

Note: Linter env variables must be formatted as <LINTER_CONFIG>, i.e. RUFF_CONFIG, CFN_LINT_CONFIG,
etc., and the values supplied will overwrite the default arguments supplied in the goat.

### Note

If adding linters to `linters.json`, the `executor` is an optional member of the linter object.
34 changes: 23 additions & 11 deletions Dockerfile
Original file line number Diff line number Diff line change
@@ -1,8 +1,10 @@
# Based on python:alpine as of February 2021
FROM github/super-linter:slim-v5.0.0
FROM ghcr.io/yannh/kubeconform:v0.6.1 as kubeconform
FROM hadolint/hadolint:v2.12.0-alpine as hadolint
FROM koalaman/shellcheck:v0.9.0 as shellcheck
FROM mvdan/shfmt:v3.6.0 as shfmt
FROM rhysd/actionlint:1.6.24 as actionlint

# Required for the github/super-linter log (cannot be disabled)
RUN mkdir -p /tmp/lint/
FROM python:3.10-alpine3.17 as base_image
JonZeolla marked this conversation as resolved.
Show resolved Hide resolved

ENV LANG=C.UTF-8
ENV LC_ALL=C.UTF-8
Expand All @@ -21,21 +23,31 @@ LABEL org.opencontainers.image.licenses="MIT"
WORKDIR /etc/opt/goat/
ENV PIP_NO_CACHE_DIR=1
COPY Pipfile Pipfile.lock ./
COPY --from=kubeconform /kubeconform /usr/bin/
COPY --from=hadolint /bin/hadolint /usr/bin/
COPY --from=shellcheck /bin/shellcheck /usr/bin/
COPY --from=shfmt /bin/shfmt /usr/bin/
COPY --from=actionlint /usr/local/bin/actionlint /usr/bin/

# hadolint ignore=DL3016,DL3018,DL3013
RUN pip install pipenv \
&& pipenv install --deploy --ignore-pipfile \
&& pip install shed ruff \
&& pipenv install --system --deploy --ignore-pipfile \
&& apk upgrade \
&& apk --no-cache add go \
jq \
npm \
tini \
ruby \
&& gem install \
rubocop \
rubocop-github \
&& npm install --no-cache -g dockerfile_lint \
bash \
git \
&& npm install --save-dev --no-cache -g dockerfile_lint \
markdownlint-cli \
textlint \
textlint-filter-rule-allowlist \
textlint-filter-rule-comments \
textlint-rule-terminology \
textlint-plugin-rst \
cspell \
jscpd \
markdown-link-check \
&& mkdir -p /opt/goat/log

Expand Down
5 changes: 5 additions & 0 deletions Pipfile
Original file line number Diff line number Diff line change
Expand Up @@ -4,6 +4,11 @@ verify_ssl = true
name = "pypi"

[packages]
ruff = "*"
mypy = "*"
cfn-lint = "*"
black = "*"
yamllint = "*"

[dev-packages]
docker = "*"
Expand Down
Loading