Issue 168 vulnerability checks #183

nigelgbanks · 2022-02-02T13:34:19Z

Also updates many packages and software to reduce the number of vulnerabilities though many still exist in the java portions of the stack.

alpine to latest stable release
imagemagick
tomcat
fits

Requires Islandora-Devops/isle-gradle-docker-plugin#12 to be reviewed and merged.

nigelgbanks · 2022-02-02T13:56:50Z

Tested with isle-dc and cantalope doesn't seem to work, on further investigation though it seems that it's looking for the files in the wrong place (i.e. in Fedora when the service file is on disc, doesn't actually seem like cantalope it self broke),

-- EDIT

Ya I just tested isle-dc with older images and cantaloupe isn't working their either it's likely not related.

nigelgbanks · 2022-02-02T21:07:13Z

Islandora-Devops/isle-dc#219 resolved the cantaloupe issue

nigelgbanks · 2022-03-03T18:19:25Z

Since you've been looking a pull requests, bump @dannylamb.

g7morris · 2022-03-14T15:38:48Z

Hi @nigelgbanks & @dannylamb curious what action or review is further required here? Should this go into the March / April 2022 - isle-buildkit 1.0.0 release ?

nigelgbanks · 2022-03-16T12:36:23Z

Just needs to be tested and merged, by someone other than me.

g7morris · 2022-03-23T18:33:56Z

Reporting results from testing process:

Test Environment

OS: Ubuntu 20.04 LTS Desktop
CPU: Intel i7-10510U (8 cores) @ 4.900 GHZ
Mem: 32 GB

Test process / steps

git clone [email protected]:Islandora-Devops/isle-buildkit.git to local laptop
git checkout issue-168-vulnerability-checks`
./gradlew build

Attempt 1

Ran for about ~8 mins with full CPU lockup the entire time. @nigelgbanks Any way we can reduce that?

#44 [stage-4 16/17] RUN ln -s /usr/local/share/.config/yarn/global/node_modules/.bin/code-server /usr/local/bin/code-server &&     ln -s /usr/local/share/.config/yarn/global/node_modules/.bin/node-gyp /usr/local/bin/node-gyp &&     ln -s /usr/local/share/.config/yarn/global/node_modules/.bin/grunt /usr/local/bin/grunt &&     ln -s /usr/local/share/.config/yarn/global/node_modules/.bin/bower /usr/local/bin/bower
#44 DONE 0.3s

#45 [stage-4 17/17] RUN chmod a=r,u+w /etc/sudo.conf
#45 DONE 0.3s

#46 exporting to image
#46 exporting layers
#46 exporting layers 3.1s done
#46 writing image sha256:164bfced1d11cf2b05c914dfac688138cbd19d4acf2bbf7ebc5c60f8456d303a done
#46 naming to docker.io/local/code-server:latest done
#46 DONE 3.1s

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':alpaca:build'.
> Process 'command 'docker'' finished with non-zero exit value 1

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

BUILD FAILED in 8m 20s
28 actionable tasks: 28 executed

Attempt 2

./gradlew build

#18 exporting to image
#18 exporting layers
#18 exporting layers 0.4s done
#18 writing image sha256:f92db08a102678c7525c8db5d0222553053d350b955af8673e0f96e87592d7de done
#18 naming to docker.io/local/alpaca:latest done
#18 DONE 0.4s

BUILD SUCCESSFUL in 2m 17s
33 actionable tasks: 7 executed, 26 up-to-date

@nigelgbanks 2nd time is the charm? What additional steps should I take from here? Attempt to use isle-dc with these created images?

nigelgbanks · 2022-03-23T18:37:45Z

@g7morris looks like I need to rebase this due to conflicts on main from just merging the fixes for code-server #198

g7morris · 2022-03-23T18:53:12Z

@nigelgbanks To be clear this is something you're intending to do this week and then I can get back to testing correct?

…availiable.

nigelgbanks · 2022-03-23T18:58:15Z

@g7morris I was just sorting out the conflict and doing a quick build to check locally.

nigelgbanks · 2022-03-23T19:00:29Z

You should be good to test when this finishes: https://github.com/Islandora-Devops/isle-buildkit/runs/5665429249?check_suite_focus=true with the commit 562ca9854343031da934dbaaea9728bca0f9371a

g7morris · 2022-03-23T20:39:37Z

Ignore previous comment. I'm too quick to test the build / push needs to finish.

nigelgbanks · 2022-03-23T20:44:28Z

@g7morris Unfortunately this one needs to do a full rebuild which includes compiling imagemagick under emulation for linux/arm64 which is super slow

nigelgbanks · 2022-03-23T20:45:12Z

@g7morris actually in this case it might be faster to build it locally as you'd only be building for your platform.

g7morris · 2022-03-23T20:46:37Z

@nigelgbanks What steps would be needed to do this? I can run it over night when I get back home on my laptop (non-Arm) or Apple M1 (ARM ish) Is there a flag for arm? Is this in isle-dc for the full monty or isle-buildkit. Sorry long day I'm probably asking something stupid.

nigelgbanks · 2022-03-23T21:03:54Z

@g7morris looks like it's finished 😌 so you can just pull 562ca9854343031da934dbaaea9728bca0f9371a.

That being said, if you do ./gradlew build on a Mac M1 it will build natively for Linux/arm64 if you use the same command on older x86 Mac it will build natively for Linux/x86_64 so it's handled automatically for you. The build and push task for the Github actions though always builds both platforms Linux/x86_64 which is relatively fast to build, and Linux/arm64 is built via emulation with qemu which is super slow. This is cause we push to Docker hub and need to support people downloading for either platform.

nigelgbanks · 2022-03-23T21:05:03Z

Typically we make user of heavy caching which cuts the time down from over an hour to like 10 mins or so, but this pull request changes all the base images so no caching can be used.

nigelgbanks · 2022-03-23T21:07:44Z

As for tests just the normal bring up the demo and muck about, the grype reports can be downloaded here:

https://github.com/Islandora-Devops/isle-buildkit/suites/5774283956/artifacts/192302600

So that bit works, it's more so just to confirm I'm not broke anything, the automated tests wouldn't pick up on.

g7morris · 2022-03-24T03:29:57Z

Okay I think this passes and using the arm images to test is much faster on a Mac both in the isle-dc make demo process (built in < 2 mins) and the gradle build.

Reporting results from testing process:

isle-buildkit

Test Environment

OS: MacOS Monterey 12.3 - (Mac Mini 9,1)
CPU: 8 cores
Mem: 16 GB

Test process / steps

git clone [email protected]:Islandora-Devops/isle-buildkit.git to local laptop
git checkout issue-168-vulnerability-checks`
./gradlew build

Attempt 1

Ran for about < 6 mins with not so full CPU lockup the entire time. Close but it did go heavy into swap ~ 8GB.

BUILD SUCCESSFUL in 5m 32s
33 actionable tasks: 33 executed

isle-dc

Test Environment

OS: MacOS Monterey 12.3 - (Mac Mini 9,1)
CPU: 8 cores
Mem: 16 GB

Steps taken to test

git clone [email protected]:Islandora-Devops/isle-dc.git to local laptop
cd isle-dc
cp sample.env .env
vi / nano .env
- Changed TAG= to TAG= 562ca9854343031da934dbaaea9728bca0f9371a
make demo

Results

@nigelgbanks Process worked great and much faster than x86. No new errors. Site is snappier and drush commands don't lag. Nice

Think we can ship this one eh?

nigelgbanks · 2022-03-24T07:39:14Z

@g7morris awesome I'll merge it then 👍

nigelgbanks requested a review from dannylamb February 2, 2022 13:34

nigelgbanks mentioned this pull request Feb 2, 2022

Issue 168 vulnerability checks Islandora-Devops/isle-gradle-docker-plugin#12

Merged

nigelgbanks mentioned this pull request Feb 8, 2022

Proposal for regular release schedule/process, like we have been doing for ISLE-7 #184

Closed

g7morris assigned dannylamb Mar 14, 2022

g7morris added security release question Further information is requested and removed question Further information is requested labels Mar 14, 2022

g7morris assigned nigelgbanks Mar 23, 2022

g7morris added question Further information is requested gradle labels Mar 23, 2022

Update plugin to latest to get vunerability analysis checks

ddfd325

Nigel Banks added 8 commits March 23, 2022 18:54

Bumped alpine version to latest stable.

6fc40e5

Updated Imagemagick to latest

539caa7

Fix for java-jna package discontinued for aarch64

87a70ef

Added link to aports issue that requires custom build of java-jna

656e055

Generate grype reports on pushs as well as pull requests.

9dd4ebe

Updated fits to latest to get security updates.

6cf84a6

Added ability to ignore specific vulnerabilities, while a fix is not …

12f47c7

…availiable.

Updated tomcat to latest

49c694d

Nigel Banks added 2 commits March 23, 2022 18:54

Documented additional flag for grype

ee35f79

Made alpine version a argument.

562ca98

nigelgbanks force-pushed the issue-168-vulnerability-checks branch from dde3f2f to 562ca98 Compare March 23, 2022 18:54

nigelgbanks mentioned this pull request Mar 23, 2022

Add vulnerability detection Github Action to scan built docker images #168

Closed

g7morris unassigned dannylamb Mar 24, 2022

g7morris requested review from g7morris and removed request for dannylamb March 24, 2022 03:31

g7morris self-assigned this Mar 24, 2022

nigelgbanks merged commit 2ee80c7 into main Mar 24, 2022

nigelgbanks deleted the issue-168-vulnerability-checks branch March 24, 2022 07:39

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue 168 vulnerability checks #183

Issue 168 vulnerability checks #183

nigelgbanks commented Feb 2, 2022 •

edited

Loading

nigelgbanks commented Feb 2, 2022 •

edited

Loading

nigelgbanks commented Feb 2, 2022

nigelgbanks commented Mar 3, 2022

g7morris commented Mar 14, 2022

nigelgbanks commented Mar 16, 2022

g7morris commented Mar 23, 2022 •

edited

Loading

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022 •

edited

Loading

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 24, 2022

nigelgbanks commented Mar 24, 2022

Issue 168 vulnerability checks #183

Issue 168 vulnerability checks #183

Conversation

nigelgbanks commented Feb 2, 2022 • edited Loading

nigelgbanks commented Feb 2, 2022 • edited Loading

nigelgbanks commented Feb 2, 2022

nigelgbanks commented Mar 3, 2022

g7morris commented Mar 14, 2022

nigelgbanks commented Mar 16, 2022

g7morris commented Mar 23, 2022 • edited Loading

Test Environment

Test process / steps

Attempt 1

Attempt 2

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 23, 2022 • edited Loading

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

nigelgbanks commented Mar 23, 2022

g7morris commented Mar 24, 2022

isle-buildkit

Test Environment

Test process / steps

Attempt 1

isle-dc

Test Environment

Steps taken to test

Results

nigelgbanks commented Mar 24, 2022

nigelgbanks commented Feb 2, 2022 •

edited

Loading

nigelgbanks commented Feb 2, 2022 •

edited

Loading

g7morris commented Mar 23, 2022 •

edited

Loading

g7morris commented Mar 23, 2022 •

edited

Loading