Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: container registry #777

Merged
merged 59 commits into from
Mar 23, 2023
Merged

feat: container registry #777

merged 59 commits into from
Mar 23, 2023

Conversation

kanadgupta
Copy link
Member

@kanadgupta kanadgupta commented Mar 4, 2023

🚥 Fixes RM-6538

🧰 Changes

The moment we've been waiting for y'all: enhancing our semantic-release workflow to automatically publish docker images to the GitHub Container Registry!

There are a bunch of moving parts, but if you look at the Linear ticket, I've added comments with links to a couple internal videos that I created

🧬 QA & Testing

I did a couple of test pushes to the Container Registry and confirmed that a working, containerized version of rdme is pushed to the registry!

And the performance gains are pretty incredible. Here's the before:

image

And here's the after:

image

There are a few aspects of this process that we can only test once we merge into next, but that's why we're doing pre-releases now amirite?

... just to see how much slower it is
... to see if the image builds with everything properly
... without the permissions object?
i'm gonna be so mad if we do lol
@kanadgupta kanadgupta added enhancement New feature or request GHA / CI Issues specific to GitHub Actions or other CI environments labels Mar 22, 2023
@kanadgupta kanadgupta marked this pull request as ready for review March 22, 2023 23:20
.github/workflows/release.yml Show resolved Hide resolved
bin/docker.js Show resolved Hide resolved
.github/workflows/ci.yml Show resolved Hide resolved
bin/verify-clis.sh Show resolved Hide resolved
Copy link
Contributor

@RyanGWU82 RyanGWU82 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't feel like I have a good grasp of the big picture of everything that happens during a release here, but if you've tested it and you're comfortable with it, then that's a good sign. Is there documentation for the build/release process and what's happening under the hood?

Can't approve it because of the unsanitized input to shell commands, but once that's fixed I'm 👍🏼.

bin/docker.js Outdated
*/
async function runCmd(cmd) {
// https://stackoverflow.com/a/63027900
const execCmd = exec(cmd);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

child_process.exec() takes a string that it passes to the shell. However that's not particularly safe, and we're not doing anything to sanitize the input when building the shell commands in run().

Never pass unsanitized user input to this function. Any input containing shell metacharacters may be used to trigger arbitrary command execution.

Source: https://nodejs.org/api/child_process.html#child_processexeccommand-options-callback

Can we switch these commands to child_process.execFile and pass an array of arguments? https://nodejs.org/api/child_process.html#child_processexecfilefile-args-options-callback

Copy link
Member Author

@kanadgupta kanadgupta Mar 23, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

So I might be misunderstanding but I'm not sure arbitrary command execution is much of an issue here — I already do some sanitization checks here to ensure that the env variable input is valid (and this is really the only "user" input):

rdme/bin/docker.js

Lines 16 to 21 in a9f14e4

if (!Object.keys(metadata.labels || {})?.length) {
throw new Error('Invalid shape (missing labels data)');
}
if (!metadata?.tags?.length) {
throw new Error('Invalid shape (missing tags data)');
}

And beyond that, everything is mostly hardcoded strings to construct the docker CLI commands and call the runDockerCmd (f.k.a. runCmd) function (and I've ran through the commands locally a bunch to confirm that they're valid).

In any case, I refactored this in 1b07bb7 to use execFile, TIL! It's much more readable now too so thanks for the suggestion 🎊

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good! My thinking was the tags, which come from git branch names and those could be anything. It looks like git does have some sanitization (I tried a semicolon and got fatal: 'foo; reboot' is not a valid branch name) but I could imagine there are ways around this if someone was motivated to try hard enough.

bin/docker.js Show resolved Hide resolved
@kanadgupta kanadgupta requested review from RyanGWU82 and erunion March 23, 2023 15:25
Copy link
Member

@erunion erunion left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:nice-rgb:

@kanadgupta kanadgupta merged commit d193416 into next Mar 23, 2023
@kanadgupta kanadgupta deleted the container-registry-v2 branch March 23, 2023 17:12
kanadgupta pushed a commit that referenced this pull request Mar 23, 2023
kanadgupta pushed a commit that referenced this pull request Mar 29, 2023
# [8.6.0](v8.5.0...v8.6.0) (2023-03-29)

### Bug Fixes

* bad merge ([e15c574](e15c574))
* bump node version in release workflow ([7f3158f](7f3158f))
* does this work? ([c81e432](c81e432))
* memory leak in large file handling within openapi-parser ([#784](#784)) ([1b1cc00](1b1cc00))
* next channel ([ce4e494](ce4e494))
* **openapi:** yaml strings would be improperly parsed as Date objects ([#779](#779)) ([72e75cb](72e75cb))
* rebuild prior to npm publish ([29b9ec6](29b9ec6))
* reformat github release header ([38c5625](38c5625))
* reformat header again ([bd2e1a2](bd2e1a2))
* remove some of the package scripts ([3eb52fd](3eb52fd))
* remove unnecessary config ([c22889c](c22889c))
* run tests but NOT release workflow on release commits ([24f885e](24f885e))
* temporarily disable release workflow ([a935268](a935268))
* try rearranging steps like this ([cac0c1d](cac0c1d))
* try this approach to lifecycle events ([4e5ecff](4e5ecff))
* try this as an alternative to @semantic-release/github ([8c343a0](8c343a0))
* try this to see if branch protections work ([f314c3f](f314c3f))
* turns out these rules weren't redundant ([f9f82f1](f9f82f1))
* upgrading `oas-normalize` to move to our `postman-to-openapi` fork ([#776](#776)) ([ee8ce0a](ee8ce0a))

### Features

* add git + changelog plugins ([85e4bfd](85e4bfd))
* container registry ([#777](#777)) ([d193416](d193416)), closes [/github.com//pull/777#discussion_r1145516673](https://github.com//github.com/readmeio/rdme/pull/777/issues/discussion_r1145516673) [/github.com//pull/777#discussion_r1145528308](https://github.com//github.com/readmeio/rdme/pull/777/issues/discussion_r1145528308)
* docker (again) ([#763](#763)) ([2144572](2144572)), closes [#746](#746)
* empty commit to trigger release ([3e3c112](3e3c112))
* fix comment ([076cfbf](076cfbf))

### Performance Improvements

* **docker:** build executable ([#764](#764)) ([af2dbce](af2dbce))

### Reverts

* Revert "feat: drop duplicative tag" ([f9fe6c6](f9fe6c6))
* bring workflow name back ([c07495a](c07495a))
* don't set header for changelog ([194489e](194489e))
* restore release workflow ([9f6bbc9](9f6bbc9))
* ugh here we go again ([0b1e429](0b1e429))

[skip ci]
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request GHA / CI Issues specific to GitHub Actions or other CI environments
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants