Allow the Docker image to be run with a random user id (#12905) #18873
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jsoriano
added
review
[zube]: In Review
Team:Platforms
|
Pinging @elastic/integrations-platforms (Team:Platforms) |
botelastic
bot
added
needs_team
|
run beats-ci/package |
💔 Tests Failed
Expand to view the summary
Build stats
Test stats 🧪
Test errorsExpand to view the tests failures
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
No worries, I should have seen these issues with strict perms come 😬
Not sure how to do this, I guess you would need to update libbeat in apm-server, or at least the packaging templates.
Yeah, I think we can merge the revert, and go on here. |
Apply the ownership changes of elastic#12905, without applying the permission changes, so it still satisfies strict perms checks.
|
run beats-ci/package |
jsoriano
changed the title
❕ Build Aborted
Expand to view the summary
Build stats
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
💔 Build Failed
Expand to view the summary
Build stats
Steps errorsExpand to view the steps failures
Log outputExpand to view the last 100 lines of log output
|
jsoriano
added
v7.9.0
needs_backport
|
run beats-ci/package |
|
@barkbay sorry for the delay, I am back with this. I have updated the branch with master. You can build an apm-server docker image with this branch running these commands from the apm-server working directory: I did some smoke tests and it seems to start correctly with and without random user ids. Could you also confirm? thanks! |
|
@jsoriano no worries ! I was quite busy myself... Thanks a lot for this PR 🙇 |
barkbay
reviewed
Jun 29, 2020
There was a problem hiding this comment.
LGTM, I did some tests on Openshift and on K8S with a restricted PSP.
The only issue I have is this test which is failing and I don't understand why:
>> Testing package contents
--- FAIL: TestDocker (13.81s)
--- FAIL: TestDocker/apm-server-7.9.0-linux-amd64.docker.tar.gz_config_file_permissions (0.00s)
package_test.go:206: file usr/share/apm-server/apm-server.yml has wrong permissions: expected=-rw-r----- actual=-rw-rw----
--- FAIL: TestDocker/apm-server-oss-7.9.0-linux-amd64.docker.tar.gz_config_file_permissions (0.00s)
package_test.go:206: file usr/share/apm-server/apm-server.yml has wrong permissions: expected=-rw-r----- actual=-rw-rw---
|
/packaging |
No, when the APM Server is run with ECK the data dir is an
The UID is randomly assigned by Openshift: you can find here the Pod as it is created with ECK on Openshift (and here the ECK manifest). |
|
I have been doing several tests and I think we can go on with merging this. As summary of my tests:
|
|
/packaging |
exekias
approved these changes
Jul 1, 2020
jsoriano
added a commit
to jsoriano/beats
that referenced
this pull request
Jul 1, 2020
elastic#18873) Prepare docker images to be run with arbitrary user ids. Following common practices and recommendations, files that need to be read by Beats have now read permissions for the group and belong to the root group. Also, the user included in the docker image is added to the root group so it can read these files when run on docker with default user and privileges. Some changes are also added to Kubernetes reference manifests to help running beats with arbitrary user ids, though this is not completely supported and it requires additional setup. Co-authored-by: Michael Morello <[email protected]> (cherry picked from commit 3ff02cb)
jsoriano
removed
the
needs_backport
|
I have created an issue to keep track of some issues I found trying to run Beats on restricted environments: #19600 |
jsoriano
added a commit
that referenced
this pull request
Jul 2, 2020
…) (#19555) Prepare docker images to be run with arbitrary user ids. Following common practices and recommendations, files that need to be read by Beats have now read permissions for the group and belong to the root group. Also, the user included in the docker image is added to the root group so it can read these files when run on docker with default user and privileges. Some changes are also added to Kubernetes reference manifests to help running beats with arbitrary user ids, though this is not completely supported and it requires additional setup. (cherry picked from commit 3ff02cb) Co-authored-by: Michael Morello <[email protected]>
1 task
15 tasks
jsoriano
added a commit
that referenced
this pull request
Aug 24, 2020
Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (#20356). * Allow the docker image to be run with random user ids (#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are.
15 tasks
jsoriano
added a commit
to jsoriano/beats
that referenced
this pull request
Aug 24, 2020
Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (elastic#20356). * Allow the docker image to be run with random user ids (elastic#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are. (cherry picked from commit e31794d)
jsoriano
added a commit
that referenced
this pull request
Aug 24, 2020
Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (#20356). * Allow the docker image to be run with random user ids (#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are. (cherry picked from commit e31794d)
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
elastic#18873) Prepare docker images to be run with arbitrary user ids. Following common practices and recommendations, files that need to be read by Beats have now read permissions for the group and belong to the root group. Also, the user included in the docker image is added to the root group so it can read these files when run on docker with default user and privileges. Some changes are also added to Kubernetes reference manifests to help running beats with arbitrary user ids, though this is not completely supported and it requires additional setup. Co-authored-by: Michael Morello <[email protected]>
melchiormoulin
pushed a commit
to melchiormoulin/beats
that referenced
this pull request
Oct 14, 2020
Add an additional docker build that builds images based on Red Hat UBI, following Red Hat requirements for certified images. Additional checks have been added to packaging tests for labels and licenses. Additional changes done to support it also in Elastic Agent images: * Home directory is prepared in a different stage (elastic#20356). * Allow the docker image to be run with random user ids (elastic#18873). * Explicitly select a Dockerfile and entry point template. * Add NOTICE.txt file to all agent packages. * Actually run package tests after building packages, added flag to allow root user. * Improved checks on required packages, so they are not re-built if they already are.
4 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Apply changes on ownership proposed in #12905, but keep the permissions, to avoid the
issues reported in #18858.
I think this could be enough to run containers with arbitrary user ids, because beats don't need to write these files, only read them.
Make changes also to the kubernetes reference manifests to help running beats with arbitrary user ids. These manifests still won't work on restricted environments.
Fixes #18871
Changes were previously reverted in #18872
Co-authored-by: Michael Morello [email protected]
How to test
PLATFORMS=linux/amd64 mage package, or use one of the pre-built snapshots including this change.--privilegedand withoutBEAT_STRICT_PERMS.It should behave the same on these scenarios (auditbeat will fail to configure audit, this is expected unless
--privileged --user 0 --pid=hostis also used):docker run -it --rm docker.elastic.co/beats/filebeat:8.0.0docker run -it --rm --user 0 docker.elastic.co/beats/filebeat:8.0.0docker run -it --rm --user 1000 docker.elastic.co/beats/filebeat:8.0.0docker run -it --rm --user 100042 docker.elastic.co/beats/filebeat:8.0.0