Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix in docker_inspect_self_mountinfo() fo Gitlab runner #1569

Open
wants to merge 1 commit into
base: main
Choose a base branch
from

Conversation

zweikaesehoch
Copy link

Crossbuilding in our buildimage on a Gitlab runner failed:
image

Container ID could not be resolved from /proc/self/mountinfo, because "/docker/" string was expected, but "/docker-data/" provided:
image

I extended string matching with "/docker-data/" and our pipeline is working again.

@zweikaesehoch zweikaesehoch requested a review from a team as a code owner October 7, 2024 10:09
Copy link
Member

@Emilgardis Emilgardis left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks!

@zweikaesehoch
Copy link
Author

zweikaesehoch commented Oct 7, 2024

We further investigated the error which led us to our custom data-root setting in /etc/docker/daemon.json:
image

The fallback implemented due to #1321 misses the container id because of our changed data-root to "docker-data" instead of default "docker".

The real issue here is, that docker_inspect_self_mountinfo() wont work, if data-root setting contains no "/docker/".

The current PR bears no value for other users now.
One would need to parse daemon.json data-root setting to make the fallback more robust.

Pseudocode:

Look into /etc/docker/daemon.json
if data-root is set
  extract last path element as DATA_ROOT
  if DATA_ROOT not equals "/docker"
    add DATA_ROOT to OR-filter strings in line 1367

@Emilgardis Emilgardis added needs-decision no changelog A valid PR without changelog (no-changelog) labels Oct 7, 2024
@andrejohansson
Copy link

This is currently blocking me and I have no authority over our agents. Is there something I can do as a workaround?

@Emilgardis
Copy link
Member

This is currently blocking me and I have no authority over our agents. Is there something I can do as a workaround?

if you add -v do you get any debug messages about docker inspect found no containers matching HOSTNAME, retrying using mountinfo? if no, try setting the HOSTNAME envvar to the name of a container

I think I have a workaround, I'll create a new PR and then you and @zweikaesehoch could maybe try it

@arakholia
Copy link

I have also run into this issue attempting to run cross on GitHub Codespaces. It seems local devcontainers use /var/lib/docker/containers/ while on the Codespaces it uses just /containers/. The following pattern match (similar to this PR) resolved the issue and seems to handle both uses cases:

.filter(|s| s.contains("/docker/") || s.contains("/containers/"))```

although there may be a better way.

@Emilgardis
Copy link
Member

Please give #1588 a try 👍🏼

@andrejohansson
Copy link

My workaround was to just use the cross-rs images directly as the base image in CI. No docker-in-docker.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
needs-decision no changelog A valid PR without changelog (no-changelog) no-ci-tests
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants