Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add process dump functionality for WCOW/LCOW #1062

Merged
merged 2 commits into from
Sep 24, 2021

Conversation

dcantah
Copy link
Contributor

@dcantah dcantah commented Jul 7, 2021

This commit adds support for generating process dumps for hypervisor isolated containers. This includes
a new annotation to specify where process dumps should get placed on creation, which is global
to all containers.

This was verified the following way through crictl:

Linux Pod spec
--------------
{
    "metadata": {
        "name": "sandbox-coredump",
        "namespace": "default",
        "attempt": 1
    },
    "annotations": {
        "io.microsoft.container.processdumplocation": "/coredumps/core.%s.%e.%P.%t" 
    }
}
Linux container spec 
-----------------
{
        "metadata": {
                "name": "coredump-test"
        },
        "image": {
                "image": "docker.io/library/ubuntu:latest"
        },
        "command": [
                "sh",
                "-c",
                "while true; do echo 'Hello, World!'; sleep 1; done"
        ],
        "mounts": [
                {
                        "container_path": "/coredumps",
                        "host_path": "sandbox:///core"
                },
                {
                         "container_path": "/host",
                         "host_path": "C:\\users\\dcanter\\desktop\\coredump"
                }
        ],
        "annotations": {
                "io.microsoft.lcow.rlimitcore": "18446744073709551615;18446744073709551615"
        },
        "linux": {}
}

Get a shell into the container and then grab the current pid (echo $$) and do a kill -6 <pid>. Re-exec in and there should be a file in /coredumps named something like "core.6.sh.176.1626339581". Then cp the file to /host and try out gdb on the file through wsl. gdb -c /mnt/c/Users/dcanter/desktop/coredump/core.6.sh.176.1626339581

Below is the Windows configs for this as well:

Windows Pod Spec:
--------------
{
    "metadata": {
        "name": "sandbox-wcow",
        "namespace": "default",
        "attempt": 1
    },
    "annotations": {
        "io.microsoft.container.processdumplocation": "C:\\crashdump"
    }
}
Windows container spec 
-----------------
{
        "metadata": {
                "name": "wcow-test"
        },
        "image": {
                "image": "cplatpublic.azurecr.io/crashdump:latest"
        },
        "command": [
                "C:\\app\\crashtest.exe",
                "so"
        ],
        "mounts": [{
                "container_path": "C:\\crashdump",
                "host_path": "C:\\users\\dcanter\\desktop\\coredump"
        }]
}

Afterwards there should a dmp at whatever the specified directory was. The container image is a slightly modified version of https://github.com/spreadex/win-docker-crash-dump. As we're setting wersvc to auto-start Stackoverflow exceptions work as well.

Signed-off-by: Daniel Canter [email protected]

@dcantah dcantah marked this pull request as ready for review July 13, 2021 16:58
@dcantah dcantah requested a review from a team as a code owner July 13, 2021 16:58
@dcantah dcantah force-pushed the coredump branch 3 times, most recently from 0b12030 to e242b1f Compare July 15, 2021 13:34
internal/guest/runtime/hcsv2/spec.go Outdated Show resolved Hide resolved
internal/hcsoci/hcsdoc_wcow.go Outdated Show resolved Hide resolved
internal/hcsoci/hcsdoc_wcow.go Show resolved Hide resolved
internal/uvm/create_wcow.go Show resolved Hide resolved
Copy link
Contributor

@anmaxvl anmaxvl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

lgtm

@dcantah dcantah force-pushed the coredump branch 2 times, most recently from 313641f to dd93f38 Compare August 17, 2021 16:13
@dcantah
Copy link
Contributor Author

dcantah commented Sep 7, 2021

@katiewasnothere Responded to feedback, ptal when you get another chance!

@katiewasnothere katiewasnothere self-assigned this Sep 7, 2021
@kevpar kevpar assigned kevpar and anmaxvl and unassigned kevpar Sep 9, 2021
@dcantah
Copy link
Contributor Author

dcantah commented Sep 14, 2021

@katiewasnothere Ping

@katiewasnothere
Copy link
Contributor

Once the windows sandbox mount PR is merged, we should add a test for it here and then I think we're good to merge.

This commit adds support for generating process dumps for hypervisor isolated containers. This includes
a new annotation to specify where process dumps should get placed on creation, which is global
to all containers.

Signed-off-by: Daniel Canter <[email protected]>
This change adds cri-containerd testcases that excercises the process dump
functionality. It sets up two containers, using images that stackoverflow/
throw an exception shortly after starting. This should generate a dump
file in the sandbox mount location in the test. Then the second container
mounts the same sandbox mount and just verify that the core dump file generated
by the first is present.

Signed-off-by: Daniel Canter <[email protected]>
@dcantah
Copy link
Contributor Author

dcantah commented Sep 24, 2021

@katiewasnothere Checking in shortly now that the sandbox pr is in and a new test

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants