Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

OpenBSD 7.4 - shutdown hangs (only x86_64, not arm64) #72

Closed
kobalicek opened this issue Nov 6, 2023 · 4 comments
Closed

OpenBSD 7.4 - shutdown hangs (only x86_64, not arm64) #72

kobalicek opened this issue Nov 6, 2023 · 4 comments
Labels
bug Something isn't working

Comments

@kobalicek
Copy link

kobalicek commented Nov 6, 2023

I'm still having shutdown failures now with OpenBSD 7.4 VM:

  Shuting down VM
  Executing command inside VM: sudo shutdown -h -p now
  /usr/bin/ssh -t [email protected]
  Pseudo-terminal will not be allocated because stdin is not a terminal.
  Error: The operation was canceled.

Here is a reference:

https://github.com/blend2d/blend2d/actions/runs/6763854194/job/18381464122

Note that this must be a deadlock or something because the runner is killed in 6 hours. The workflow is using sync_files: runner-to-vm, but that doesn't fix the problem.

One interesting thing is that this doesn't happen with ARM64 VM, only x86_64 one.

So, opening this issue to see whether there are others having the same problem.

@jacob-carlborg jacob-carlborg added the bug Something isn't working label Nov 8, 2023
@jacob-carlborg
Copy link
Contributor

I've added some more debug information and it seems to stop making progress in the SSH command to shut down the VM. Specifically, just before authentication is supposed to succeed.

@opcm
Copy link

opcm commented Dec 8, 2023

@jacob-carlborg
Copy link
Contributor

jacob-carlborg commented Dec 9, 2023

same here: https://github.com/intel/pcm/actions/runs/7128218686/job/19443586521

That's for FreeBSD, so it's not just a problem with OpenBSD. It might be an issue with xhyve since it doesn't seem to be a problem on ARM64. QEMU is used on ARM64.

It might also not be just be a problem with shutting down the VM. Here's one of my projects that stops making progress in the middle of the job, not at the end: https://github.com/jacob-carlborg/dstep/actions/runs/7040324159/job/19161045455#step:6:429.

I think I can specify a timeout for the SSH commands that are used to communicate with the VM. Combining that with a retry might help. Or at least fail the job earlier.

jacob-carlborg added a commit that referenced this issue Dec 18, 2023
This is more efficient and this will hopefully mitigate very frequent
freezing of VM during teardown
([#61](#61),
[#72](#72)).
@jacob-carlborg
Copy link
Contributor

Hopefully fixed/mitigated in 01856ae and d918493.

knightjoel added a commit to knightjoel/awscliv2-on-openbsd that referenced this issue Jan 20, 2024
korli added a commit to korli/action that referenced this issue Mar 15, 2024
Added
- Added support for using the action in multiple steps in the same job ([cross-platform-actions#26](cross-platform-actions#26)).
    All the inputs need to be the same for all steps, except for the following
    inputs: `sync_files`, `shutdown_vm` and `run`.

- Added support for specifying that the VM should not shutdown after the action
    has run. This adds a new input parameter: `shutdown_vm`. When set to `false`,
    this will hopefully mitigate very frequent freezing of VM during teardown ([cross-platform-actions#61](cross-platform-actions#61), [cross-platform-actions#72](cross-platform-actions#72)).

Changed
- Always terminate VM instead of shutting down. This is more efficient and this
    will hopefully mitigate very frequent freezing of VM during teardown
    ([cross-platform-actions#61](cross-platform-actions#61),
    [cross-platform-actions#72](cross-platform-actions#72)).

- Use `unsafe` as the cache mode for QEMU disks. This should improve performance ([cross-platform-actions#67](cross-platform-actions#67)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants