Cannot start a Clear Container on an Ubuntu VM using VMware Fusion #1053
Comments
|
Hi @danielscottt thanks for report the issue - some questions: What runtime version are you using?: What docker version are you using ?: In order to get the runtime logs please try the following: And verify that following lines are not commented in For the proxy logs please enable debug logs in the proxy service adding After that try to run a container: The runtime logs should be located in: To access to the proxy logs you can use journal: |
|
Hi @jcvenegas! Thanks so much for your reply. I think we're on to something! TL;DR: In the runtime's log: Here's the logs and information you requested: $ cc-oci-runtime version
cc-oci-runtime version: 2.2.3
spec version: 1.0.0-rc1
commit: 7f314dc3764e53f366676894994ed44340f4f836And $ sudo docker info
Containers: 13
Running: 0
Paused: 0
Stopped: 13
Images: 2
Server Version: 17.06.0-ce
Storage Driver: aufs
Root Dir: /var/lib/docker/aufs
Backing Filesystem: extfs
Dirs: 28
Dirperm1 Supported: true
Logging Driver: json-file
Cgroup Driver: cgroupfs
Plugins:
Volume: local
Network: bridge host macvlan null overlay
Log: awslogs fluentd gcplogs gelf journald json-file logentries splunk syslog
Swarm: inactive
Runtimes: cor runc
Default Runtime: cor
Init Binary: docker-init
containerd version: cfb82a876ecc11b5ca0977d1733adbe58599088a
runc version: 2d41c047c83e09a6d61d464906feb2a2f3c52aa4
init version: 949e6fa
Security Options:
apparmor
seccomp
Profile: default
Kernel Version: 4.4.0-62-generic
Operating System: Ubuntu 16.04.3 LTS
OSType: linux
Architecture: x86_64
CPUs: 2
Total Memory: 3.842GiB
Name: qemu-ubuntu
ID: LHOR:H7PC:4LPM:53LM:CU2A:TFJT:T5IX:UECZ:VN34:5BEC:QPBT:V4SB
Docker Root Dir: /var/lib/docker
Debug Mode (client): false
Debug Mode (server): true
File Descriptors: 17
Goroutines: 24
System Time: 2017-08-01T08:56:44.348308372-07:00
EventsListeners: 0
Registry: https://index.docker.io/v1/
Experimental: false
Insecure Registries:
127.0.0.0/8
Live Restore Enabled: false
WARNING: No swap limit supportRunning the container: $ sudo docker run -ti fedora bash
docker: Error response from daemon: json: cannot unmarshal number into Go value of type runtime.message.
|
|
@danielscottt, thanks for the quick response, seems you hit #1021 . Currently |
|
Yes! I just downgraded to 17.03, as it was what was available: Now I'm seeing this from And the runtime reports this:2017-08-01T17:00:56.394593Z:1862:cc-oci-runtime:debug:cc-oci-runtime 2.2.3 called as: /usr/bin/cc-oci-runtime create --bundle /var/run/docker/libcontainerd/7b7269c406cece41e7cfcaea6bc9261f58d4f9d9048dbb74c2fe5d77a9eb9451 --console-socket /tmp/pty664753388/pty.sock --pid-file /run/docker/libcontainerd/containerd/7b7269c406cece41e7cfcaea6bc9261f58d4f9d9048dbb74c2fe5d77a9eb9451/init/pid 7b7269c406cece41e7cfcaea6bc9261f58d4f9d9048dbb74c2fe5d77a9eb9451 2017-08-01T17:00:56.394784Z:1862:cc-oci-runtime:critical:/usr/bin/cc-oci-runtime: create: Unknown option --console-socket It looks like the rub is in this line: |
|
I attempted to run the qemu command generated by the runtime, and am getting this error: With a little Googling, I landed on this bug report on qemu, which states that VMware is likely disallowing the PMU MSR accesses, and previously this error was ignored. When using the suggested remedy of adding That said, I think it's pretty likely that this issue resides in either VMware or qemu, or is just straight-up a limitation of my VMware-based environment ¯_(ツ)_/¯. |
|
I'm going to close this for now and try and track down a physical host. Should the same issues arise, I'll reopen. Thanks for your help! |
|
This rang a bell - we've seen this before some time ago: #666 (comment) |
|
@danielscottt thank you!, looks like the VM is not working correctly, After do Can you provide more information about the guest that being used?:
Also in order to have hypervisor logs , could you also do: Then try to run again a container and see if there is something we could get from: |
|
Hey, thanks for the responses! I'll reopen. @grahamwhaley @mcastelino @anthonyzxu - I'd also love to know if there were any further developments! @jcvenegas - Thanks! I'll start gathering this info now. |
|
@danielscottt sorry, I did not seen your comments before last my comment, sure let us know if you find something interesting |
Guest memory is 4 GB: Yep, the hypervisor's |
jcvenegas
changed the title
|
Thanks for confirm, we will investigate more about it. In the in the meantime, I will add to our docs this limitation. |
|
@danielscottt We should have these fixed in 3.0 once this PR merges This containers/virtcontainers@dd13e0b#diff-82527fac73a6f9f5442c1db98ce1784aR589 fixes the PMU issue. There were two issues, A kvm bug for which we have a workaround and the PMU issue. I have verified this works on Workstation Pro 12 12.5.7 build-5813279 |
|
As a bit of an update, the root cause of the failure under VMware has been found over on CC3.x, and the fix is to turn off the pmu support on the qemu command line (I believe). |
|
(doh - thanks @mcastelino - beat me to it) |
A lot of what I've experienced thus far aligns with the issue relates to #950.
This is a VM running in VMware Fusion with VT passthrough enabled:
I installed following the Ubuntu installation directions and nothing failed during installation. However, when I run:
and
I see
and
respectively.
Thus far, I've been unable to track down any logs for
cc-oci-runtimeor thecc-proxy.The hardware precheck succeeds:
While following along in #950, I found that the qemu command used to execute the Clear Container. Here's mine, altered slightly to reflect version and name changes:
$ sudo /usr/bin/qemu-lite-system-x86_64 \ -name dan-test \ -machine pc-lite,accel=kvm,kernel_irqchip,nvdimm \ -object memory-backend-file,id=mem0,mem-path=/usr/share/clear-containers/clear-15550-containers.img,size=235929600 \ -kernel /usr/share/clear-containers/vmlinux-4.9.35-62.container \ -chardev stdio,id=stdio,mux=on \ -device virtio-serial-pci \ -device virtconsole,chardev=stdio \ -mon chardev=stdio \ -display noneLike in the aforementioned issue, this command hangs and qemu-lite pegs the CPU at 100%.
Thanks so much!
The text was updated successfully, but these errors were encountered: