Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Parallels driver support for darwin/arm64 #19373

Merged

Conversation

Razorr1996
Copy link
Contributor

fixes: #11219

minikube start --driver=parallels # creating new cluster
❯ minikube start --driver=parallels
😄  minikube v1.33.1 on Darwin 14.5 (arm64)
✨  Using the parallels driver based on user configuration
👍  Starting "minikube" primary control-plane node in "minikube" cluster
🔥  Creating parallels VM (CPUs=2, Memory=6000MB, Disk=20000MB) ...
🐳  Preparing Kubernetes v1.30.3 on Docker 27.1.1 ...
    ▪ Generating certificates and keys ...
    ▪ Booting up control plane ...
    ▪ Configuring RBAC rules ...
🔗  Configuring bridge CNI (Container Networking Interface) ...
🔎  Verifying Kubernetes components...
    ▪ Using image gcr.io/k8s-minikube/storage-provisioner:v5
🌟  Enabled addons: storage-provisioner, default-storageclass
🏄  Done! kubectl is now configured to use "minikube" cluster and "default" namespace by default
minikube status
❯ minikube status
minikube
type: Control Plane
host: Running
kubelet: Running
apiserver: Running
kubeconfig: Configured
kubectl get nodes -o wide
❯ kubectl get nodes -o wide
NAME       STATUS   ROLES           AGE   VERSION   INTERNAL-IP    EXTERNAL-IP   OS-IMAGE              KERNEL-VERSION   CONTAINER-RUNTIME
minikube   Ready    control-plane   18s   v1.30.3   10.211.55.76   <none>        Buildroot 2023.02.9   5.10.207         docker://27.1.1
kubectl describe nodes minikube
❯ kubectl describe nodes minikube
Name:               minikube
Roles:              control-plane
Labels:             beta.kubernetes.io/arch=arm64
                    beta.kubernetes.io/os=linux
                    kubernetes.io/arch=arm64
                    kubernetes.io/hostname=minikube
                    kubernetes.io/os=linux
                    minikube.k8s.io/commit=732b969929aa87c65157b4450d065c16abacfc41-dirty
                    minikube.k8s.io/name=minikube
                    minikube.k8s.io/primary=true
                    minikube.k8s.io/updated_at=2024_08_04T18_48_25_0700
                    minikube.k8s.io/version=v1.33.1
                    node-role.kubernetes.io/control-plane=
                    node.kubernetes.io/exclude-from-external-load-balancers=
Annotations:        kubeadm.alpha.kubernetes.io/cri-socket: unix:///var/run/cri-dockerd.sock
                    node.alpha.kubernetes.io/ttl: 0
                    volumes.kubernetes.io/controller-managed-attach-detach: true
CreationTimestamp:  Sun, 04 Aug 2024 18:48:22 +0200
Taints:             <none>
Unschedulable:      false
Lease:
  HolderIdentity:  minikube
  AcquireTime:     <unset>
  RenewTime:       Sun, 04 Aug 2024 18:48:44 +0200
Conditions:
  Type             Status  LastHeartbeatTime                 LastTransitionTime                Reason                       Message
  ----             ------  -----------------                 ------------------                ------                       -------
  MemoryPressure   False   Sun, 04 Aug 2024 18:48:45 +0200   Sun, 04 Aug 2024 18:48:21 +0200   KubeletHasSufficientMemory   kubelet has sufficient memory available
  DiskPressure     False   Sun, 04 Aug 2024 18:48:45 +0200   Sun, 04 Aug 2024 18:48:21 +0200   KubeletHasNoDiskPressure     kubelet has no disk pressure
  PIDPressure      False   Sun, 04 Aug 2024 18:48:45 +0200   Sun, 04 Aug 2024 18:48:21 +0200   KubeletHasSufficientPID      kubelet has sufficient PID available
  Ready            True    Sun, 04 Aug 2024 18:48:45 +0200   Sun, 04 Aug 2024 18:48:28 +0200   KubeletReady                 kubelet is posting ready status
Addresses:
  InternalIP:  10.211.55.76
  Hostname:    minikube
Capacity:
  cpu:                2
  ephemeral-storage:  17734596Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  hugepages-32Mi:     0
  hugepages-64Ki:     0
  memory:             5910208Ki
  pods:               110
Allocatable:
  cpu:                2
  ephemeral-storage:  17734596Ki
  hugepages-1Gi:      0
  hugepages-2Mi:      0
  hugepages-32Mi:     0
  hugepages-64Ki:     0
  memory:             5910208Ki
  pods:               110
System Info:
  Machine ID:                 b66cc9e9b275473192aba1fa28ec81ad
  System UUID:                54cb3b06-1738-4af0-8156-a35887ffe37b
  Boot ID:                    29d14d48-9f25-44c6-b124-6d1dcd6ea811
  Kernel Version:             5.10.207
  OS Image:                   Buildroot 2023.02.9
  Operating System:           linux
  Architecture:               arm64
  Container Runtime Version:  docker://27.1.1
  Kubelet Version:            v1.30.3
  Kube-Proxy Version:         v1.30.3
PodCIDR:                      10.244.0.0/24
PodCIDRs:                     10.244.0.0/24
Non-terminated Pods:          (7 in total)
  Namespace                   Name                                CPU Requests  CPU Limits  Memory Requests  Memory Limits  Age
  ---------                   ----                                ------------  ----------  ---------------  -------------  ---
  kube-system                 coredns-7db6d8ff4d-956br            100m (5%)     0 (0%)      70Mi (1%)        170Mi (2%)     14s
  kube-system                 etcd-minikube                       100m (5%)     0 (0%)      100Mi (1%)       0 (0%)         28s
  kube-system                 kube-apiserver-minikube             250m (12%)    0 (0%)      0 (0%)           0 (0%)         28s
  kube-system                 kube-controller-manager-minikube    200m (10%)    0 (0%)      0 (0%)           0 (0%)         28s
  kube-system                 kube-proxy-rxgt2                    0 (0%)        0 (0%)      0 (0%)           0 (0%)         14s
  kube-system                 kube-scheduler-minikube             100m (5%)     0 (0%)      0 (0%)           0 (0%)         29s
  kube-system                 storage-provisioner                 0 (0%)        0 (0%)      0 (0%)           0 (0%)         27s
Allocated resources:
  (Total limits may be over 100 percent, i.e., overcommitted.)
  Resource           Requests    Limits
  --------           --------    ------
  cpu                750m (37%)  0 (0%)
  memory             170Mi (2%)  170Mi (2%)
  ephemeral-storage  0 (0%)      0 (0%)
  hugepages-1Gi      0 (0%)      0 (0%)
  hugepages-2Mi      0 (0%)      0 (0%)
  hugepages-32Mi     0 (0%)      0 (0%)
  hugepages-64Ki     0 (0%)      0 (0%)
Events:
  Type    Reason                   Age   From             Message
  ----    ------                   ----  ----             -------
  Normal  Starting                 12s   kube-proxy
  Normal  NodeHasSufficientMemory  32s   kubelet          Node minikube status is now: NodeHasSufficientMemory
  Normal  Starting                 28s   kubelet          Starting kubelet.
  Normal  NodeAllocatableEnforced  28s   kubelet          Updated Node Allocatable limit across pods
  Normal  NodeHasSufficientMemory  28s   kubelet          Node minikube status is now: NodeHasSufficientMemory
  Normal  NodeHasNoDiskPressure    28s   kubelet          Node minikube status is now: NodeHasNoDiskPressure
  Normal  NodeHasSufficientPID     28s   kubelet          Node minikube status is now: NodeHasSufficientPID
  Normal  NodeReady                24s   kubelet          Node minikube status is now: NodeReady
  Normal  RegisteredNode           15s   node-controller  Node minikube event: Registered Node minikube in Controller
minikube --alsologtostderr -v=3 stop # stopping cluster
❯ minikube --alsologtostderr -v=3 stop
I0804 18:49:10.157781   74614 out.go:291] Setting OutFile to fd 1 ...
I0804 18:49:10.158322   74614 out.go:343] isatty.IsTerminal(1) = true
I0804 18:49:10.158330   74614 out.go:304] Setting ErrFile to fd 2...
I0804 18:49:10.158335   74614 out.go:343] isatty.IsTerminal(2) = true
I0804 18:49:10.158458   74614 root.go:338] Updating PATH: /Users/abasalaev/.minikube/bin
I0804 18:49:10.158730   74614 out.go:298] Setting JSON to false
I0804 18:49:10.158865   74614 mustload.go:65] Loading cluster: minikube
I0804 18:49:10.159066   74614 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:49:10.159113   74614 profile.go:143] Saving config to /Users/abasalaev/.minikube/profiles/minikube/config.json ...
I0804 18:49:10.159463   74614 mustload.go:65] Loading cluster: minikube
I0804 18:49:10.159540   74614 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:49:10.159753   74614 stop.go:39] StopHost: minikube
I0804 18:49:10.165336   74614 out.go:177] ✋  Stopping node "minikube"  ...
✋  Stopping node "minikube"  ...
I0804 18:49:10.173187   74614 machine.go:157] backing up vm config to /var/lib/minikube/backup: [/etc/cni /etc/kubernetes]
I0804 18:49:10.173282   74614 ssh_runner.go:195] Run: sudo mkdir -p /var/lib/minikube/backup
I0804 18:49:10.173528   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:49:10.295991   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl list -i minikube
I0804 18:49:10.482311   74614 main.go:141] libmachine: Found lease: 10.211.55.76 for MAC: 001C42FCF2F8, expiring at 1722791878, leased for 1800 s.

I0804 18:49:10.482355   74614 main.go:141] libmachine: Found IP lease: 10.211.55.76 for MAC address 001C42FCF2F8

I0804 18:49:10.482374   74614 sshutil.go:53] new ssh client: &{IP:10.211.55.76 Port:22 SSHKeyPath:/Users/abasalaev/.minikube/machines/minikube/id_rsa Username:docker}
I0804 18:49:10.507939   74614 ssh_runner.go:195] Run: sudo rsync --archive --relative /etc/cni /var/lib/minikube/backup
I0804 18:49:10.552969   74614 ssh_runner.go:195] Run: sudo rsync --archive --relative /etc/kubernetes /var/lib/minikube/backup
I0804 18:49:10.600149   74614 main.go:141] libmachine: Stopping "minikube"...
I0804 18:49:10.600192   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:49:10.722496   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl stop minikube
I0804 18:50:09.749292   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:50:10.822492   74614 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:50:10.949578   74614 main.go:141] libmachine: Machine "minikube" was stopped.
I0804 18:50:10.949606   74614 stop.go:75] duration metric: took 1m0.775807167s to stop
I0804 18:50:10.952477   74614 lock.go:35] WriteFile acquiring /Users/abasalaev/.kube/config: {Name:mk2511360dcfc3b3142e141744412eed433ed661 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0804 18:50:10.957659   74614 out.go:177] 🛑  1 node stopped.
🛑  1 node stopped.
minikube --alsologtostderr -v=3 delete # deleting cluster
❯ minikube --alsologtostderr -v=3 delete
I0804 18:51:32.873074   76402 out.go:291] Setting OutFile to fd 1 ...
I0804 18:51:32.873454   76402 out.go:343] isatty.IsTerminal(1) = true
I0804 18:51:32.873462   76402 out.go:304] Setting ErrFile to fd 2...
I0804 18:51:32.873466   76402 out.go:343] isatty.IsTerminal(2) = true
I0804 18:51:32.873586   76402 root.go:338] Updating PATH: /Users/abasalaev/.minikube/bin
I0804 18:51:32.873854   76402 out.go:298] Setting JSON to false
I0804 18:51:32.874039   76402 cli_runner.go:164] Run: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}
I0804 18:51:50.601967   76402 cli_runner.go:217] Completed: docker ps -a --filter label=name.minikube.sigs.k8s.io --format {{.Names}}: (17.72766875s)
I0804 18:51:50.605109   76402 config.go:182] Loaded profile config "bridge-193000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.30.2
I0804 18:51:50.605414   76402 config.go:182] Loaded profile config "flannel-193000": Driver=docker, ContainerRuntime=docker, KubernetesVersion=v1.30.2
I0804 18:51:50.605530   76402 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:51:50.605664   76402 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:51:50.605691   76402 delete.go:301] DeleteProfiles
I0804 18:51:50.605702   76402 delete.go:329] Deleting minikube
I0804 18:51:50.605713   76402 delete.go:334] minikube configuration: &{Name:minikube KeepContext:false EmbedCerts:false MinikubeISO:https://storage.googleapis.com/minikube-builds/iso/19339/minikube-v1.33.1-1722248113-19339-arm64.iso KicBaseImage:gcr.io/k8s-minikube/kicbase-builds:v0.0.44-1721902582-19326@sha256:540fb5dc7f38be17ff5276a38dfe6c8a4b1d9ba1c27c62244e6eebd7e37696e7 Memory:6000 CPUs:2 DiskSize:20000 Driver:parallels HyperkitVpnKitSock: HyperkitVSockPorts:[] DockerEnv:[] ContainerVolumeMounts:[] InsecureRegistry:[] RegistryMirror:[] HostOnlyCIDR:192.168.59.1/24 HypervVirtualSwitch: HypervUseExternalSwitch:false HypervExternalAdapter: KVMNetwork:default KVMQemuURI:qemu:///system KVMGPU:false KVMHidden:false KVMNUMACount:1 APIServerPort:8443 DockerOpt:[] DisableDriverMounts:false NFSShare:[] NFSSharesRoot:/nfsshares UUID: NoVTXCheck:false DNSProxy:false HostDNSResolver:true HostOnlyNicType:virtio NatNicType:virtio SSHIPAddress: SSHUser:root SSHKey: SSHPort:22 KubernetesConfig:{KubernetesVersion:v1.30.3 ClusterName:minikube Namespace:default APIServerHAVIP: APIServerName:minikubeCA APIServerNames:[] APIServerIPs:[] DNSDomain:cluster.local ContainerRuntime:docker CRISocket: NetworkPlugin:cni FeatureGates: ServiceCIDR:10.96.0.0/12 ImageRepository: LoadBalancerStartIP: LoadBalancerEndIP: CustomIngressCert: RegistryAliases: ExtraOptions:[] ShouldLoadCachedImages:true EnableDefaultCNI:false CNI:} Nodes:[{Name: IP:10.211.55.76 Port:8443 KubernetesVersion:v1.30.3 ContainerRuntime:docker ControlPlane:true Worker:true}] Addons:map[default-storageclass:true storage-provisioner:true] CustomAddonImages:map[] CustomAddonRegistries:map[] VerifyComponents:map[apiserver:true system_pods:true] StartHostTimeout:6m0s ScheduledStop:<nil> ExposedPorts:[] ListenAddress: Network: Subnet: MultiNodeRequested:false ExtraDisks:0 CertExpiration:26280h0m0s Mount:false MountString:/Users:/minikube-host Mount9PVersion:9p2000.L MountGID:docker MountIP: MountMSize:262144 MountOptions:[] MountPort:0 MountType:9p MountUID:docker BinaryMirror: DisableOptimizations:false DisableMetrics:false CustomQemuFirmwarePath: SocketVMnetClientPath: SocketVMnetPath: StaticIP: SSHAuthSock: SSHAgentPID:0 GPUs: AutoPauseInterval:1m0s}
I0804 18:51:50.605900   76402 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:51:50.606229   76402 config.go:182] Loaded profile config "minikube": Driver=parallels, ContainerRuntime=docker, KubernetesVersion=v1.30.3
I0804 18:51:50.607063   76402 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:51:50.738839   76402 out.go:177] 🔥  Deleting "minikube" in parallels ...
🔥  Deleting "minikube" in parallels ...
I0804 18:51:50.744500   76402 main.go:141] libmachine: executing: /usr/local/bin/prlctl list minikube --output status --no-header
I0804 18:51:50.886207   76402 main.go:141] libmachine: executing: /usr/local/bin/prlctl stop minikube --kill
I0804 18:51:52.233598   76402 main.go:141] libmachine: executing: /usr/local/bin/prlctl delete minikube
I0804 18:51:52.371577   76402 lock.go:35] WriteFile acquiring /Users/abasalaev/.kube/config: {Name:mk2511360dcfc3b3142e141744412eed433ed661 Clock:{} Delay:500ms Timeout:1m0s Cancel:<nil>}
I0804 18:51:52.379028   76402 out.go:177] 💀  Removed all traces of the "minikube" cluster.
💀  Removed all traces of the "minikube" cluster.

@k8s-ci-robot k8s-ci-robot added cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 4, 2024
@k8s-ci-robot
Copy link
Contributor

Hi @Razorr1996. Thanks for your PR.

I'm waiting for a kubernetes member to verify that this patch is reasonable to test. If it is, they should reply with /ok-to-test on its own line. Until that is done, I will not automatically test new commits in this PR, but the usual testing commands by org members will still work. Regular contributors should join the org to skip this step.

Once the patch is verified, the new status will be reflected by the ok-to-test label.

I understand the commands that are listed here.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository.

@k8s-ci-robot k8s-ci-robot added the size/XS Denotes a PR that changes 0-9 lines, ignoring generated files. label Aug 4, 2024
@minikube-bot
Copy link
Collaborator

Can one of the admins verify this patch?

@Razorr1996
Copy link
Contributor Author

/assign

@Razorr1996
Copy link
Contributor Author

/cc @afbjorklund

@medyagh
Copy link
Member

medyagh commented Aug 5, 2024

/ok-to-test

@k8s-ci-robot k8s-ci-robot added ok-to-test Indicates a non-member PR verified by an org member that is safe to test. and removed needs-ok-to-test Indicates a PR that requires an org member to verify it is safe to test. labels Aug 5, 2024
@minikube-pr-bot
Copy link

kvm2 driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19373) |
+----------------+----------+---------------------+
| minikube start | 54.4s    | 54.1s               |
| enable ingress | 27.3s    | 27.3s               |
+----------------+----------+---------------------+

Times for minikube ingress: 29.8s 27.6s 27.7s 23.7s 27.7s
Times for minikube (PR 19373) ingress: 27.1s 26.2s 27.6s 29.3s 26.2s

Times for minikube start: 51.8s 56.0s 53.3s 56.7s 54.2s
Times for minikube (PR 19373) start: 52.5s 56.6s 54.5s 54.0s 53.0s

docker driver with docker runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19373) |
+----------------+----------+---------------------+
| minikube start | 24.8s    | 24.5s               |
| enable ingress | 21.9s    | 21.8s               |
+----------------+----------+---------------------+

Times for minikube ingress: 22.4s 21.4s 21.9s 21.9s 21.9s
Times for minikube (PR 19373) ingress: 21.9s 21.9s 21.9s 21.9s 21.4s

Times for minikube start: 26.3s 23.1s 25.3s 23.6s 25.6s
Times for minikube (PR 19373) start: 25.6s 27.2s 22.8s 23.0s 23.7s

docker driver with containerd runtime

+----------------+----------+---------------------+
|    COMMAND     | MINIKUBE | MINIKUBE (PR 19373) |
+----------------+----------+---------------------+
| minikube start | 24.3s    | 22.7s               |
| enable ingress | 47.9s    | 40.6s               |
+----------------+----------+---------------------+

Times for minikube start: 25.3s 23.2s 24.7s 22.4s 26.0s
Times for minikube (PR 19373) start: 25.0s 22.0s 21.0s 22.5s 22.8s

Times for minikube ingress: 48.9s 47.9s 46.9s 46.9s 48.9s
Times for minikube (PR 19373) ingress: 40.1s 31.9s 48.9s 47.9s 34.0s

@medyagh
Copy link
Member

medyagh commented Aug 6, 2024

/lgtm

@medyagh medyagh merged commit ee4532d into kubernetes:master Aug 6, 2024
36 of 47 checks passed
@k8s-ci-robot k8s-ci-robot added the lgtm "Looks good to me", indicates that a PR is ready to be merged. label Aug 6, 2024
@k8s-ci-robot
Copy link
Contributor

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: medyagh, Razorr1996

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@k8s-ci-robot k8s-ci-robot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Aug 6, 2024
@minikube-pr-bot
Copy link

Here are the number of top 10 failed tests in each environments with lowest flake rate.

Environment Test Name Flake Rate
Docker_Linux_crio_arm64 (5 failed) TestFunctional/parallel/TunnelCmd/serial/WaitService/Setup(gopogh) 0.00% (chart)
Docker_Linux_crio_arm64 (5 failed) TestFunctional/parallel/TunnelCmd/serial/AccessDirect(gopogh) 0.00% (chart)
Docker_Linux_crio_arm64 (5 failed) TestFunctional/parallel/PersistentVolumeClaim(gopogh) 2.37% (chart)
Docker_Linux_containerd_arm64 (2 failed) TestStartStop/group/old-k8s-version/serial/SecondStart(gopogh) 46.75% (chart)

Besides the following environments also have failed tests:

To see the flake rates of all tests by environment, click here.

@Razorr1996 Razorr1996 deleted the add-parallels-driver-for-darwin-arm64 branch August 6, 2024 08:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
approved Indicates a PR has been approved by an approver from all required OWNERS files. cncf-cla: yes Indicates the PR's author has signed the CNCF CLA. lgtm "Looks good to me", indicates that a PR is ready to be merged. ok-to-test Indicates a non-member PR verified by an org member that is safe to test. size/XS Denotes a PR that changes 0-9 lines, ignoring generated files.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

starting minikube on arm64 Mac with Parallels
5 participants