Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[BUG]yurt-hub restarts abnormally, and an error occurs on the yurt-manager #1670

Closed
yuluxs007 opened this issue Aug 23, 2023 · 11 comments
Closed

[BUG]yurt-hub restarts abnormally, and an error occurs on the yurt-manager #1670

yuluxs007 opened this issue Aug 23, 2023 · 11 comments
Labels
kind/bug kind/bug wontfix

Comments

@yuluxs007
Copy link

What happened:
yurt-hub restarts abnormally, and an error occurs on the yurt-manager

Error log[yurt-hub]:

yurthub version: projectinfo.Info{GitVersion:"-d15078f", GitCommit:"d15078f", BuildDate:"2023-08-21T01:47:19Z", GoVersion:"go1.18.10", Compiler:"gc", Platform:"linux/amd64", AllVersions:[]string{""}}
I0822 02:15:25.410040       1 start.go:66] FLAG: --access-server-through-hub="true"
I0822 02:15:25.410266       1 start.go:66] FLAG: --add_dir_header="false"
I0822 02:15:25.410302       1 start.go:66] FLAG: --alsologtostderr="false"
I0822 02:15:25.410337       1 start.go:66] FLAG: --bind-address="127.0.0.1"
I0822 02:15:25.410397       1 start.go:66] FLAG: --bind-proxy-address="127.0.0.1"
I0822 02:15:25.410426       1 start.go:66] FLAG: --bootstrap-file=""
I0822 02:15:25.410445       1 start.go:66] FLAG: --bootstrap-mode="token"
I0822 02:15:25.410475       1 start.go:66] FLAG: --coordinator-server-addr="https://yurt-coordinator-apiserver:443"
I0822 02:15:25.410501       1 start.go:66] FLAG: --coordinator-storage-addr="https://yurt-coordinator-etcd:2379"
I0822 02:15:25.410531       1 start.go:66] FLAG: --coordinator-storage-prefix="/registry"
I0822 02:15:25.410564       1 start.go:66] FLAG: --daemonpodupdater-workers="3"
I0822 02:15:25.410586       1 start.go:66] FLAG: --disabled-resource-filters="[]"
I0822 02:15:25.410671       1 start.go:66] FLAG: --discovery-token-ca-cert-hash="[]"
I0822 02:15:25.410706       1 start.go:66] FLAG: --discovery-token-unsafe-skip-ca-verification="true"
I0822 02:15:25.410735       1 start.go:66] FLAG: --disk-cache-path="/etc/kubernetes/cache/"
I0822 02:15:25.410757       1 start.go:66] FLAG: --dummy-if-ip=""
I0822 02:15:25.410786       1 start.go:66] FLAG: --dummy-if-name="yurthub-dummy0"
I0822 02:15:25.410815       1 start.go:66] FLAG: --enable-coordinator="false"
I0822 02:15:25.410837       1 start.go:66] FLAG: --enable-dummy-if="true"
I0822 02:15:25.410857       1 start.go:66] FLAG: --enable-iptables="true"
I0822 02:15:25.410889       1 start.go:66] FLAG: --enable-node-pool="true"
I0822 02:15:25.410911       1 start.go:66] FLAG: --enable-resource-filter="true"
I0822 02:15:25.410941       1 start.go:66] FLAG: --gc-frequency="120"
I0822 02:15:25.410974       1 start.go:66] FLAG: --heartbeat-failed-retry="3"
I0822 02:15:25.410994       1 start.go:66] FLAG: --heartbeat-healthy-threshold="2"
I0822 02:15:25.411014       1 start.go:66] FLAG: --heartbeat-interval-seconds="10"
I0822 02:15:25.411049       1 start.go:66] FLAG: --heartbeat-timeout-seconds="2"
I0822 02:15:25.411073       1 start.go:66] FLAG: --help="false"
I0822 02:15:25.411174       1 start.go:66] FLAG: --hub-cert-organizations="[]"
I0822 02:15:25.411214       1 start.go:66] FLAG: --join-token="eyJWdioWP60StgIrhF7qXKGLyU7pj42mVNdx81CEwqCZdATDUOZwg"
I0822 02:15:25.411335       1 start.go:66] FLAG: --kubeconfig=""
I0822 02:15:25.411368       1 start.go:66] FLAG: --kubelet-health-grace-period="40s"
I0822 02:15:25.411413       1 start.go:66] FLAG: --lb-mode="rr"
I0822 02:15:25.411436       1 start.go:66] FLAG: --leader-elect="true"
I0822 02:15:25.411470       1 start.go:66] FLAG: --leader-elect-lease-duration="15s"
I0822 02:15:25.411495       1 start.go:66] FLAG: --leader-elect-renew-deadline="10s"
I0822 02:15:25.411518       1 start.go:66] FLAG: --leader-elect-resource-lock="leases"
I0822 02:15:25.411548       1 start.go:66] FLAG: --leader-elect-resource-name="yurthub"
I0822 02:15:25.411571       1 start.go:66] FLAG: --leader-elect-resource-namespace="kube-system"
I0822 02:15:25.411619       1 start.go:66] FLAG: --leader-elect-retry-period="2s"
I0822 02:15:25.411646       1 start.go:66] FLAG: --log-flush-frequency="5s"
I0822 02:15:25.411668       1 start.go:66] FLAG: --log_backtrace_at=":0"
I0822 02:15:25.411708       1 start.go:66] FLAG: --log_dir=""
I0822 02:15:25.411742       1 start.go:66] FLAG: --log_file=""
I0822 02:15:25.411763       1 start.go:66] FLAG: --log_file_max_size="1800"
I0822 02:15:25.411785       1 start.go:66] FLAG: --logtostderr="true"
I0822 02:15:25.411818       1 start.go:66] FLAG: --max-requests-in-flight="250"
I0822 02:15:25.411842       1 start.go:66] FLAG: --min-request-timeout="30m0s"
I0822 02:15:25.411873       1 start.go:66] FLAG: --namespace="kube-system"
I0822 02:15:25.411905       1 start.go:66] FLAG: --node-name="n2.iot1.iotdc1.uat"
I0822 02:15:25.411929       1 start.go:66] FLAG: --nodepool-name=""
I0822 02:15:25.411948       1 start.go:66] FLAG: --one_output="false"
I0822 02:15:25.411974       1 start.go:66] FLAG: --profiling="true"
I0822 02:15:25.411997       1 start.go:66] FLAG: --proxy-port="10261"
I0822 02:15:25.412030       1 start.go:66] FLAG: --proxy-secure-port="10268"
I0822 02:15:25.412065       1 start.go:66] FLAG: --root-dir="/var/lib/yurthub"
I0822 02:15:25.412088       1 start.go:66] FLAG: --serve-port="10267"
I0822 02:15:25.412157       1 start.go:66] FLAG: --server-addr="https://10.10.14.168:6443"
I0822 02:15:25.412181       1 start.go:66] FLAG: --skip_headers="false"
I0822 02:15:25.412203       1 start.go:66] FLAG: --skip_log_headers="false"
I0822 02:15:25.412236       1 start.go:66] FLAG: --stderrthreshold="2"
I0822 02:15:25.412267       1 start.go:66] FLAG: --v="2"
I0822 02:15:25.412287       1 start.go:66] FLAG: --version="false"
I0822 02:15:25.412306       1 start.go:66] FLAG: --vmodule=""
I0822 02:15:25.412328       1 start.go:66] FLAG: --working-mode="edge"
I0822 02:15:25.412433       1 options.go:260] dummy ip not set, will use 169.254.2.1 as default
I0822 02:15:25.412557       1 config.go:232] yurthub would connect remote servers: https://10.10.14.168:6443
I0822 02:15:25.412811       1 storage.go:86] yurthub disk storage will run in enhancement mode
I0822 02:15:25.414488       1 filter.go:96] Filter servicetopology registered successfully
I0822 02:15:25.414539       1 filter.go:96] Filter masterservice registered successfully
I0822 02:15:25.414556       1 filter.go:96] Filter discardcloudservice registered successfully
I0822 02:15:25.414569       1 filter.go:96] Filter inclusterconfig registered successfully
I0822 02:15:25.414587       1 filter.go:96] Filter nodeportisolation registered successfully
I0822 02:15:25.414604       1 filter.go:96] Filter hostnetworkpropagation registered successfully
I0822 02:15:25.414729       1 filter.go:76] Filter servicetopology initialize successfully
I0822 02:15:25.414763       1 filter.go:76] Filter masterservice initialize successfully
I0822 02:15:25.414785       1 filter.go:76] Filter discardcloudservice initialize successfully
I0822 02:15:25.414811       1 filter.go:76] Filter inclusterconfig initialize successfully
I0822 02:15:25.414831       1 filter.go:76] Filter nodeportisolation initialize successfully
I0822 02:15:25.414859       1 filter.go:76] Filter hostnetworkpropagation initialize successfully
I0822 02:15:25.415052       1 approver.go:191] current filter setting: map[coredns/endpoints/list:map[servicetopology:{}] coredns/endpoints/watch:map[servicetopology:{}] coredns/endpointslices/list:map[servicetopology:{}] coredns/endpointslices/watch:map[servicetopology:{}] kube-proxy/endpoints/list:map[servicetopology:{}] kube-proxy/endpoints/watch:map[servicetopology:{}] kube-proxy/endpointslices/list:map[servicetopology:{}] kube-proxy/endpointslices/watch:map[servicetopology:{}] kube-proxy/services/list:map[discardcloudservice:{} nodeportisolation:{}] kube-proxy/services/watch:map[discardcloudservice:{} nodeportisolation:{}] kubelet/configmaps/get:map[inclusterconfig:{}] kubelet/configmaps/list:map[inclusterconfig:{}] kubelet/configmaps/watch:map[inclusterconfig:{}] kubelet/pods/list:map[hostnetworkpropagation:{}] kubelet/pods/watch:map[hostnetworkpropagation:{}] kubelet/services/list:map[masterservice:{}] kubelet/services/watch:map[masterservice:{}] nginx-ingress-controller/endpoints/list:map[servicetopology:{}] nginx-ingress-controller/endpoints/watch:map[servicetopology:{}] nginx-ingress-controller/endpointslices/list:map[servicetopology:{}] nginx-ingress-controller/endpointslices/watch:map[servicetopology:{}]] after init
W0822 02:15:25.415500       1 filestore_wrapper.go:49] unexpected error occurred when loading the certificate: no cert/key files read at "/var/lib/yurthub/yurthub/pki/yurthub-current.pem", ("", "") or ("/var/lib/yurthub/yurthub/pki", "/var/lib/yurthub/yurthub/pki"), will regenerate it
W0822 02:15:25.415618       1 filestore_wrapper.go:49] unexpected error occurred when loading the certificate: no cert/key files read at "/var/lib/yurthub/yurthub/pki/yurthub-server-current.pem", ("", "") or ("/var/lib/yurthub/yurthub/pki", "/var/lib/yurthub/yurthub/pki"), will regenerate it
E0822 02:15:25.415818       1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\\A([a-z0-9]{6})\\.([a-z0-9]{16})\\z"
I0822 02:15:25.415919       1 certificate_manager.go:318] kubernetes.io/kubelet-serving: Certificate rotation is enabled
E0822 02:15:25.416031       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0822 02:15:25.416198       1 certificate_manager.go:446] kubernetes.io/kubelet-serving: Rotating certificates
E0822 02:15:30.416742       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0822 02:15:30.417969       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0822 02:15:35.416734       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0822 02:15:35.417970       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0822 02:15:40.416317       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0822 02:15:40.417407       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0822 02:15:45.417018       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0822 02:15:45.418509       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
F0822 02:19:25.416843       1 start.go:74] complete yurthub configuration error, hub certificates preparation failed, timed out waiting for the condition
goroutine 1 [running]:
k8s.io/klog/v2.stacks(0x1)
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1026 +0x8a
k8s.io/klog/v2.(*loggingT).output(0x306ef80, 0x3, {0x0, 0x0}, 0xc00063fc00, 0x0, {0x263bfe7?, 0xc0009204d0?}, 0x0?, 0x0)
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:975 +0x648
k8s.io/klog/v2.(*loggingT).printf(0x1e1109d?, 0x77bc10?, {0x0, 0x0}, {0x0, 0x0}, {0x1e452ad, 0x23}, {0xc0003de5a0, 0x2, ...})
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:753 +0x1e5
k8s.io/klog/v2.Fatalf(...)
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1514
github.com/openyurtio/openyurt/cmd/yurthub/app.NewCmdStartYurtHub.func1(0xc0005fa300?, {0x1e11a31?, 0x4?, 0x4?})
	/build/cmd/yurthub/app/start.go:74 +0x39d
github.com/spf13/cobra.(*Command).execute(0xc0005fa300, {0xc00004c0b0, 0x4, 0x4})
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:944 +0x847
github.com/spf13/cobra.(*Command).ExecuteC(0xc0005fa300)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:1068 +0x3bc
github.com/spf13/cobra.(*Command).Execute(...)
	/go/pkg/mod/github.com/spf13/[email protected]/command.go:992
main.main()
	/build/cmd/yurthub/yurthub.go:35 +0x194

goroutine 4 [chan receive]:
k8s.io/klog/v2.(*loggingT).flushDaemon(0x0?)
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:1169 +0x6a
created by k8s.io/klog/v2.init.0
	/go/pkg/mod/k8s.io/klog/[email protected]/klog.go:420 +0xf6

goroutine 137 [syscall, 4 minutes]:
os/signal.signal_recv()
	/usr/local/go/src/runtime/sigqueue.go:151 +0x2f
os/signal.loop()
	/usr/local/go/src/os/signal/signal_unix.go:23 +0x19
created by os/signal.Notify.func1.1
	/usr/local/go/src/os/signal/signal.go:151 +0x2a

goroutine 147 [chan receive, 4 minutes]:
k8s.io/apiserver/pkg/server.SetupSignalContext.func1()
	/go/pkg/mod/k8s.io/[email protected]/pkg/server/signal.go:48 +0x2b
created by k8s.io/apiserver/pkg/server.SetupSignalContext
	/go/pkg/mod/k8s.io/[email protected]/pkg/server/signal.go:47 +0xe5

goroutine 148 [select]:
k8s.io/apimachinery/pkg/util/wait.WaitForWithContext({0x210ed60, 0xc00005a020}, 0xc0002f6708, 0xb1b0ca?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:655 +0xe5
k8s.io/apimachinery/pkg/util/wait.poll({0x210ed60, 0xc00005a020}, 0x98?, 0xb19d25?, 0x18?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:591 +0x9a
k8s.io/apimachinery/pkg/util/wait.PollInfiniteWithContext({0x210ed60, 0xc00005a020}, 0x40d947?, 0x18?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:495 +0x46
k8s.io/apimachinery/pkg/util/wait.PollInfinite(0x1a4?, 0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:485 +0x46
github.com/openyurtio/openyurt/pkg/yurthub/certificate/server.NewHubServerCertificateManager.func1(0xc000149d00?)
	/build/pkg/yurthub/certificate/server/server.go:54 +0x8f
k8s.io/client-go/util/certificate.(*manager).getClientset(0xc000149d00)
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:429 +0xb7
k8s.io/client-go/util/certificate.(*manager).rotateCerts(0xc000149d00)
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:458 +0x23e
k8s.io/apimachinery/pkg/util/wait.ConditionFunc.WithContext.func1({0x1bd8640, 0x1})
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:217 +0x1b
k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtectionWithContext({0x210ed60?, 0xc00005a028?}, 0x5940682?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:230 +0x57
k8s.io/apimachinery/pkg/util/wait.runConditionWithCrashProtection(0x1bd2bc0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:223 +0x39
k8s.io/apimachinery/pkg/util/wait.ExponentialBackoff({0x77359400, 0x4000000000000000, 0x3fb999999999999a, 0x5, 0x0}, 0x306eae0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:418 +0x5f
k8s.io/client-go/util/certificate.(*manager).Start.func1()
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:353 +0x3d7
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x10000000000?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x3e
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0?, {0x20f25c0, 0xc0003e2000}, 0x1, 0xc000098540)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0xb6
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x0?, 0x3b9aca00, 0x0, 0x0?, 0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x89
k8s.io/apimachinery/pkg/util/wait.Until(0x0?, 0x0?, 0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x25
created by k8s.io/client-go/util/certificate.(*manager).Start
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:321 +0x178

goroutine 149 [select, 4 minutes]:
k8s.io/client-go/util/certificate.(*manager).Start.func2()
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:371 +0xda
k8s.io/apimachinery/pkg/util/wait.BackoffUntil.func1(0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:155 +0x3e
k8s.io/apimachinery/pkg/util/wait.BackoffUntil(0x0?, {0x20f25c0, 0xc0007e2e10}, 0x1, 0xc000098540)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:156 +0xb6
k8s.io/apimachinery/pkg/util/wait.JitterUntil(0x0?, 0x3b9aca00, 0x0, 0x0?, 0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:133 +0x89
k8s.io/apimachinery/pkg/util/wait.Until(0x0?, 0x0?, 0x0?)
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:90 +0x25
created by k8s.io/client-go/util/certificate.(*manager).Start
	/go/pkg/mod/k8s.io/[email protected]/util/certificate/certificate_manager.go:360 +0x23b

goroutine 138 [select]:
k8s.io/apimachinery/pkg/util/wait.poller.func1.1()
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:705 +0x1c5
created by k8s.io/apimachinery/pkg/util/wait.poller.func1
	/go/pkg/mod/k8s.io/[email protected]/pkg/util/wait/wait.go:688 +0xca

Error log[yurt-manager]:

0822 09:27:03.961073       1 podbinding_controller.go:92] node request: n2.iot1.iotdc1.uat
I0822 09:27:03.985728       1 podbinding_controller.go:92] node request: m.iot1.iotdc1.uat
I0822 09:27:03.986100       1 nodepool_enqueue_handlers.go:70] nodepool: node(m.iot1.iotdc1.uat) does not belong to any nodepool
I0822 09:27:04.713932       1 yurtstaticset_controller.go:281] yurtstaticset: Reconcile YurtStaticSet yurt-hub
E0822 09:27:04.717336       1 yurtstaticset_controller.go:338] fail to init worker pod info, cause worker pod yss-upgrade-worker-yurt-hub-n2.iot1.iotdc1.uat-67dc9678cf failed
E0822 09:27:04.717432       1 controller.go:317] setup/controller/yurtstaticset "msg"="Reconciler error" "error"="fail to init worker pod info, cause worker pod yss-upgrade-worker-yurt-hub-n2.iot1.iotdc1.uat-67dc9678cf failed" "name"="yurt-hub" "namespace"="kube-system" 
I0822 09:27:04.717562       1 recorder.go:103] setup/events "msg"="Warning"  "message"="fail to init worker pod info, cause worker pod yss-upgrade-worker-yurt-hub-n2.iot1.iotdc1.uat-67dc9678cf failed" "object"={"kind":"YurtStaticSet","namespace":"kube-system","name":"yurt-hub","uid":"c9c813fb-5ef5-4b98-a7c2-7f57dfe7e79a","apiVersion":"apps.openyurt.io/v1alpha1","resourceVersion":"758367"} "reason"="YurtStaticSet Upgrade Failed"
W0822 09:27:09.249763       1 configuration.go:89] Ignore webhook for /validate-iot-openyurt-io-v1alpha2-platformadmin in configuration
I0822 09:27:13.993917       1 podbinding_controller.go:92] node request: n1.iot1.iotdc1.uat
I0822 09:27:14.041864       1 podbinding_controller.go:92] node request: n2.iot1.iotdc1.uat
I0822 09:27:14.101920       1 podbinding_controller.go:92] node request: m.iot1.iotdc1.uat
I0823 02:22:36.927543       1 podbinding_controller.go:92] node request: n1.iot1.iotdc1.uat
I0823 02:22:36.928053       1 podbinding_controller.go:92] node request: n2.iot1.iotdc1.uat
I0823 02:22:37.016222       1 nodepool_enqueue_handlers.go:70] nodepool: node(m.iot1.iotdc1.uat) does not belong to any nodepool
I0823 02:22:37.017283       1 podbinding_controller.go:92] node request: m.iot1.iotdc1.uat
W0823 02:22:41.727487       1 configuration.go:89] Ignore webhook for /validate-iot-openyurt-io-v1alpha2-platformadmin in configuration
I0823 02:22:47.026362       1 podbinding_controller.go:92] node request: n2.iot1.iotdc1.uat
I0823 02:22:47.027110       1 podbinding_controller.go:92] node request: n1.iot1.iotdc1.uat
I0823 02:22:47.119123       1 nodepool_enqueue_handlers.go:70] nodepool: node(m.iot1.iotdc1.uat) does not belong to any nodepool
I0823 02:22:47.119357       1 podbinding_controller.go:92] node request: m.iot1.iotdc1.uat
I0823 02:22:48.678294       1 reflector.go:535] pkg/mod/k8s.io/[email protected]/tools/cache/reflector.go:167: Watch close - *v1.Secret total 0 items received
[root@m ~]# kubectl logs -n kube-system yurt-hub-n2.iot1.iotdc1.uat --tail=50
E0823 02:21:12.273036       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:12.274338       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:17.272835       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:17.274097       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:22.272912       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:22.274157       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:27.272610       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:27.273857       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:32.273456       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:32.274688       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:37.273390       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]
I0823 02:21:37.274805       1 server.go:59] waiting for the controller-manager to sign the yurthub client certificate
E0823 02:21:42.273502       1 manager.go:133] hub certificates are not ready: [APIServer client certificate, ca.crt file, hub server certificate]

What you expected to happen:
I expect both of the above components to work properly with no exception logs

Anything else we need to know?:
k8s 版本为1.21 ,openyurt 版本为 v1.3

Environment:

  • OpenYurt version: v1.3
  • Kubernetes version (use kubectl version): v1.21.6
  • OS (e.g: cat /etc/os-release): CentOS Linux 7 (Core)
  • Kernel (e.g. uname -a):3.10.0-1160.21.1.el7.x86_64
  • Install tools:
  • Others:

others

/kind bug
@yuluxs007 yuluxs007 added the kind/bug kind/bug label Aug 23, 2023
@rambohe-ch
Copy link
Member

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6})\.([a-z0-9]{16})\z"

@rambohe-ch
Copy link
Member

rambohe-ch commented Aug 23, 2023 •
edited
Loading

@YTGhost It seems we should add a entry check for join token(such as format check) in yurtadm join command. If the join token is invalid, yurtadm join command can short-circuit and return error message.

@YTGhost
Copy link
Member

YTGhost commented Aug 23, 2023 •
edited
Loading

@YTGhost It seems we should add a entry check for join token(such as format check) in yurtadm join command. If the join token is invalid, yurtadm join command can short-circuit and return error message.

@rambohe-ch I think you're right, even though it's not in kubeadm's pre-checks, I think we should also check the token's format in yurtadm's pre-checks, I'll raise an new issue about that in a moment.

@yuluxs007
Copy link
Author

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6}).([a-z0-9]{16})\z"

What do I need to do to make the token valid, I followed the v1.3 documentation, Please give a suggestion or provide a link to relevant documents.

@YTGhost
Copy link
Member

YTGhost commented Aug 23, 2023 •
edited
Loading

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6}).([a-z0-9]{16})\z"

What do I need to do to make the token valid, I followed the v1.3 documentation, Please give a suggestion or provide a link to relevant documents.

@yuluxs007 Because yurtadm integrates with kubeadm, currently supports tokens in the form to match [a-z0-9]{6}.[a-z0-9]{16}, FYI: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/#synopsis

@yuluxs007
Copy link
Author

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6}).([a-z0-9]{16})\z"

What do I need to do to make the token valid, I followed the v1.3 documentation, Please give a suggestion or provide a link to relevant documents.

@yuluxs007 Because yurtadm integrates with kubeadm, currently supports tokens in the form to match [a-z0-9]{6}.[a-z0-9]{16}, FYI: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/#synopsis

Create bootstrap tokens on the server
[root@m kubernetes]# kubeadm token create --print-join-command
kubeadm join 127.0.0.1:6443 --token mz5k5t.4hgpzg6weuf07cf0 --discovery-token-ca-cert-hash sha256:113cdaed12905c6152b45030146738aab95b006c019d3f7331a48e49e1c23425
[root@m kubernetes]# kubeadm token list
TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS
mz5k5t.4hgpzg6weuf07cf0 23h 2023-08-24T06:36:48Z authentication,signing system:bootstrappers:kubeadm:default-node-token

yurthub.yaml information

    command:
    - yurthub
    - --v=2
    - --server-addr=https://10.10.14.168:6443
    - --node-name=$(NODE_NAME)
    - --join-token=mz5k5t.4hgpzg6weuf07cf0

yurt-hub err log

E0823 07:05:13.186877       1 cache_manager.go:392] yurthub /api/v1/namespaces/kube-system/configmaps watch decode ended with: EOF
I0823 07:05:13.186914       1 cache_manager.go:386] yurthub watch configmaps: /api/v1/namespaces/kube-system/configmaps get 0 objects(add:0/update:0/del:0)
E0823 07:05:13.186949       1 loadbalancer.go:391] lb failed to cache req yurthub watch configmaps: https://10.10.14.168:6443/api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=829117&timeout=8m42s&timeoutSeconds=522&watch=true in local cache, EOF
I0823 07:05:13.187773       1 util.go:289] start proxying: get /api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=831847&timeout=8m52s&timeoutSeconds=532&watch=true, in flight requests: 3
I0823 07:05:56.163939       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:05:58.375797       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:13.648220       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:06:34.652392       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:36.290116       1 util.go:248] yurthub watch services: /api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true with status code 200, spent 8m39.003719799s
E0823 07:06:36.290460       1 cache_manager.go:392] yurthub /api/v1/services watch decode ended with: EOF
I0823 07:06:36.290495       1 cache_manager.go:386] yurthub watch services: /api/v1/services get 0 objects(add:0/update:0/del:0)
E0823 07:06:36.290532       1 loadbalancer.go:391] lb failed to cache req yurthub watch services: https://10.10.14.168:6443/api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true in local cache, EOF
I0823 07:06:36.291251       1 util.go:289] start proxying: get /api/v1/services?allowWatchBookmarks=true&resourceVersion=831663&timeout=8m36s&timeoutSeconds=516&watch=true, in flight requests: 3
I0823 07:06:55.681823       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:07:13.812921       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:07:55.672536       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:13.835990       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:08:19.501952       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:21.289704       1 util.go:248] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true with status code 200, spent 5m26.002895926s
E0823 07:08:21.289858       1 cache_manager.go:392] yurthub /apis/apps.openyurt.io/v1beta1/nodepools watch decode ended with: EOF
I0823 07:08:21.289910       1 cache_manager.go:386] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools get 0 objects(add:0/update:0/del:0)
E0823 07:08:21.289955       1 loadbalancer.go:391] lb failed to cache req yurthub watch nodepools: https://10.10.14.168:6443/apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true in local cache, EOF
I0823 07:08:21.290698       1 util.go:289] start proxying: get /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=831941&timeoutSeconds=367&watch=true, in flight requests: 3
I0823 07:09:14.011750       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:10:13.807870       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps

@rambohe-ch
Copy link
Member

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6}).([a-z0-9]{16})\z"

What do I need to do to make the token valid, I followed the v1.3 documentation, Please give a suggestion or provide a link to relevant documents.

@yuluxs007 Because yurtadm integrates with kubeadm, currently supports tokens in the form to match [a-z0-9]{6}.[a-z0-9]{16}, FYI: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/#synopsis

Create bootstrap tokens on the server [root@m kubernetes]# kubeadm token create --print-join-command kubeadm join 127.0.0.1:6443 --token mz5k5t.4hgpzg6weuf07cf0 --discovery-token-ca-cert-hash sha256:113cdaed12905c6152b45030146738aab95b006c019d3f7331a48e49e1c23425 [root@m kubernetes]# kubeadm token list TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS mz5k5t.4hgpzg6weuf07cf0 23h 2023-08-24T06:36:48Z authentication,signing system:bootstrappers:kubeadm:default-node-token

yurthub.yaml information

    command:
    - yurthub
    - --v=2
    - --server-addr=https://10.10.14.168:6443
    - --node-name=$(NODE_NAME)
    - --join-token=mz5k5t.4hgpzg6weuf07cf0

yurt-hub err log

E0823 07:05:13.186877       1 cache_manager.go:392] yurthub /api/v1/namespaces/kube-system/configmaps watch decode ended with: EOF
I0823 07:05:13.186914       1 cache_manager.go:386] yurthub watch configmaps: /api/v1/namespaces/kube-system/configmaps get 0 objects(add:0/update:0/del:0)
E0823 07:05:13.186949       1 loadbalancer.go:391] lb failed to cache req yurthub watch configmaps: https://10.10.14.168:6443/api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=829117&timeout=8m42s&timeoutSeconds=522&watch=true in local cache, EOF
I0823 07:05:13.187773       1 util.go:289] start proxying: get /api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=831847&timeout=8m52s&timeoutSeconds=532&watch=true, in flight requests: 3
I0823 07:05:56.163939       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:05:58.375797       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:13.648220       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:06:34.652392       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:36.290116       1 util.go:248] yurthub watch services: /api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true with status code 200, spent 8m39.003719799s
E0823 07:06:36.290460       1 cache_manager.go:392] yurthub /api/v1/services watch decode ended with: EOF
I0823 07:06:36.290495       1 cache_manager.go:386] yurthub watch services: /api/v1/services get 0 objects(add:0/update:0/del:0)
E0823 07:06:36.290532       1 loadbalancer.go:391] lb failed to cache req yurthub watch services: https://10.10.14.168:6443/api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true in local cache, EOF
I0823 07:06:36.291251       1 util.go:289] start proxying: get /api/v1/services?allowWatchBookmarks=true&resourceVersion=831663&timeout=8m36s&timeoutSeconds=516&watch=true, in flight requests: 3
I0823 07:06:55.681823       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:07:13.812921       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:07:55.672536       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:13.835990       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:08:19.501952       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:21.289704       1 util.go:248] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true with status code 200, spent 5m26.002895926s
E0823 07:08:21.289858       1 cache_manager.go:392] yurthub /apis/apps.openyurt.io/v1beta1/nodepools watch decode ended with: EOF
I0823 07:08:21.289910       1 cache_manager.go:386] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools get 0 objects(add:0/update:0/del:0)
E0823 07:08:21.289955       1 loadbalancer.go:391] lb failed to cache req yurthub watch nodepools: https://10.10.14.168:6443/apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true in local cache, EOF
I0823 07:08:21.290698       1 util.go:289] start proxying: get /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=831941&timeoutSeconds=367&watch=true, in flight requests: 3
I0823 07:09:14.011750       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:10:13.807870       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps

@yuluxs007 It seems that yurthub bootstrap is completed successfully. what's the problem for the above log? and how about upload the entire detail logs of yurthub?

@YTGhost YTGhost mentioned this issue Aug 23, 2023
Closed
@yuluxs007
Copy link
Author

@yuluxs007 Thanks for raising the issue. From the following detail logs of yurthub, the reason of yurthub can not startup is that join token is invalid.

E0822 02:15:25.415818 1 token.go:152] failed to prepare config and ca file, failed to retrieve bootstrap config: couldn't retrieve bootstrap config info: the bootstrap token "eyJhbGciOiJSUzI1NiIsImtpZCI6IkF2mVNdx81CEwqCZdATDUOZwg" was not of the form "\A([a-z0-9]{6}).([a-z0-9]{16})\z"

What do I need to do to make the token valid, I followed the v1.3 documentation, Please give a suggestion or provide a link to relevant documents.

@yuluxs007 Because yurtadm integrates with kubeadm, currently supports tokens in the form to match [a-z0-9]{6}.[a-z0-9]{16}, FYI: https://kubernetes.io/docs/reference/setup-tools/kubeadm/kubeadm-token/#synopsis

Create bootstrap tokens on the server [root@m kubernetes]# kubeadm token create --print-join-command kubeadm join 127.0.0.1:6443 --token mz5k5t.4hgpzg6weuf07cf0 --discovery-token-ca-cert-hash sha256:113cdaed12905c6152b45030146738aab95b006c019d3f7331a48e49e1c23425 [root@m kubernetes]# kubeadm token list TOKEN TTL EXPIRES USAGES DESCRIPTION EXTRA GROUPS mz5k5t.4hgpzg6weuf07cf0 23h 2023-08-24T06:36:48Z authentication,signing system:bootstrappers:kubeadm:default-node-token
yurthub.yaml information

    command:
    - yurthub
    - --v=2
    - --server-addr=https://10.10.14.168:6443
    - --node-name=$(NODE_NAME)
    - --join-token=mz5k5t.4hgpzg6weuf07cf0

yurt-hub err log

E0823 07:05:13.186877       1 cache_manager.go:392] yurthub /api/v1/namespaces/kube-system/configmaps watch decode ended with: EOF
I0823 07:05:13.186914       1 cache_manager.go:386] yurthub watch configmaps: /api/v1/namespaces/kube-system/configmaps get 0 objects(add:0/update:0/del:0)
E0823 07:05:13.186949       1 loadbalancer.go:391] lb failed to cache req yurthub watch configmaps: https://10.10.14.168:6443/api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=829117&timeout=8m42s&timeoutSeconds=522&watch=true in local cache, EOF
I0823 07:05:13.187773       1 util.go:289] start proxying: get /api/v1/namespaces/kube-system/configmaps?allowWatchBookmarks=true&fieldSelector=metadata.name%3Dyurt-hub-cfg&resourceVersion=831847&timeout=8m52s&timeoutSeconds=532&watch=true, in flight requests: 3
I0823 07:05:56.163939       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:05:58.375797       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:13.648220       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:06:34.652392       1 cache_manager.go:447] get bookmark with rv 831663 for yurthub watch services
I0823 07:06:36.290116       1 util.go:248] yurthub watch services: /api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true with status code 200, spent 8m39.003719799s
E0823 07:06:36.290460       1 cache_manager.go:392] yurthub /api/v1/services watch decode ended with: EOF
I0823 07:06:36.290495       1 cache_manager.go:386] yurthub watch services: /api/v1/services get 0 objects(add:0/update:0/del:0)
E0823 07:06:36.290532       1 loadbalancer.go:391] lb failed to cache req yurthub watch services: https://10.10.14.168:6443/api/v1/services?allowWatchBookmarks=true&resourceVersion=830181&timeout=8m39s&timeoutSeconds=519&watch=true in local cache, EOF
I0823 07:06:36.291251       1 util.go:289] start proxying: get /api/v1/services?allowWatchBookmarks=true&resourceVersion=831663&timeout=8m36s&timeoutSeconds=516&watch=true, in flight requests: 3
I0823 07:06:55.681823       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:07:13.812921       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:07:55.672536       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:13.835990       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:08:19.501952       1 cache_manager.go:447] get bookmark with rv 831941 for yurthub watch nodepools
I0823 07:08:21.289704       1 util.go:248] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true with status code 200, spent 5m26.002895926s
E0823 07:08:21.289858       1 cache_manager.go:392] yurthub /apis/apps.openyurt.io/v1beta1/nodepools watch decode ended with: EOF
I0823 07:08:21.289910       1 cache_manager.go:386] yurthub watch nodepools: /apis/apps.openyurt.io/v1beta1/nodepools get 0 objects(add:0/update:0/del:0)
E0823 07:08:21.289955       1 loadbalancer.go:391] lb failed to cache req yurthub watch nodepools: https://10.10.14.168:6443/apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=830568&timeoutSeconds=326&watch=true in local cache, EOF
I0823 07:08:21.290698       1 util.go:289] start proxying: get /apis/apps.openyurt.io/v1beta1/nodepools?allowWatchBookmarks=true&resourceVersion=831941&timeoutSeconds=367&watch=true, in flight requests: 3
I0823 07:09:14.011750       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps
I0823 07:10:13.807870       1 cache_manager.go:447] get bookmark with rv 831930 for yurthub watch configmaps

@yuluxs007 It seems that yurthub bootstrap is completed successfully. what's the problem for the above log? and how about upload the entire detail logs of yurthub?

There are no other error messages ahead, now it looks like Yurthub is running,kubelet needs to connect to Yurthub to work.
Basic situation of my cluster: first install 1 master 2 node K8S cluster, and then deploy openyurt.
Do I need to do the following manually? https://openyurt.io/docs/installation/yurtadm-join/,
2. Install OpenYurt node components ? Configure Kubelet manually ?

@stale Stale
Copy link

stale bot commented Nov 22, 2023

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Nov 22, 2023
@rambohe-ch
Copy link
Member

@yuluxs007 Do you have any progress about this issue?

@stale stale bot removed the wontfix label Nov 23, 2023
@stale Stale
Copy link

stale bot commented Feb 21, 2024

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the wontfix label Feb 21, 2024
@stale stale bot closed this as completed Feb 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
kind/bug kind/bug wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@yuluxs007 @YTGhost @rambohe-ch