Volcano adapts to the k8s v1.29 #3295

guoqinwill · 2024-01-09T07:02:48Z

1、Volcano adapts to the k8s v1.29
2、In k8s v1.29, some apis have changed. For example, cpuset from "k8s.io/kubernetes/pkg/kubelet/cm/cpuset/cpuset.go" export to "k8s.io/utils/cpuset". Remove 'selectorspread' plugin and use 'podTopologySpread' plugin instead .
3、In k8s v1.29.Migrated the volumebinding scheduler plugins to use contextual logging.Graduated the ReadWriteOncePod and PodSecurity feature gate to GA, and PodSecurity have been removed .Added 'return Skip in PreFilter' when calculates the NodePorts and PodTopologySpread for each existing pod on each node. This update adapts to this change.

Monokaix · 2024-01-17T03:29:10Z

pkg/scheduler/plugins/predicates/predicates.go

+					skipPlugins[taskKey] = plugins
+				}
+				skipPlugins[taskKey].Insert(podTopologySpreadFilter.Name())
+				klog.V(3).Infof("pod(%s/%s) affinity require information is nil, plugin podTopologySpreadFilter is skipped",


Change log level to 5 to avoid rredundant logs, and so does other plugin's skip check.

Monokaix · 2024-01-17T03:32:11Z

pkg/scheduler/plugins/predicates/predicates.go

@@ -401,7 +409,16 @@ func (pp *predicatesPlugin) OnSessionOpen(ssn *framework.Session) {
 		// the processing logic needs to be added to the return value result.
 		if predicate.podTopologySpreadEnable {
 			_, status := podTopologySpreadFilter.PreFilter(context.TODO(), state, task.Pod)
-			if !status.IsSuccess() {
+			if status.IsSkip() {


Abstract this logic to a function because many plugins do the same thing.

Monokaix · 2024-01-17T03:36:10Z

pkg/scheduler/plugins/nodeorder/nodeorder.go

 		for _, node := range nodes {
-			nodeScores[node.Name] = podAffinityScores[node.Name] + nodeTolerationScores[node.Name] + podTopologySpreadScores[node.Name] + selectorSpreadScores[node.Name]


What's the compatibility policy of volcano with kubernets version support? User with lower kubernetes version than 1.29 will miss the plugin.

After review, the plugin of SelectorSpread can be deleted for the following reasons: First, k8s provides the PodTopologySpread plugin to replace the SelectorSpread plugin since version 1.19; secondly, volcano introduced both plugins at the same time, and the selector plugin is not used by default, but the podtopology plugin is used，users can use PodTopologySpread instead of SelectorSpread.

Monokaix · 2024-01-17T03:45:19Z

Please also update Kubernetes compatibility matrix in README.

Monokaix · 2024-01-17T03:46:50Z

The commit "fix some ci errors" should only contains fossa ci fix, helm yaml fix is a seperate part.

wangyang0616 · 2024-01-24T06:42:24Z

docs/ut_coverage/UT_coverage_v1.4.0.html

@@ -8812,7 +8812,7 @@
 import (
        v1 "k8s.io/api/core/v1"
        "k8s.io/kubernetes/pkg/kubelet/cm/cpumanager/topology"
-        "k8s.io/kubernetes/pkg/kubelet/cm/cpuset"
+        "k8s.io/utils/cpuset"


This is the UT coverage report of the previous version.
Why do we need to change it?

wangyang0616 · 2024-01-24T07:08:10Z

Please make CI happy. @guoqinwill

Signed-off-by: guoqin <[email protected]>

…rsions Signed-off-by: guoqin <[email protected]>

…âier versions of Kubernetes Signed-off-by: guoqin <[email protected]>

…apis changed Signed-off-by: guoqin <[email protected]>

…s and PodTopologySpread for each existing pod on each node Signed-off-by: guoqin <[email protected]>

Signed-off-by: guoqin <[email protected]>

wangyang0616 · 2024-01-26T06:39:53Z

/lgtm

Monokaix · 2024-01-26T06:46:44Z

Please modify file installer/dockerfile/webhook-manager/Dockerfile, change KUBE_VERISON to 1.29

…and modify some klogs formats Signed-off-by: guoqin <[email protected]> Signed-off-by: wangyang <[email protected]>

Monokaix · 2024-01-27T07:15:08Z

/lgtm

Thor-wl

/lgtm

william-wang

/lgtm

volcano-sh-bot · 2024-02-01T08:19:54Z

[APPROVALNOTIFIER] This PR is APPROVED

This pull-request has been approved by: william-wang

The full list of commands accepted by this bot can be found here.

The pull request process is described here

Needs approval from an approver in each of these files:

~~OWNERS~~ [william-wang]

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

volcano-sh-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 9, 2024

volcano-sh-bot requested review from hudson741, lowang-bh and shinytang6 January 9, 2024 07:02

volcano-sh-bot added the size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. label Jan 9, 2024

guoqinwill force-pushed the us-adapt-1.29 branch 4 times, most recently from a410b6d to a68909d Compare January 12, 2024 09:50

guoqinwill force-pushed the us-adapt-1.29 branch 3 times, most recently from a4580f9 to e50065b Compare January 16, 2024 12:16

guoqinwill changed the title ~~[WIP]Volcano adapts to the k8s v1.29~~ Volcano adapts to the k8s v1.29 Jan 16, 2024

volcano-sh-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Jan 16, 2024

guoqinwill force-pushed the us-adapt-1.29 branch 3 times, most recently from c616764 to 473a7ef Compare January 16, 2024 14:15

Monokaix reviewed Jan 17, 2024

View reviewed changes

guoqinwill force-pushed the us-adapt-1.29 branch 3 times, most recently from 8fa5c88 to a3492c3 Compare January 17, 2024 11:30

guoqinwill closed this Jan 17, 2024

guoqinwill reopened this Jan 17, 2024

wangyang0616 reviewed Jan 24, 2024

View reviewed changes

guoqinwill force-pushed the us-adapt-1.29 branch from a3492c3 to 7674bdf Compare January 25, 2024 08:15

guoqinwill added 6 commits January 25, 2024 16:19

modify gomod for volcano supports k8s v1.29

a9dff4a

Signed-off-by: guoqin <[email protected]>

The volume binding function is applicable to K8s v1.23 and earlier ve…

5428e8f

…rsions Signed-off-by: guoqin <[email protected]>

csiStorageCapacity is defined in v1beta1 and is compatible with earli…

a664073

…âier versions of Kubernetes Signed-off-by: guoqin <[email protected]>

k8s v1.29 migrated the volume binding to contextual logging and some …

a3becfb

…apis changed Signed-off-by: guoqin <[email protected]>

k8s1.29 added 'return Skip in PreFilter' when calculates the NodePort…

33b920a

…s and PodTopologySpread for each existing pod on each node Signed-off-by: guoqin <[email protected]>

fix ci error

8c07654

Signed-off-by: guoqin <[email protected]>

guoqinwill force-pushed the us-adapt-1.29 branch 2 times, most recently from cd532d0 to 43a9684 Compare January 25, 2024 08:47

volcano-sh-bot assigned wangyang0616 Jan 26, 2024

volcano-sh-bot added the lgtm Indicates that a PR is ready to be merged. label Jan 26, 2024

update development information and Licenses Lint for adapt k8s 1.29 ;…

97f9aa8

…and modify some klogs formats Signed-off-by: guoqin <[email protected]> Signed-off-by: wangyang <[email protected]>

guoqinwill force-pushed the us-adapt-1.29 branch from 43a9684 to 97f9aa8 Compare January 26, 2024 07:07

volcano-sh-bot removed the lgtm Indicates that a PR is ready to be merged. label Jan 26, 2024

volcano-sh-bot assigned Monokaix Jan 27, 2024

volcano-sh-bot added the lgtm Indicates that a PR is ready to be merged. label Jan 27, 2024

Thor-wl reviewed Jan 30, 2024

View reviewed changes

volcano-sh-bot assigned Thor-wl Jan 30, 2024

william-wang approved these changes Feb 1, 2024

View reviewed changes

volcano-sh-bot assigned william-wang Feb 1, 2024

volcano-sh-bot added the approved Indicates a PR has been approved by an approver from all required OWNERS files. label Feb 1, 2024

volcano-sh-bot merged commit a0742c8 into volcano-sh:master Feb 1, 2024
14 checks passed

guoqinwill mentioned this pull request Feb 19, 2024

Volcano adapts to the k8s v1.29 volcano-sh/apis#124

Merged

guoqinwill mentioned this pull request Mar 14, 2024

fix jobflow CRD metadata.annotations: Too long error #3347

Merged

guoqinwill mentioned this pull request Mar 30, 2024

Request to be a member of Volcano community volcano-sh/community#52

Closed

william-wang added this to the v1.9 milestone Jun 12, 2024

wangyysde mentioned this pull request Jun 20, 2024

Volcano adapts to the k8s v1.30 #3534

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Volcano adapts to the k8s v1.29 #3295

Volcano adapts to the k8s v1.29 #3295

guoqinwill commented Jan 9, 2024 •

edited

Loading

Monokaix Jan 17, 2024

guoqinwill Jan 17, 2024

Monokaix Jan 17, 2024

guoqinwill Jan 17, 2024

Monokaix Jan 17, 2024

guoqinwill Jan 25, 2024 •

edited

Loading

Monokaix commented Jan 17, 2024

Monokaix commented Jan 17, 2024

wangyang0616 Jan 24, 2024

guoqinwill Jan 25, 2024

wangyang0616 commented Jan 24, 2024

wangyang0616 commented Jan 26, 2024

Monokaix commented Jan 26, 2024

Monokaix commented Jan 27, 2024

Thor-wl left a comment

william-wang left a comment

volcano-sh-bot commented Feb 1, 2024

		for _, node := range nodes {
		nodeScores[node.Name] = podAffinityScores[node.Name] + nodeTolerationScores[node.Name] + podTopologySpreadScores[node.Name] + selectorSpreadScores[node.Name]

Volcano adapts to the k8s v1.29 #3295

Volcano adapts to the k8s v1.29 #3295

Conversation

guoqinwill commented Jan 9, 2024 • edited Loading

Monokaix Jan 17, 2024

Choose a reason for hiding this comment

guoqinwill Jan 17, 2024

Choose a reason for hiding this comment

Monokaix Jan 17, 2024

Choose a reason for hiding this comment

guoqinwill Jan 17, 2024

Choose a reason for hiding this comment

Monokaix Jan 17, 2024

Choose a reason for hiding this comment

guoqinwill Jan 25, 2024 • edited Loading

Choose a reason for hiding this comment

Monokaix commented Jan 17, 2024

Monokaix commented Jan 17, 2024

wangyang0616 Jan 24, 2024

Choose a reason for hiding this comment

guoqinwill Jan 25, 2024

Choose a reason for hiding this comment

wangyang0616 commented Jan 24, 2024

wangyang0616 commented Jan 26, 2024

Monokaix commented Jan 26, 2024

Monokaix commented Jan 27, 2024

Thor-wl left a comment

Choose a reason for hiding this comment

william-wang left a comment

Choose a reason for hiding this comment

volcano-sh-bot commented Feb 1, 2024

guoqinwill commented Jan 9, 2024 •

edited

Loading

guoqinwill Jan 25, 2024 •

edited

Loading