Skip to content

Releases: volcano-sh/volcano

v1.10.0

19 Sep 06:47
c601aed
Compare
Choose a tag to compare

What's New

Support Queue Priority Scheduling Strategy

In traditional big data processing scenarios, users can directly set queue priorities to control the scheduling order of jobs. To ease the migration from Hadoop/Yarn to cloud-native platforms, Volcano supports setting priorities at the queue level, reducing migration costs for big data users while enhancing user experience and resource utilization efficiency.

Queues are a fundamental resource in Volcano, each with its own priority. By default, a queue's priority is determined by its share value, which is calculated by dividing the resources allocated to the queue by its total capacity. This is done automatically, with no manual configuration needed. The smaller the share value, the fewer resources the queue has, making it less saturated and more likely to receive resources first. Thus, queues with smaller share values have higher priority, ensuring fairness in resource allocation.

In production environmentsβ€”especially in big data scenariosβ€”users often prefer to manually set queue priorities to have a clearer understanding of the order in which queues are scheduled. Since the share value is dynamic and changes in real-time as resources are allocated, Volcano introduces a priority field to allow users to set queue priorities more intuitively. The higher the priority, the higher the queue's standing. High-priority queues receive resources first, while low-priority queues have their jobs reclaimed earlier when resources need to be recycled.

To ensure compatibility with the share mechanism, Volcano also considers the share value when calculating queue priorities. By default, if a user has not set a specific queue priority or if priorities are equal, Volcano will fall back to comparing share values. In this case, the queue with the smaller share has higher priority. Users have the flexibility to choose between different priority strategies based on their specific needsβ€”either by using the priority or the share method.

Queue priority design doc: Queue priority

Related PRs: (#132, #3700, @TaiPark)

Enable Fine-Grained GPU Resource Sharing and Reclaim

Volcano introduced the elastic queue capacity scheduling feature in version v1.9, allowing users to directly set the capacity for each resource dimension within a queue. This feature also supports elastic scheduling based on the deserved value, enabling more fine-grained resource sharing and recycling across queues.

For detailed design information on elastic queue capacity scheduling, refer to the Capacity Scheduling Design Document.

For a step-by-step guide on using the capacity plugin, see the Capacity Plugin User Guide.

In version v1.10, Volcano extends its support to include reporting different types of GPU resources within elastic queue capacities. NVIDIA's default Device Plugin does not distinguish between GPU models, instead reporting all resources uniformly as nvidia.com/gpu. This limits AI training and inference tasks from selecting specific GPU models, such as A100 or T4, based on their particular needs. To address this, Volcano now supports reporting distinct GPU models at the Device Plugin level, working with the capacity plugin to enable more precise GPU resource sharing and recycling.

For instructions on using the Device Plugin to report various GPU models, refer to the GPU Resource Naming Guide.

Note:

In version v1.10.0, the capacity plugin is the default for queue management. Note that the capacity and proportion plugins are incompatible, so after upgrading to v1.10.0, you must set the deserved field for queues to ensure proper functionality. For detailed instructions, refer to the Capacity Plugin User Guide.

The capacity plugin allocates cluster resources based on the deserved value set by the user, while the proportion plugin dynamically allocates resources according to queue weight. Users can select either the capacity or proportion plugin for queue management based on their specific needs. For more details on the proportion plugin, visit: Proportion Plugin.

Related PR: (#68, @MondayCha)

Introduce Pod Scheduling Readiness Support

Once a Pod is created, it is considered ready for scheduling. In Kube-scheduler, it will try its best to find a suitable node to place all pending Pods. However, in reality, some Pods may be in a "lack of necessary resources" state for a long time. These Pods actually interfere with the decision-making and operation of the scheduler (and downstream components such as Cluster AutoScaler) in an unnecessary way, causing problems such as resource waste. Pod Scheduling Readiness is a new feature of Kube-sheduler. In Kubernetes v.1.30 GA, it has become a stable feature. It controls the scheduling timing of Pods by setting the schedulingGates field of the Pod.

Volcano supports unified scheduling of online and offline jobs. In order to better support the scheduling of microservices, we also support Pod Scheduling Readiness scheduling in Volcano v1.10 to meet the scheduling needs of users in multiple scenarios.

For the documentation of Pod Scheduling Readiness features, please refer to: Pod Scheduling Readiness | Kubernetes

Related PR: (#3581, @ykcai-daniel)

Add Sidecar Container Scheduling Capabilities

A Sidecar container is an auxiliary container designed to support the main business container by handling tasks such as logging, monitoring, and network initialization.

Prior to Kubernetes v1.28, the concept of Sidecar containers existed only informally, with no dedicated API to distinguish them from business containers. Both types of containers were treated equally, which meant that Sidecar containers could be started after the business container and might end before it. Ideally, Sidecar containers should start before and finish after the business container to ensure complete collection of logs and monitoring data.

Kubernetes v1.28 introduces formal support for Sidecar containers at the API level, implementing unified lifecycle management for init containers, Sidecar containers, and business containers. This update also adjusts how resource requests and limits are calculated for Pods, and the feature will enter Beta status in v1.29.

The development of this feature involved extensive discussions, mainly focusing on maintaining compatibility with existing APIs and minimizing disruptive changes. Rather than introducing a new container type, Kubernetes reuses the init container type and designates Sidecar containers by setting the init container’s restartPolicy to Always. This approach addresses both API compatibility and lifecycle management issues effectively.

With this update, the scheduling of Pods now considers the Sidecar container’s resource requests as part of the business container’s total requests. Consequently, the Volcano scheduler has been updated to support this new calculation method, allowing users to schedule Sidecar containers with Volcano.

For more information on Sidecar containers, visit Sidecar Containers | Kubernetes.

Related PR: (#3706, @Monokaix, @7h3-3mp7y-m4n)

Enhance Vcctl Command Line Tool

vcctl is a command line tool for operating Volcano's built-in CRD resources. It can be conveniently used to view/delete/pause/resume vcjob resources, and supports viewing/deleting/opening/closing/updating queue resources. Volcano has enhanced vcctl in the new version, adding the following features:

  • Support creating/deleting/viewing/describing jobflow and jobtemplate resources

  • Support querying vcjob in a specified queue

  • Support querying Pods by queue and vcjob filtering

For detailed guidance documents on vcctl, please refer to: vcctl Command Line Enhancement.

Relared PRs: (#3584, #3543, #3530, #3524, #3508, @googs1025)

Ensure Compatibility with Kubernetes v1.30

Volcano closely follows the pace of Kubernetes community versions and supports every major version of Kubernetes. The latest supported version is v1.30, and runs complete UT and E2E use cases to ensure functionality and reliability.

If you want to participate in the development of Volcano adapting to the new version of Kubernetes, please refer to: adapt-k8s-todo for community contributions.

Related PR: (#3556, @guoqinwill, @wangyysde)

Strengthen Volcano Security Measures

Volcano has always attached great importance to the security of the open source software supply chain. It follows the specifications defined by OpenSSF in terms of license compliance, security vulnerability disclosure and repair, warehouse branch protection, CI inspection, etc. Volcano recently added a new workflow to Github Action, which will run OpenSSF security checks when the code is merged, and update the software security score in real time to continuously improve software security.

At the same time, Volcano has reduced the RBAC permissions of each component, retaining only the neces...

Read more

v1.9.0

21 May 01:52
7f73c6e
Compare
Choose a tag to compare

What's New

Support elastic queue capacity scheduling

Volcano now uses the proportion plugin for queue management. Users can set the guarantee, capacity and other fields of the queue to set the reserved resources and capacity limit of the queue. And by setting the weight value of the queue to realize the resource sharing within the cluster, the queue is proportionally divided into cluster resources according to the weight value, but this queue management method has the following problems:

  • The capacity of the resources divided by the queue is reflected by the weight, which is not intuitive enough.
  • All resources in the queue are divided using the same ratio, and the capacity cannot be set separately for each dimension of the queue.

Based on the above considerations, Volcano implements a new queue elasticity capacity management capability, it supports:

  • Allows users to directly set the capacity of each dimension of resources for the queue instead of setting a weight value.
  • Elastic capacity scheduling based deserved resources, and queue's resources can be shared and reclaimed back.

For example, in AI large model training scenario, setting different resource capacities for different GPU models in the queue, such as A100 and V100, respectively. At the same time, when the cluster resources are idle, the queue can reuse the resources of other idle queues, and when needed, reclaim the resources set by the user for the queue, that is, the amount of resources deserved, so as to realize the elastic capacity scheduling.

To use this feature, you need to set the deserved field of the queue and set the amount of resources to be deserved for each dimension. At the same time, you need to turn on the capacity plugin and turn off the proportion plugin in the scheduling configuration.

Please refer to Capacity Scheduling Design for more detail.

Capacity scheduling example: How to use capacity plugin.

Related PR: (#3277, #121, #3283, @Monokaix)

Support affinity scheduling between queues and nodes

Queues are usually associated with departments within the company, and different departments usually need to use different heterogeneous resource types. For example, the large model training team needs to use NIVDIA’s Tesla GPU, and the recommendation team needs to use AMD’s GPU. When users submit jobs to the queue , the job needs to be automatically scheduled to the node of the corresponding resource type according to the attributes of the queue.

Volcano has implemented affinity scheduling capabilities for queues and nodes. Users only need to set the node label that require affinity in the affinity field of the queue. Volcano will automatically schedule jobs submitted to the current queue to the nodes associated with the queue. Users do not need to Set the affinity of the job separately, and only need to set the affinity of the queue uniformly. Jobs submitted to the queue will be scheduled to the corresponding node based on the affinity of the queue and the node.

This feature supports hard affinity, soft affinity, and anti-affinity scheduling at the same time. When using it, you need to set a label with the key volcano.sh/nodegroup-name for the node, and then set the affinity field of the queue to specify hard affinity, soft affinity label values.

The scheduling plugin for this feature is called nodegroup, for a complete example of its use see: How to use nodegroup plugin.

For detailed design documentation, see The nodegroup design.

Related PR: (#3132, @qiankunli, @wuyueandrew)

GPU sharing feature supports node scoring scheduling

GPU Sharing is a GPU sharing and isolation solution introduced in Volcano v1.8, which provides GPU sharing and device memory control capabilities to enhance the GPU resource utilization in AI training and inference scenarios. v1.9 adds a new scoring strategy for GPU nodes on top of this feature, so that the optimal node can be selected during job assignment to further enhance resource utilization. Users can set different scoring strategies. Currently, the following two strategies are supported:

  • Binpack: Provides a binpack algorithm for GPU card granularity, prioritizing to fill up a node with GPU cards that have already been allocated resources to avoid resource fragmentation and waste.

  • Spread: Prioritizes the use of idle GPU cards over shared cards that have already been allocated resources.

For detailed usage documentation, please refer to: How to use gpu sharing.

Related PR: (#3471, @archlitchi)

Volcano support Kubernetes v1.29

Volcano version follows the Kubernetes community version tempo and supports every base version of Kubernetes. The latest supported version is v1.29 and ran full UT, E2E use cases to ensure functionality and reliability. If you would like to participate in the development of Volcano adapting to new versions of Kubernetes, please refer to: #3459 to make community contributions.

Related PR: (#3295, @guoqinwill)

Enhance scheduler metrics

Volcano uses the client-go to talk with Kubernetes. Although the client can set the QPS to avoid requests from being flow-limited, it is difficult to observe how many QPS is actually used by the client, so in order to observe the frequency of requests from the client in real time, Volcano has added a new client-go metrics, which allows users to access the metrics to see the number of GET, POST and other requests per second, so as to get the actual QPS used per second, and thus decide whether or not the client needs to adjust the QPS. The client-go metrics also include client certificate rotation cycle statistics, response size per request statistics, etc.

Users can use curl http://$volcano_scheduler_pod_ip:8080/metrics to get all the detailed metrics of volcano scheduler.

Related PR: (#3274, @Monokaix)

Add license compliance check

In order to enhance the open source license compliance governance standards of the Volcano community, avoid the introduction of infectious open source protocols, and avoid potential risks, the Volcano community has introduced an open source license compliance checking tool. The so-called infectious protocol refers to software that uses this protocol as an open source license. Derivative works generated after modification, use, and copying must also be open sourced under this agreement. If the third-party library introduced by the PR submitted by the developer contains infectious open source protocols such as GPL, LGPL, etc., CI Access Control will intercept it. The developer needs to replace the third-party library with a loose free software license protocol such as MIT, Apache 2.0, BSD, etc. , to pass the open source license compliance check.

Related PR: (#3308, @Monokaix)

Improve scheduling stability

Volcano v1.9.0 has done more optimization in preemption, retry for scheduling failure, avoiding memory leaks, security enhancement, etc. The details include:

  • Fix the problem of pods not being able to be scheduled due to frequent expansion and contraction of deployment in extreme cases, see PR for details: (#3376, @guoqinwill)
  • Fix Pod preemption: see PR for details: (#3458, @LivingCcj)
  • Optimize Pod scheduling failure retry mechanism: see PR for details: (#3435,@bibiboxοΌ‰
  • Metrics metrics optimization: (#3463, @Monokaix)
  • Security enhancements: (#3449, @lekaf974)

Changes

Read more

v1.8.2

10 Jan 03:43
6167390
Compare
Choose a tag to compare

Changes since v1.8.1

  • fix wrong pods field format output of queue status (#3287 @Monokaix)
  • add ignored csi provisioner when compute csi resources (#3286 @Monokaix)
  • fix k8s.io/dynamic-resource-allocation go mod not found err (#3272 @Monokaix)
  • fix: json marsh error for unsupport type: func() (#3282 @lowang-bh)
  • fix job CRD metadata.annotations: Too long error (#3267 @Monokaix)
  • fix queue update validation err when status.allocated empty ( #3266 @Monokaix)
  • fix grafana dashboard format err (#3265 @Monokaix)
  • update parameter BestEffort of taskInfo after changing parameter InitResreq (#3232 @Lily922)
  • fix: allocated field in queue status is calcutated error (#3221 @shusley244)
  • Avoid repeatedly creating links to obtain node metrics (#3229 @wangyang0616)
  • skip 'pods' resource when checking if the Resource is empty (#3224 @Lily922)
  • queue realcapability change to min dimension of queue capability and … (#3219 @Monokaix)
  • support preemption when the number of pods of a node reaches the upper limit (#3202 @Lily922)
  • Delete duplicate logs generated by the predicate_helper method (#3214 @guoqinwill)
  • support preempting task with bound status (#3209 @Lily922)
  • support preemption when the number of attachment volumes of a node reaches the upper limit (#3212 @Lily922)
  • fix: task scheduling latancy metrics is not accurate (#3128 @lowang-bh)
  • backfill add score process (#3164 @lowang-bh)
  • Obtains the actual load data of a node from the custom metrics API (#3181 @wangyang0616)
  • Update the default value of parameter worker-threads-for-podgroup to 5 (#3180 @Lily922)
  • update volcano.sh/apis version (#3166 @Lily922)

v1.8.1

08 Oct 12:38
14570c6
Compare
Choose a tag to compare

Changes since v1.8.0

What's Changed

  • [cherry-pick for release-1.8] msg information optimization; preemption logic optimization by @wangyang0616 in #3082
  • [cherry-pick for release-1.8]Add featuregates for volcano capabilities by @Monokaix in #3093
  • [cherry-pick for release 1.8]volcano adapt k8s v1.27 by @Mufengzhe in #3101
  • [cherry-pick for release-1.8]successfully scheduled events will not be reported repeatedly for podGroup resource by @Lily922 in #3117
  • [cherry-pick for release-1.8]Add reSync task callback by @Monokaix in #3119
  • [cherry-pick for release-1.8]Add podGroup status to session cache, fix the bug of repeatedly sending pordGroup update request when there is no condations field. by @Lily922 in #3125
  • Update image version for release v1.8.1 by @Mufengzhe in #3136
  • [cherry-pick for release-1.8]fix: the pod anti-affinity constraint fails by @wangyang0616 in #3140
  • [cherry-pick for release-1.8]:feat:add printing of MemStats in dumpall by @xiao-jay in #3098

Full Changelog: v1.8.0...v1.8.1

v1.8.0

17 Aug 08:24
aa57168
Compare
Choose a tag to compare

What's New

Add JobFlow to support lightweight workflow orchestration

The workflow orchestration engine is widely used in high-performance computing, AI biomedicine, image processing, beauty, game AGI, scientific computing and other scenarios, helping users simplify the management of multiple parallel tasks and dependencies, and greatly improving the overall computing efficiency.

JobFlow is a lightweight task flow orchestration engine that focuses on Volcano job orchestration. It provides Volcano with job probes, job completion dependencies, job failure rate tolerance, and other diverse job dependency types, and supports complex process control primitives. The specific capabilities are as follows:

  • Support large-scale job management and complex task flow orchestration.
  • Support real-time query of the running status and task progress of all associated jobs.
  • Support automatic operation of jobs and scheduled start to release labor costs.
  • Various action strategies can be set for different tasks, and corresponding actions can be triggered when the task meets certain conditions, such as timeout retry, node failure drift.

Refer to the links for more details. (JobFlow doc, @hwdef, @lowang-bh, @zhoumingcheng)

Support vGPU scheduling and isolation

Since the outbreak of ChatGPT, there have been more and more research and development of AI large models, and different types of AI large models have been launched one after another. In production environment, users have pain points such as low resource utilization and inflexible GPU resource allocation. They have to purchase a large amount of redundant heterogeneous computing power to meet business needs, and heterogeneous computing power itself is expensive. It has brought a great burden to the development of the enterprise.

Starting from version 1.8, Volcano provides an abstract general framework for sharing devices (GPU, NPU, FPGA...), developers can customize multiple types of shared devices based on this framework. Currently Volcano has supported GPU device multiplexing, resource isolation based on this framework, details are as follows:

  • GPU sharing: Each task can apply to use part of the resources of a GPU card, and the GPU card can be shared among multiple tasks.
  • Device memory control: GPU can be allocated according to device memory (for example: 3000M) or allocated in proportion (for example: 50%) to realize GPU virtualization resource isolation capability.

Refer to the links for more details.

Support the preemption capability for GPU and user-defined resources

Currently, Volcano supports CPU, Memory and other basic resource preemption. GPU resources and user self-managed resources such as NPU, network resources have not been supported yet.

In version 1.8, the predication is refactored to provide more detailed response such as Unschedulable and UnschedulableAndUnresolvable for different scenarios.
The GPU preemption function has been released based on the optimized framework, and the user developed scheduling plugins based on Volcano can be adapted and upgraded according to business scenarios.

Refer to the link for more details. (#2916, @wangyang0616)

Support ElasticSearch monitoring systems in node load-aware scheduling and rescheduling

The status of the kubernetes cluster changes in real time with the creation and termination of tasks. In some scenarios such as adding or deleting nodes, changing the affinity of Pods and Nodes, and dynamically changing the lifecycle of jobs, etc. The following problems will occur. Resource utilization is unbalanced, node performance bottlenecks are offline, etc. At this time, load aware scheduling and rescheduling can help user solve the above problems.

Prior to Volcano version 1.8, the load awareness scheduling and rescheduling only supports Prometheus. Starting from version 1.8, Volcano optimizes the monitoring index acquisition framework and adds support for ElasticSearch monitoring system.

Refer to the links for more details.

Optimize Volcano's ability to schedule microservices

Add Kubernetes default scheduler plugin enable and disable switch

Volcano is a unified integrated scheduling system that not only supports computing jobs such as AI and BigData, but also supports microservice workloads. It is compatible with scheduling plugins such as PodTopologySpread, VolumeZone, VolumeLimits, NodeAffinity, and PodAffinity of the Kubernetes default scheduler, and Kubernetes default scheduling plugins capabilities Enabled by default in Volcano.

Since Volcano 1.8, the Kubernetes default scheduling plugins can be freely selected to be turned on and off through the configuration file, and all of them are turned on by default. If you choose to turn off some plugins, such as: turn off the PodTopologySpread and VolumeZone plugins, you can set the corresponding values ​​in the predicate plugin is false.
Refer to the links for more details. (#2748, @jiangkaihua)

Enhance scheduler to keep compatibility with ClusterAutoscaler

In the Kubernetes platform, Volcano is not only used as a scheduler for batch computing services, but also used as a scheduler for general services. Node horizontal scaling is one of the core functions of Kubernetes, which plays an important role in coping with the surge of user traffic and saving operating costs. Volcano optimizes job scheduling and other related logic, and enhances the compatibility and interaction with ClusterAutoscaler, mainly in the following two aspects:

  • The pod that enters the pipeline state in the scheduling phase triggers capacity expansion in time.
  • Candidate nodes are graded in gradients to reduce the impact of cluster terminating pods on scheduling load, and prevent pods from entering invalid pipeline states, resulting in cluster expansion by mistake.

Refer to the links for more details. (#2782, #3000, @wangyang0616)

Provide tolerance for exception of device plugin

When device plugin crashs or fails to report resouces for some reason and the total resource amount of the node is less than the allocated resource amount, Volcano considers that the node data is inconsistent, make the node as OutOfSync and isolates the node, and stops scheduling any new workload to the node. The isolocation machinism brought some impact to the cluster for example device plugin has no chance to be scheduled to the OutOfSync node. In Volcano v1.8, the machinism is enhanced to tolerate the exception of device plugin, the non-GPU workload like device plugin is still allowed to be scheduled to OutOfSync node.

Refer to the link for more details. (#2999, @Monokaix)

Add helm charts for Volcano

As Volcano is used in production environments and cloud environments with more and more users, simple and standard installation actions are crucial. Since version 1.8, Volcano has optimized charts package publishing and archiving actions, standardized the installation and use process, and completed the migration of historical versions v1.6 and v1.7 to the new helm warehouse.
Refer to the link for more details. (Volcano helm-charts, @wangyang0616)

Other Notable Changes

Read more

v1.7.0

08 Jan 16:18
1933d46
Compare
Choose a tag to compare

What's New

Enhanced Plugin for PyTorch Jobs

As one of the most popular AI frameworks, PyTorch has been widely used in deep learning fields such as computer vision and natural language processing. More and more users turn to Kubernetes to run PyTorch in containers for higher resource utilization and parallel processing efficiency.

Volcano 1.7 enhanced the plugin for PyTorch Jobs, freeing you from the manual configuration of container ports, MASTER_ADDR, MASTER_PORT, WORLD_SIZE, and RANK environment variables.

Other enhanced plugins include those for TensorFlow, MPI, and PyTorch Jobs. They are designed to help you run computing jobs on desired training frameworks with ease.

Volcano also provides an extended development framework for you to tailor Job plugins to your needs.

Refer to the links for more details. (#2313, @ccchenjiahuan)

Ray on Volcano

Ray is a unified framework for extending AI and Python applications. It can run on any machine, cluster, cloud, and Kubernetes cluster. Its community and ecosystem are growing steadily.

As machine learning workloads are hosting computing jobs at a density higher than ever before, single-node environments are failing in providing enough resources for training tasks. Here's where Ray comes in, which seamlessly coordinates resources of the entire cluster, instead of a single node, to run the same set of code. Ray is designed for common scenarios and any type of workloads.

For users running multiple types of Jobs, Volcano partners with Ray to provide high-performance batch scheduling. Ray on Volcano has been released in KubeRay 0.4.

Refer to the links for more details. (#2601(#755) @tgaddair)

Enhance Scheduling for Kubernetes long-running services

This enhancement makes Volcano fully compatible with the Kubernetes default scheduler for long-running services. With this enhancement, users can use Volcano to uniformly schedule long-running services and batch workloads in a single cluster.

Refer to the links for more details:

Support Kubernetes v1.25

This feature is designed to make Volcano compatible with Kubernetes 1.25.

Refer to the links for more details. (#2533, @wangyang0616)

Support multi-arch images for Volcano

This feature is designed to cross-compile volcano images of different architectures. For example, compile an image for the ARM64 architecture on an AMD64 machine.

Refer to the links for more details.(#2435, @ccchenjiahuan)

Optimize Queue Status Information

This feature is designed to enrich the information of the queue. Through this function, users can view the resource allocation of queues in real time, which is convenient for administrators to dynamically plan resources.

Refer to the links for more details.(#2592, @jiangkaihua)

Other Notable Changes

Bug Fixes

Read more

v1.6.0

11 Jun 17:20
e843edd
Compare
Choose a tag to compare

What's New

Support Dynamic Scheduling Based on Real Node Load

This feature aims to schedule pods based on real node load instead of request resource, which will optimize the node resource utilization.Currently the pod is scheduled based on the request resources and node allocatable resources other than the node usage. This leads to the unbalanced resource usage of compute nodes. Pod is scheduled to node with higher usage and lower allocation rate. This is not what users expect. Users expect the usage of each node to be balanced. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/usage-based-scheduling.md. (#2023, #2129 @william-wang )

Support Rescheduling Based on Real Node Load

This feature enables users to rebalance the node utilization based on real node resource usage reqularlly, which is quite suitable for long-running workloads such as deployment. All the rescheduling policies and check interval can be configured according to custom scenarios. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/rescheduling.md. (#2174, #2184 @Thor-wl )

Support Elastic Job Scheduling

This feature allows Volcano to schedule volcano job based on the [min,max] configuration in the job, which will improve resource utilization rate and shorten the execution time of training job. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/elastic-scheduler.md. (#2105, @qiankunli )

Add MPI Job Plugin

This feature provides a new volcano job plugin - MPI Plugin. It will be more convient for MPI users to make use of volcano job instead of manually making connections for hosts of different roles, registering required environment variables and so on. More details can be referred to https://github.com/volcano-sh/volcano/blob/master/docs/design/distributed-framework-plugins.md. (#2237, @hwdef )

Other Notable Changes

Bug Fixes

v1.5.1

15 Mar 15:09
183a178
Compare
Choose a tag to compare

Changes since v1.5.0

  • bug fix: fix the driver pod can not be created due to unreasonable admit (#2081 @william-wang )
  • bug fix: fix error message in TestValidateJobCreate ( #2077 @william-wang )
  • bug fix: Open state queue can be deleted ( #2077 @Yikun )
  • bug fix: upgrade webhook from v1beta1 to v1 to make sure volcano webhook work on K8S 1.22+ ( #2077 @william-wang )
  • bug fix: fix the proportion plugin that ignore the inqueue resource in running jobs( #2057 @Thor-wl )
  • bug fix: set the initial phase to be pending for podgroup ( #2057 @Thor-wl )
  • bug fix: regenerate installer/volcano-development-arm64.yaml to fix arm64 deployment ( #2030 @hwdef )
  • bug fix: fix queue allocated exceeds capability ( #2035 @aidaizyy @Thor-wl )

v1.5.0

20 Feb 06:59
Compare
Choose a tag to compare

Changes since v1.5.0-Beta

v1.4.1

20 Feb 05:45
Compare
Choose a tag to compare

Changes since v1.4.0