Merge remote-tracking branch 'Tencent/master' into issue-8672-2

CHANGELOG/CHANGELOG-1.11.md

@@ -1,4 +1,7 @@
 <!-- BEGIN MUNGE: GENERATED_TOC -->
+- [v1.11.0-rc.6](#v1110-rc6)
+   - [Changelog since v1.11.0-rc.2](#changelog-since-v1110-rc2)
+
 - [v1.11.0-rc.1](#v1110-rc1)
    - [Changelog since v1.10.0-rc.17](#changelog-since-v1100-rc17)
 
@@ -7,6 +10,50 @@
 
 
 <!-- NEW RELEASE NOTES ENTRY -->
+# v1.11.0-rc.6
+## Changelog since v1.11.0-rc.2
+#### 新增
+- [新增] 蓝盾对接权限中心RBAC [链接](http://github.com/TencentBlueKing/bk-ci/issues/7794)
+- [新增] 流水线国际化相关优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8683)
+- [新增] 下线二进制的定时任务 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8888)
+- [新增] 增加获取代码库详情openapi接口 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8850)
+- [新增] 关闭jvm进程之前等待30秒 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8875)
+- [新增] [stream]兼容snakeyaml解析出错 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8823)
+- [新增] 网关新增强制容器化header [链接](http://github.com/TencentBlueKing/bk-ci/issues/8863)
+- [新增] worker路径的tag需要去掉容器化标志 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8869)
+- [新增] 流水线构建详情页重构需求 [链接](http://github.com/TencentBlueKing/bk-ci/issues/7983)
+- [新增] 容器化route_tag区分ci和codecc [链接](http://github.com/TencentBlueKing/bk-ci/issues/8848)
+- [新增] 主集群和子集群间构建制品网关与流量网关的配置优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8505)
+- [新增] BkRepoClient请求增加X-BKREPO-PROJECT-ID请求头 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8814)
+- [新增] spring 优雅停机 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8833)
+- [新增] 优化首页文案 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8612)
+- [新增] 新增环境页面优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8622)
+- [新增] 关联代码库页面优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8621)
+- [新增] callback禁用后应该发送通知 Tencent [链接](http://github.com/TencentBlueKing/bk-ci/issues/7939)
+- [新增] issue: 网关支持切换整个route_tag到容器化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8827)
+- [新增] 修复kubernetes构建容器ID格式兼容 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8811)
+- [新增] P4事件触发支持自主选择对大小写是否敏感 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8540)
+- [新增] 无编译构建机调度优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8692)
+- [新增] [bug] 并发组排队异常 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8794)
+- [新增] 构建详情中制品查询提供新接口 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8097)
+- [新增] 修改Git触发的判断条件 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8570)
+- [新增] 记录vo、v3鉴权结果 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8500)
+- [新增] 修复chart的BUG [链接](http://github.com/TencentBlueKing/bk-ci/issues/8662)
+
+#### 优化
+- [优化] 蓝盾国际化方案优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8211)
+- [优化] 日志单行输出超过限制（16KB）自动分行 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8856)
+- [优化] 最新动态的时间展示 xxxx-xx-xx hh:mm:ss [链接](http://github.com/TencentBlueKing/bk-ci/issues/8838)
+
+#### 修复
+- [修复] 上下文的状态字段异常 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8861)
+- [修复] 日志时间戳乱序导致重复查询 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8804)
+- [修复] stream的CodeCC红线指标没有关联忽略问题链接 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8821)
+- [修复] stream 登录调试 vmSeqId 参数 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8818)
+- [修复] 第三方构建机工作空间变量解析缺少上下文 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8803)
+- [修复] 并发组优化 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8809)
+- [修复] Linux开机自动启动的agent读取到HOME 变量为空的问题 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8798)
+- [修复] 运行开机内置插件时，点击取消可能取消不了构建 [链接](http://github.com/TencentBlueKing/bk-ci/issues/8789)
 # v1.11.0-rc.1
 ## Changelog since v1.10.0-rc.17
 #### 新增

docker-images/core/ci/dockerfile/backend.bkci.Dockerfile

@@ -1,4 +1,4 @@
-FROM bkci/jdk:0.0.3
+FROM bkci/jdk:0.0.4
 
 LABEL maintainer="Tencent BlueKing Devops"
 

docker-images/core/ci/dockerfile/jdk.Dockerfile

@@ -28,4 +28,6 @@ RUN mkdir -p /data/tools && \
 # 安装第三方构建机JRE
 RUN wget "https://github.com/bkdevops-projects/devops-jre/raw/main/linux/jre.zip" -P /data/workspace/agent-package/jre/linux/ &&\
     wget "https://github.com/bkdevops-projects/devops-jre/raw/main/windows/jre.zip" -P /data/workspace/agent-package/jre/windows/ &&\
-    wget "https://github.com/bkdevops-projects/devops-jre/raw/main/macos/jre.zip" -P /data/workspace/agent-package/jre/macos/ 
+    wget "https://github.com/bkdevops-projects/devops-jre/raw/main/macos/jre.zip" -P /data/workspace/agent-package/jre/macos/ &&\
+    wget "https://github.com/bkdevops-projects/devops-jre/raw/main/linux_arm64/jre.zip" -P /data/workspace/agent-package/jre/linux_arm64/ &&\
+    wget "https://github.com/bkdevops-projects/devops-jre/raw/main/macos_arm64/jre.zip" -P /data/workspace/agent-package/jre/macos_arm64/ 

helm-charts/core/ci/Chart.lock

@@ -16,7 +16,7 @@ dependencies:
   version: 8.20.1
 - name: influxdb
   repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
-  version: 2.3.7
+  version: 5.3.1
 - name: nginx-ingress-controller
   repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
   version: 7.6.21
@@ -26,5 +26,5 @@ dependencies:
 - name: kubernetes-manager
   repository: file://./local_chart/kubernetes-management
   version: 0.0.36
-digest: sha256:521a93d5e92e46288b16ef26391656c9a7041dd880a01cf5c04389f1bb4f4186
-generated: "2023-03-14T17:35:25.523011238+08:00"
+digest: sha256:9a64627edf1e223f51c5aac9ebef4ea399b13b48d0ac241f3a1fe55c5f974bc7
+generated: "2023-07-03T15:53:26.797752554+08:00"

helm-charts/core/ci/Chart.yaml

@@ -27,7 +27,7 @@ dependencies:
   repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
   condition: rabbitmq.enabled
 - name: influxdb
-  version: 2.3.7
+  version: 5.3.1
   repository: https://raw.githubusercontent.com/bitnami/charts/eb5f9a9513d987b519f0ecd732e7031241c50328/bitnami
   condition: influxdb.enabled
 - name: nginx-ingress-controller

helm-charts/core/ci/base/default_env.yaml

@@ -45,3 +45,4 @@ bkCiRedisSsl: false
 bkCiEsHttps: false
 bkCiDefaultLocale: zh_CN
 bkCiSupportLanguages: zh_CN,en_US
+bkEsbEnabled: false

helm-charts/core/ci/base/values.yaml

@@ -1,3 +1,9 @@
+# 镜像源公共配置
+global:
+  imageRegistry: ""
+  imagePullSecrets: []
+  storageClass: ""
+
 # RBAC配置
 rbac:
   enabled: true
@@ -75,7 +81,7 @@ init:
   bkrepo: true
   defaultImage: true
   plugin:
-    enabled: true
+    enabled: false
     atoms:
     - url: https://github.com/TencentBlueKing/ci-checkout/releases/download/v2.0.13/checkout_en.zip
       code: checkout

helm-charts/core/ci/templates/_helpers.tpl

@@ -316,3 +316,32 @@ Create a default fully qualified app name.
 {{- define "bkci.names.fullname" -}}
 {{- printf "%s-%s" .Release.Name .Chart.Name | trunc 63 | trimSuffix "-" -}}
 {{- end -}}
+
+{{/*
+Return the bkci gateway image name
+*/}}
+{{- define "bkci-gateway.image" -}}
+{{ include "common.images.image" (dict "imageRoot" .Values.gatewayImage "global" .Values.global) }}
+{{- end -}}
+
+{{/*
+Return the bkci frontend image name
+*/}}
+{{- define "bkci-frontend.image" -}}
+{{ include "common.images.image" (dict "imageRoot" .Values.frontendImage "global" .Values.global) }}
+{{- end -}}
+
+{{/*
+Return the bkci backend image name
+*/}}
+{{- define "bkci-backend.image" -}}
+{{ include "common.images.image" (dict "imageRoot" .Values.backendImage "global" .Values.global) }}
+{{- end -}}
+
+
+{{/*
+Return the bkci turbo image name
+*/}}
+{{- define "bkci-turbo.image" -}}
+{{ include "common.images.image" (dict "imageRoot" .Values.turbo.image "global" .Values.global) }}
+{{- end -}}

helm-charts/core/ci/templates/artifactory/statefulset.yaml

@@ -16,6 +16,13 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: artifactory
 spec:
+  {{ if ne .Values.config.bkCiArtifactoryRealm "local"}}
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
+  {{ end }}
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -61,7 +68,7 @@ spec:
       {{- end }}        
       containers:
         - name: artifactory
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.artifactory.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.artifactory.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/auth/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: auth
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: auth
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.auth.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.auth.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/dispatch/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: dispatch
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: dispatch
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.dispatch.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.dispatch.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/dispatchDocker/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: dispatch-docker
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: dispatch-docker
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.dispatchDocker.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.dispatchDocker.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/dispatchKubernetes/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: dispatch-kubernetes
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: dispatch-kubernetes
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.dispatchKubernetes.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.dispatchKubernetes.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/environment/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: environment
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: environment
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.environment.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.environment.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/gateway/deployment.yaml

@@ -52,7 +52,7 @@ spec:
       {{- end }}
       initContainers:
         - name: frontend
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.frontendImage "global" $) }}
+          image: {{ include "bkci-frontend.image" . }}
           volumeMounts:
           - mountPath: /tmp/frontend
             name: frontend
@@ -63,7 +63,7 @@ spec:
               cp -r /data/workspace/frontend/* /tmp/frontend/
       containers:
         - name: gateway
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.gatewayImage "global" $) }}
+          image: {{ include "bkci-gateway.image" . }}
           imagePullPolicy: {{ .Values.gatewayImage.pullPolicy }}
           {{- if .Values.gateway.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.gateway.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/image/deployment.yaml

@@ -11,6 +11,11 @@ metadata:
     app.kubernetes.io/component: microservice
     app.kubernetes.io/name: image
 spec:
+  strategy:
+    rollingUpdate:
+      maxSurge: 1
+      maxUnavailable: 1
+    type: RollingUpdate
   selector:
     matchLabels: {{- include "bkci.labels.matchLabels" . | nindent 6 }}
       app.kubernetes.io/component: microservice
@@ -53,7 +58,7 @@ spec:
       {{- end }}        
       containers:
         - name: image
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           {{- if .Values.image.containerSecurityContext.enabled }}
           securityContext: {{- omit .Values.image.containerSecurityContext "enabled" | toYaml | nindent 12 }}

helm-charts/core/ci/templates/init/init.bkrepo.yaml

@@ -19,7 +19,7 @@ spec:
     spec:
       containers:
         - name: init-bkrepo
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/support-files/file/
           command:

helm-charts/core/ci/templates/init/init.defaultImages.yaml

@@ -18,7 +18,7 @@ spec:
     spec:
       containers:
         - name: init-default-image
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/
           command:

helm-charts/core/ci/templates/init/init.iam-rbac.yaml

@@ -27,15 +27,15 @@ spec:
             - "-c"
             - |
               until getent hosts {{ include "bkci.mysql.fullname" . }}
-              do 
+              do
                 echo waiting for {{ include "bkci.mysql.fullname" . }};
                 sleep 2;
               done
               echo "MySQL is available";
       {{- end }}
       containers:
         - name: init-iam
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/support-files/bkiam-rbac
           {{ $mysqlData := split ":" (include "bkci.mysqlAddr" .) }}
@@ -62,6 +62,9 @@ spec:
               INSERT IGNORE INTO T_AUTH_IAM_CALLBACK (ID,GATEWAY,PATH, RESOURCE,SYSTEM) VALUES (8,"http://{{ include "bkci.names.fullname" . }}-gateway.{{ .Release.Namespace }}", "/ms/quality/api/open/quality/callback/group", "quality_group", "ci");
               INSERT IGNORE INTO T_AUTH_IAM_CALLBACK (ID,GATEWAY,PATH, RESOURCE,SYSTEM) VALUES (11,"http://{{ include "bkci.names.fullname" . }}-gateway.{{ .Release.Namespace }}", "/ms/project/api/open/project/callback/", "project", "ci");
               INSERT IGNORE INTO T_AUTH_IAM_CALLBACK (ID,GATEWAY,PATH, RESOURCE,SYSTEM) VALUES (12,"http://{{ include "bkci.names.fullname" . }}-gateway.{{ .Release.Namespace }}", "/ms/process/api/open/pipeline/callback/pipelineGroupInfo", "pipeline_group", "ci");
+              {{- if ne .Values.config.bkCodeccPrivateUrl "" }}
+              INSERT IGNORE INTO T_AUTH_IAM_CALLBACK (ID,GATEWAY,PATH, RESOURCE,SYSTEM) VALUES (13,"{{ .Values.config.bkCodeccPrivateUrl }}", "/ms/openapi/api/open/v2/callback/instances/list", "codecc_task", "codecc");
+              {{- end }}
               EOF
               mysql -u{{- include "bkci.mysqlUsername" . }} -p{{- include "bkci.mysqlPassword" . }} -h{{ $mysqlData._0 }} -P{{ $mysqlData._1 }} devops_ci_auth < iam.sql;
       restartPolicy: OnFailure

helm-charts/core/ci/templates/init/init.iam.yaml

@@ -35,7 +35,7 @@ spec:
       {{- end }}
       containers:
         - name: init-iam
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/support-files/bkiam
           {{ $mysqlData := split ":" (include "bkci.mysqlAddr" .) }}

helm-charts/core/ci/templates/init/init.plugin.yaml

@@ -18,7 +18,7 @@ spec:
     spec:
       containers:
         - name: init-plugin
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/
           {{ $mysqlData := split ":" (include "bkci.mysqlAddr" .) }}

helm-charts/core/ci/templates/init/init.sql.yaml

@@ -34,7 +34,7 @@ spec:
       {{- end }}
       containers:
         - name: init-sql
-          image: {{ include "common.images.image" ( dict "imageRoot" .Values.backendImage "global" $) }}
+          image: {{ include "bkci-backend.image" . }}
           imagePullPolicy: {{ .Values.backendImage.pullPolicy }}
           workingDir: /data/workspace/support-files/sql
           {{ $mysqlData := split ":" (include "bkci.mysqlAddr" .) }}