Merge branch 'master' of github.com:Qihoo360/wayne into hotfix/ingress

CHANGELOG.md

@@ -1,24 +1,62 @@
+# Change Log
 
-1.1.0 / 2018-12-07
-==================
+## [v1.1.0](https://github.com/Qihoo360/wayne/tree/v1.1.0) (2018-12-07)
+[Full Changelog](https://github.com/Qihoo360/wayne/compare/v1.0.1...v1.1.0)
 
-- Feature: Adding internationalization features
-- Feature: Add Ingress support
-- Feature: User can edit description in app page
-- Feature: Add kubernetes deployment migration
-- Feature: OpenAPI add RESTART_DEPLOYMENT
-- Fix: Cancel forced to uppercase parameter cluster #73
-- Fix: Add the missing encoding in Sign-In form  #82
-- Fix: fix sidenav overflow style #85
+**Implemented enhancements:**
 
-1.0.1 / 2018-11-22
-==================
+- Fix go admin description [\#97](https://github.com/Qihoo360/wayne/pull/97) ([chinaboard](https://github.com/chinaboard))
+- Feature/format frontend code [\#90](https://github.com/Qihoo360/wayne/pull/90) ([wilhelmguo](https://github.com/wilhelmguo))
+- Feature/rename oauth2 qihoo to oauth2 [\#67](https://github.com/Qihoo360/wayne/pull/67) ([wilhelmguo](https://github.com/wilhelmguo))
+- Feature/default config optimization [\#59](https://github.com/Qihoo360/wayne/pull/59) ([wilhelmguo](https://github.com/wilhelmguo))
+- Feature/kubernetes example dependency resource adjust [\#53](https://github.com/Qihoo360/wayne/pull/53) ([wilhelmguo](https://github.com/wilhelmguo))
+- Fix notification model orm warning [\#52](https://github.com/Qihoo360/wayne/pull/52) ([chengyumeng](https://github.com/chengyumeng))
 
-- Fix the problem of creating a user verification in the background is not strict.
-- Fix init namespace error
-- Ace editor optimization
+**Fixed bugs:**
 
+- Background does not set up cluster resources online display problem [\#94](https://github.com/Qihoo360/wayne/issues/94)
+- 后台删除的项目，在回收站无法恢复 [\#93](https://github.com/Qihoo360/wayne/issues/93)
+- 集群name不是全部大写，调用开放api报错 [\#66](https://github.com/Qihoo360/wayne/issues/66)
+- PVC详情的扩展看板功能无效，js错误 Uncaught TypeError: Cannot read property 'nativeElement' of undefined [\#43](https://github.com/Qihoo360/wayne/issues/43)
+- 计划任务提交后报错，并无法再点开高级配置 [\#38](https://github.com/Qihoo360/wayne/issues/38)
+- Fix modal size change bug [\#60](https://github.com/Qihoo360/wayne/pull/60) ([BennieMeng](https://github.com/BennieMeng))
+- Fix CronJob bug that opens advanced mode paste yaml error r… [\#58](https://github.com/Qihoo360/wayne/pull/58) ([wilhelmguo](https://github.com/wilhelmguo))
 
+**Merged pull requests:**
 
+- add restart deployment API [\#108](https://github.com/Qihoo360/wayne/pull/108) ([chengyumeng](https://github.com/chengyumeng))
+- Feature/add kubernetes deployment migration [\#102](https://github.com/Qihoo360/wayne/pull/102) ([wilhelmguo](https://github.com/wilhelmguo))
+- Feature/translate [\#72](https://github.com/Qihoo360/wayne/pull/72) ([BennieMeng](https://github.com/BennieMeng))
 
+## [v1.0.1](https://github.com/Qihoo360/wayne/tree/v1.0.1) (2018-11-22)
+[Full Changelog](https://github.com/Qihoo360/wayne/compare/v1.0.0...v1.0.1)
 
+**Implemented enhancements:**
+
+- English version of README? [\#18](https://github.com/Qihoo360/wayne/issues/18)
+- 未校验用户邮箱格式 [\#11](https://github.com/Qihoo360/wayne/issues/11)
+- 资源单位的显示 [\#7](https://github.com/Qihoo360/wayne/issues/7)
+- kubernetes install optimization [\#51](https://github.com/Qihoo360/wayne/pull/51) ([wilhelmguo](https://github.com/wilhelmguo))
+- Feature/update kubernetes install [\#47](https://github.com/Qihoo360/wayne/pull/47) ([wilhelmguo](https://github.com/wilhelmguo))
+-  Fix the problem of creating a user verification in the background is not strict. [\#35](https://github.com/Qihoo360/wayne/pull/35) ([chengyumeng](https://github.com/chengyumeng))
+- Feature/ace switch error notify [\#34](https://github.com/Qihoo360/wayne/pull/34) ([BennieMeng](https://github.com/BennieMeng))
+- 添加创建部署资源单位,样式调整 [\#27](https://github.com/Qihoo360/wayne/pull/27) ([BennieMeng](https://github.com/BennieMeng))
+
+**Fixed bugs:**
+
+- 上线次数统计  日期选择后格式不正确 [\#20](https://github.com/Qihoo360/wayne/issues/20)
+- 在前台界面，选择“部门概览”，再选择“上线记录”会报错 [\#19](https://github.com/Qihoo360/wayne/issues/19)
+- switch start\_time or end\_time to query online statistics, api/v1/publish/statistics报错400 [\#13](https://github.com/Qihoo360/wayne/issues/13)
+- resource api/v1/kubernetes/namespaces 报错500 [\#10](https://github.com/Qihoo360/wayne/issues/10)
+- APIKeys 报错403 [\#9](https://github.com/Qihoo360/wayne/issues/9)
+- 创建namespace，按钮无法点击 [\#4](https://github.com/Qihoo360/wayne/issues/4)
+- Fix the problem that the first startup does not have namespace, causing the front desk to report an error. [\#44](https://github.com/Qihoo360/wayne/pull/44) ([chengyumeng](https://github.com/chengyumeng))
+- Fix load apikeys error when not set APIKey [\#33](https://github.com/Qihoo360/wayne/pull/33) ([wilhelmguo](https://github.com/wilhelmguo))
+- Fix edit namespace error bug [\#28](https://github.com/Qihoo360/wayne/pull/28) ([wilhelmguo](https://github.com/wilhelmguo))
+- Fix admin reportform dateformat error [\#26](https://github.com/Qihoo360/wayne/pull/26) ([wilhelmguo](https://github.com/wilhelmguo))
+- Fix init namespace error [\#6](https://github.com/Qihoo360/wayne/pull/6) ([wilhelmguo](https://github.com/wilhelmguo))
+
+## [v1.0.0](https://github.com/Qihoo360/wayne/tree/v1.0.0) (2018-11-19)
+
+
+\* *This Change Log was automatically generated by [github_changelog_generator](https://github.com/skywinder/Github-Changelog-Generator)*

Makefile

@@ -22,6 +22,9 @@ run-backend:
 run-worker:
 	cd src/backend/ && bee run -main=./main.go -runargs="worker -t AuditWorker -c 2"
 
+run-webhook:
+	cd src/backend/ && bee run -main=./main.go -runargs="worker -t WebhookWorker -c 2"
+
 run-frontend:
 	cd src/frontend/ && npm start
 

docker-compose.yaml

@@ -3,6 +3,8 @@ services:
   rabbitmq:
     environment:
       RABBITMQ_NODENAME: "rabbit"
+      RABBITMQ_DEFAULT_USER: "guest"
+      RABBITMQ_DEFAULT_PASS: "guest"
     image: rabbitmq:3.7.8-management
     networks:
     - default

hack/dev/webhook/echo.py

@@ -0,0 +1,21 @@
+#!/usr/bin/env python
+
+import SimpleHTTPServer
+import SocketServer
+
+PORT = 8000
+
+class GetHandler(SimpleHTTPServer.SimpleHTTPRequestHandler):
+    def do_GET(self):
+        print(self.headers)
+        self.send_response(200, "")
+    def do_POST(self):
+        print(self.headers)
+        content_length = self.headers.getheaders('content-length')
+        length = int(content_length[0]) if content_length else 0
+        print(self.rfile.read(length))
+        self.send_response(200, "")
+
+Handler = GetHandler
+httpd = SocketServer.TCPServer(("", PORT), Handler)
+httpd.serve_forever()

src/backend/controllers/kubernetes/deployment/deployment.go

@@ -206,6 +206,7 @@ func (c *KubeDeploymentController) Deploy() {
 			Cluster:      publishHistory.Cluster,
 			Status:       publishHistory.Status,
 			Message:      publishHistory.Message,
+			Object:       kubeDeployment,
 		})
 		c.Success("ok")
 	} else {

src/backend/controllers/kubernetes/ingress/ingress.go

@@ -6,8 +6,10 @@ import (
 	"github.com/Qihoo360/wayne/src/backend/client"
 	"github.com/Qihoo360/wayne/src/backend/controllers/base"
 	"github.com/Qihoo360/wayne/src/backend/models"
+	"github.com/Qihoo360/wayne/src/backend/models/response"
 	"github.com/Qihoo360/wayne/src/backend/resources/ingress"
 	"github.com/Qihoo360/wayne/src/backend/util/logs"
+	"github.com/Qihoo360/wayne/src/backend/workers/webhook"
 	kapiv1beta1 "k8s.io/api/extensions/v1beta1"
 )
 
@@ -67,7 +69,7 @@ func (c *KubeIngressController) Deploy() {
 			logs.Critical("insert log into database failed: %s", err)
 		}
 	}()
-	_, err = ingress.CreateOrUpdateService(k8sClient, &kubeIngress)
+	_, err = ingress.CreateOrUpdateIngress(k8sClient, &kubeIngress)
 	if err != nil {
 		publishHistory.Status = models.ReleaseFailure
 		publishHistory.Message = err.Error()
@@ -88,6 +90,16 @@ func (c *KubeIngressController) Deploy() {
 		c.HandleError(err)
 		return
 	}
+	webhook.PublishEventIngress(c.NamespaceId, c.AppId, c.User.Name, c.Ctx.Input.IP(), webhook.OnlineIngress, response.Resource{
+		Type:         publishHistory.Type,
+		ResourceId:   publishHistory.ResourceId,
+		ResourceName: publishHistory.ResourceName,
+		TemplateId:   publishHistory.TemplateId,
+		Cluster:      publishHistory.Cluster,
+		Status:       publishHistory.Status,
+		Message:      publishHistory.Message,
+		Object:       kubeIngress,
+	})
 	c.Success("ok")
 }
 
@@ -100,7 +112,7 @@ func (c *KubeIngressController) Get() {
 		c.AbortBadRequestFormat("Cluster")
 		return
 	}
-	res, err := ingress.GetServiceDetail(k8sClinet, name, namespace)
+	res, err := ingress.GetIngressDetail(k8sClinet, name, namespace)
 	if err != nil {
 		logs.Error("get ingress error cluster: %s, namespace: %s", cluster, namespace)
 		c.HandleError(err)
@@ -118,11 +130,15 @@ func (c *KubeIngressController) Offline() {
 		c.AbortBadRequestFormat("Cluster")
 		return
 	}
-	if err = ingress.DeleteService(k8sClient, name, namespace); err != nil {
+	if err = ingress.DeleteIngress(k8sClient, name, namespace); err != nil {
 		logs.Error("delete ingress: %s in namespace: %s, error: %s", name, namespace, err.Error())
 		c.HandleError(err)
 		return
 	}
+	webhook.PublishEventIngress(c.NamespaceId, c.AppId, c.User.Name, c.Ctx.Input.IP(), webhook.OfflineIngress, response.Resource{
+		Type:         models.PublishTypeIngress,
+		ResourceName: name,
+		Cluster:      cluster,
+	})
 	c.Success("OK")
-	return
 }

src/backend/controllers/kubernetes/service/service.go

@@ -102,6 +102,7 @@ func (c *KubeServiceController) Deploy() {
 			Cluster:      publishHistory.Cluster,
 			Status:       publishHistory.Status,
 			Message:      publishHistory.Message,
+			Object:       kubeService,
 		})
 
 		c.Success("ok")

src/backend/controllers/openapi/openapi.go

@@ -45,6 +45,7 @@ import (
 
 const (
 	GetPodInfoAction          = "GET_POD_INFO"
+	GetPodInfoFromIPAction    = "GET_POD_INFO_FROM_IP"
 	GetResourceInfoAction     = "GET_RESOURCE_INFO"
 	GetDeploymentStatusAction = "GET_DEPLOYMENT_STATUS"
 	UpgradeDeploymentAction   = "UPGRADE_DEPLOYMENT"

src/backend/controllers/openapi/pod.go

@@ -3,6 +3,7 @@ package openapi
 import (
 	"fmt"
 	"net/http"
+	"strings"
 
 	"github.com/Qihoo360/wayne/src/backend/client"
 	"github.com/Qihoo360/wayne/src/backend/models"
@@ -32,6 +33,17 @@ type PodInfoParam struct {
 	Cluster string `json:"cluster"`
 }
 
+// swagger:parameters PodInfoFromIPParam
+type PodInfoFromIPParam struct {
+	// A list of ip.
+	// in: query
+	// Required: true
+	IPS string          `json:"ips"`
+	ips map[string]bool `json:"-"`
+	// Required: true
+	Cluster string `json:"cluster"`
+}
+
 // swagger:route GET /get_pod_info pod PodInfoParam
 //
 // 用于获取线上所有 pod 中包含请求条件中 labelSelector 指定的特定 label 的 pod
@@ -77,3 +89,54 @@ func (c *OpenAPIController) GetPodInfo() {
 	}
 	c.HandleResponse(podList.Body)
 }
+
+// swagger:route GET /get_pod_info_from_ip pod PodInfoFromIPParam
+//
+// 用于通过线上 kubernetes Pod IP 反查对应 Pod 信息的接口
+//
+// 返回 每个 pod 的 pod IP 和 所有 label 列表。
+// 需要绑定全局 apikey 使用。该接口的权限控制为只能使用全局 apikey 的原因是查询条件为 IP ，是对所有 app 的 条件过滤。
+//
+//     Responses:
+//       200: resppodlist
+//       401: responseState
+//       500: responseState
+// @router /get_pod_info_from_ip [get]
+func (c *OpenAPIController) GetPodInfoFromIP() {
+	if !c.CheckoutRoutePermission(GetPodInfoFromIPAction) {
+		return
+	}
+	if c.APIKey.Type != models.GlobalAPIKey {
+		c.AddErrorAndResponse("You can only use global APIKey in this action!", http.StatusUnauthorized)
+		return
+	}
+	params := PodInfoFromIPParam{IPS: c.GetString("ips"), Cluster: c.GetString("cluster")}
+	if params.Cluster == "" {
+		c.AddErrorAndResponse("Invalid cluster parameter:must required!", http.StatusBadRequest)
+		return
+	}
+	params.ips = make(map[string]bool)
+	for _, ip := range strings.Split(params.IPS, ",") {
+		params.ips[ip] = true
+	}
+	manager, err := client.Manager(params.Cluster)
+	if err != nil {
+		c.AddErrorAndResponse("Invalid cluster parameter:not exist!", http.StatusBadRequest)
+		return
+	}
+	pods := pod.GetPodsBySelectorFromCache(manager.Indexer, "", nil)
+	if err != nil {
+		logs.Error(fmt.Sprintf("Failed to parse metadata: %s", err.Error()))
+		c.AddErrorAndResponse(fmt.Sprintf("Maybe a problematic k8s cluster(%s)!", params.Cluster), http.StatusInternalServerError)
+		return
+	}
+	podList := resppodlist{}
+	podList.Body.Code = http.StatusOK
+	for _, p := range pods {
+		if params.ips[p.Status.PodIP] {
+			podList.Body.Pods = append(podList.Body.Pods, response.Pod{Labels: p.Labels, PodIp: p.Status.PodIP})
+		}
+	}
+	c.HandleResponse(podList.Body)
+
+}

src/backend/controllers/permission/namespace_user.go

@@ -6,6 +6,7 @@ import (
 	"github.com/Qihoo360/wayne/src/backend/controllers/base"
 	"github.com/Qihoo360/wayne/src/backend/models"
 	"github.com/Qihoo360/wayne/src/backend/util/logs"
+	"github.com/Qihoo360/wayne/src/backend/workers/webhook"
 	"github.com/mitchellh/mapstructure"
 )
 
@@ -118,6 +119,7 @@ func (c *NamespaceUserController) Create() {
 		c.HandleError(err)
 		return
 	}
+	webhook.PublishEventMember(c.NamespaceId, c.AppId, c.User.Name, c.Ctx.Input.IP(), webhook.AddMember, *namespaceUser.User)
 	c.Success(namespaceUser)
 }
 
@@ -173,6 +175,7 @@ func (c *NamespaceUserController) Update() {
 		c.HandleError(err)
 		return
 	}
+	webhook.PublishEventMember(c.NamespaceId, c.AppId, c.User.Name, c.Ctx.Input.IP(), webhook.UpdateMember, *namespaceUser.User)
 	c.Success(namespaceUser)
 }
 
@@ -190,13 +193,19 @@ func (c *NamespaceUserController) Delete() {
 	if oneGroup != "" {
 		groupsFlag = false
 	}
-
-	err := models.NamespaceUserModel.DeleteById(int64(id), groupsFlag)
+	namespaceUser, err := models.NamespaceUserModel.GetById(int64(id), groupsFlag)
+	if err != nil {
+		logs.Error("get by %d error.%v", id, err)
+		c.HandleError(err)
+		return
+	}
+	err = models.NamespaceUserModel.DeleteById(int64(id), groupsFlag)
 	if err != nil {
 		logs.Error("delete %d error.%v", id, err)
 		c.HandleError(err)
 		return
 	}
+	webhook.PublishEventMember(c.NamespaceId, c.AppId, c.User.Name, c.Ctx.Input.IP(), webhook.DeleteMember, *namespaceUser.User)
 	c.Success(nil)
 }
 

src/backend/models/hookevent/event.go

@@ -11,20 +11,26 @@ var (
 
 	EventDeployment = &HookEvent{
 		Key:         "deployment",
-		Name:        "部署",
-		Description: "部署相关动作行为，如创建、上下线、实例数量调整等",
+		Name:        "Deployment(部署)",
+		Description: "Related actions of \"deployment\", such as creation, online and offline, instance number adjustment, etc.",
 	}
 
 	EventService = &HookEvent{
 		Key:         "service",
-		Name:        "负载均衡",
-		Description: "负载均衡相关动作行为，如创建、上下线等",
+		Name:        "Service(负载均衡)",
+		Description: "Related actions of \"Service\", such as the above online, offline, etc.",
+	}
+
+	EventIngress = &HookEvent{
+		Key:         "ingress",
+		Name:        "Ingress",
+		Description: "Related actions of \"Ingress\", such as the above online, offline, etc.",
 	}
 
 	EventMember = &HookEvent{
 		Key:         "member",
-		Name:        "成员",
-		Description: "项目/部门成员增删或权限变更",
+		Name:        "Member(成员)",
+		Description: "Monitor the additions, deletions, or changes in permissions of members of a application or namespace.",
 	}
 )
 
@@ -36,4 +42,5 @@ func init() {
 	Registry(EventDeployment)
 	Registry(EventService)
 	Registry(EventMember)
+	Registry(EventIngress)
 }

src/backend/models/namespace_user.go

@@ -116,10 +116,9 @@ func (*namespaceUserModel) Add(m *NamespaceUser, allGroupFlag bool) (id int64, e
 
 func (n *namespaceUserModel) GetById(id int64, allGroupFlag bool) (v *NamespaceUser, err error) {
 	v = &NamespaceUser{Id: id}
-	if err = Ormer().Read(v); err != nil {
+	if err = Ormer().QueryTable(TableNameNamespaceUser).Filter("Id", id).RelatedSel("User").One(v); err != nil {
 		return nil, err
 	}
-
 	if allGroupFlag {
 		namespaceUsers := []NamespaceUser{}
 		_, err := Ormer().QueryTable(TableNameNamespaceUser).Filter("Namespace", v.Namespace.Id).Filter("User", v.User.Id).RelatedSel("Group").All(&namespaceUsers)

src/backend/models/response/response.go

@@ -98,6 +98,7 @@ type Resource struct {
 	Cluster      string               `json:"cluster,omitempty"`
 	Status       models.ReleaseStatus `json:"status,omitempty"`
 	Message      string               `json:"message,omitempty"`
+	Object       interface{}          `json:"object,omitempty"` // 用于存储 kubernetes 资源对象的配置细节
 }
 
 // OpenAPI 通用 失败 返回接口