merged

CONTRIBUTING.md

@@ -3,13 +3,6 @@
 We'd love to accept your patches and contributions to this project. There are
 just a few small guidelines you need to follow.
 
-## Project Structure
-
-Kubeflow Pipelines consists of multiple components. Before you begin, learn how to [build the Kubeflow Pipelines component container images](./developer_guide.md##build-image). To get started, see the development guides:
-
-* [Frontend development guide](./frontend/README.md)
-* [Backend development guide](./backend/README.md)
-
 ## Contributor License Agreement
 
 Contributions to this project must be accompanied by a Contributor License
@@ -22,6 +15,68 @@ You generally only need to submit a CLA once, so if you've already submitted one
 (even if it was for a different project), you probably don't need to do it
 again.
 
+## Contribution Guidelines
+
+To propose a new feature or a change that alters some existing user experience
+or creates a new user experience, follow these steps:
+
+### Step 1: Establish Context
+
+Search on KFP GitHub issues list to see if the same or similar proposal has been
+made in the past.  The historical context can help you draft a better
+proposal. Sometimes you will find a very similar proposal was already presented,
+discussed thoroughly, and that it is either awaiting contribution (in active
+development) or was rejected (often due to timing or conflicting scope with
+other plans). To avoid confusion and conflicts, where possible, please
+contribute to existing issues before creating new ones.
+
+### Step 2: Create Feature Request
+
+Create a new issue using the “Feature Request” template if no existing issue is
+found. Fill in answers to the template questions. To avoid delays, provide as
+much information as needed for initial review. Keep in mind that new features
+should comply with backward-compatibility and platform-portability requirements.
+
+### Step 3: Initial Team Triage
+
+Wait for a member from the Kubeflow Pipelines team (under
+orgs/kubeflow/teams/pipelines/ in
+[org.yaml](https://raw.githubusercontent.com/kubeflow/internal-acls/master/github-orgs/kubeflow/org.yaml))
+to comment on the issue. The team aims for triaging new issues on a weekly
+basis, but cannot at this time provide a guarantee on when your issue will be
+reviewed. The team will work with you to determine if your change is trivial and
+can proceed or whether it is nontrivial and needs a more detailed design
+document and review.
+
+### Step 4: Design Review
+
+If the team agreed with the overall proposal, you would be asked to write a
+design documentation, explaining why you want to make a change, what changes are
+you proposing, and how do you plan to implement it. The design review process
+would be required by default unless the team agreed that the change is too
+trivial. It is recommended that you use this [Google doc template](https://docs.google.com/document/d/1VrfuMo8ZeMmV75a-rUq9SO-E6KotBodAf-P0WZeFDZA/edit?usp=sharing&resourcekey=0-BklOgu8ivhdLCplZuPDZZg)
+for your design, and share it with [email protected] for
+commenting. After sharing the design documentation, you could optionally join a
+session of the bi-weekly Kubeflow Pipelines community meetings
+[[agenda](http://bit.ly/kfp-meeting-notes)] to present or further discuss your
+proposal. A proposal may still get rejected at this stage if it comes with
+unresolved drawbacks or if it does not align with the long term plans for the
+project.
+
+### Step 5: Implementation
+
+After you get formal approval from a Kubeflow Pipelines team member, you can
+implement your design and send a pull request. Make sure existing tests are all
+passing and new tests are added when applicable. Remember to link to the feature
+request issue to help reviewers catch up on the context.
+
+## Project Structure
+
+Kubeflow Pipelines consists of multiple components. Before you begin, learn how to [build the Kubeflow Pipelines component container images](./developer_guide.md##build-image). To get started, see the development guides:
+
+* [Frontend development guide](./frontend/README.md)
+* [Backend development guide](./backend/README.md)
+
 ## Coding style
 
 ### SDK

api/v2alpha1/pipeline_spec.proto

@@ -176,6 +176,9 @@ message ComponentInputsSpec {
   // Definition of an artifact input.
   message ArtifactSpec {
     ArtifactTypeSchema artifact_type = 1;
+
+    // Indicates whether input is a single artifact or list of artifacts
+    bool is_artifact_list = 2;
   }
   // Definition of a parameter input.
   message ParameterSpec {
@@ -218,6 +221,9 @@ message ComponentOutputsSpec {
         [deprecated = true];
     // Properties of the Artifact.
     google.protobuf.Struct metadata = 4;
+
+    // Indicates whether output is a single artifact or list of artifacts
+    bool is_artifact_list = 5;
   }
   // Definition of a parameter output.
   message ParameterSpec {

api/v2alpha1/python/setup.py

@@ -15,7 +15,7 @@
 import setuptools
 
 NAME = "kfp-pipeline-spec"
-VERSION = "0.1.16"
+VERSION = "0.1.17"
 
 setuptools.setup(
     name=NAME,

backend/api/hack/generator.sh

@@ -75,6 +75,7 @@ jq -s 'reduce .[] as $item ({}; . * $item) | .info.title = "Kubeflow Pipelines A
     backend/api/${API_VERSION}/swagger/pipeline.upload.swagger.json \
     backend/api/${API_VERSION}/swagger/healthz.swagger.json \
     > "backend/api/${API_VERSION}/swagger/kfp_api_single_file.swagger.json"
+
 # Generate go_http_client from swagger json.
 swagger generate client \
     -f backend/api/${API_VERSION}/swagger/job.swagger.json \
@@ -125,11 +126,15 @@ swagger generate client \
     -c healthz_client \
     -m healthz_model \
     -t backend/api/${API_VERSION}/go_http_client
-# Hack to fix an issue with go-swagger
-# See https://github.com/go-swagger/go-swagger/issues/1381 for details.
-sed -i -- 's/MaxConcurrency int64 `json:"max_concurrency,omitempty"`/MaxConcurrency int64 `json:"max_concurrency,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_job.go
-sed -i -- 's/IntervalSecond int64 `json:"interval_second,omitempty"`/IntervalSecond int64 `json:"interval_second,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_periodic_schedule.go
-sed -i -- 's/MaxConcurrency string `json:"max_concurrency,omitempty"`/MaxConcurrency int64 `json:"max_concurrency,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_job.go
-sed -i -- 's/IntervalSecond string `json:"interval_second,omitempty"`/IntervalSecond int64 `json:"interval_second,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_periodic_schedule.go
+
+if [ $API_VERSION = "v1beta1" ]
+then 
+    # Hack to fix an issue with go-swagger
+    # See https://github.com/go-swagger/go-swagger/issues/1381 for details.
+    sed -i -- 's/MaxConcurrency int64 `json:"max_concurrency,omitempty"`/MaxConcurrency int64 `json:"max_concurrency,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_job.go
+    sed -i -- 's/IntervalSecond int64 `json:"interval_second,omitempty"`/IntervalSecond int64 `json:"interval_second,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_periodic_schedule.go
+    sed -i -- 's/MaxConcurrency string `json:"max_concurrency,omitempty"`/MaxConcurrency int64 `json:"max_concurrency,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_job.go
+    sed -i -- 's/IntervalSecond string `json:"interval_second,omitempty"`/IntervalSecond int64 `json:"interval_second,omitempty,string"`/g' backend/api/${API_VERSION}/go_http_client/job_model/api_periodic_schedule.go
+fi
 # Execute the //go:generate directives in the generated code.
 cd backend/api && go generate ./...

backend/api/v1beta1/swagger/kfp_api_single_file.swagger.json

@@ -1406,6 +1406,12 @@
             "in": "query",
             "required": false,
             "type": "string"
+          },
+          {
+            "name": "namespace",
+            "in": "query",
+            "required": false,
+            "type": "string"
           }
         ],
         "tags": [
@@ -1461,6 +1467,12 @@
             "in": "query",
             "required": false,
             "type": "string"
+          },
+          {
+            "name": "namespace",
+            "in": "query",
+            "required": false,
+            "type": "string"
           }
         ],
         "tags": [

backend/api/v1beta1/swagger/pipeline.upload.swagger.json

@@ -51,6 +51,12 @@
               "in": "query",
               "required": false,
               "type": "string"
+            },
+            {
+              "name": "namespace",
+              "in": "query",
+              "required": false,
+              "type": "string"
             }
           ],
           "tags": [
@@ -106,6 +112,12 @@
               "in": "query",
               "required": false,
               "type": "string"
+            },
+            {
+              "name": "namespace",
+              "in": "query",
+              "required": false,
+              "type": "string"
             }
           ],
           "tags": [
@@ -298,4 +310,4 @@
         "Bearer": []
       }
     ]
-  }
+  }

backend/api/v2beta1/error.proto

@@ -0,0 +1,31 @@
+// Copyright 2022 The Kubeflow Authors
+//
+// Licensed under the Apache License, Version 2.0 (the "License");
+// you may not use this file except in compliance with the License.
+// You may obtain a copy of the License at
+//
+//      http://www.apache.org/licenses/LICENSE-2.0
+//
+// Unless required by applicable law or agreed to in writing, software
+// distributed under the License is distributed on an "AS IS" BASIS,
+// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+// See the License for the specific language governing permissions and
+// limitations under the License.
+
+syntax = "proto3";
+
+option go_package = "github.com/kubeflow/pipelines/backend/api/v2beta1/go_client";
+package v2beta1;
+
+import "google/protobuf/any.proto";
+
+message Error {
+  string error_message = 1;
+  string error_details = 2;
+}
+
+message Status {
+  string error = 1;
+  int32 code = 2;
+  repeated google.protobuf.Any details = 3;
+}