Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: [dataproc] add optional parameters (tarball_gcs_dir, diagnosis_interval, jobs, yarn_application_ids) in DiagnoseClusterRequest #4661

Merged
merged 2 commits into from
Sep 15, 2023
Merged

feat: [dataproc] add optional parameters (tarball_gcs_dir, diagnosis_interval, jobs, yarn_application_ids) in DiagnoseClusterRequest #4661

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
adb4fab
feat: add optional parameters (tarball_gcs_dir, diagnosis_interval, j…
gcf-owl-bot[bot] Sep 15, 2023
b42c999
🦉 Updates from OwlBot post-processor
gcf-owl-bot[bot] Sep 15, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
55 changes: 37 additions & 18 deletions packages/google-cloud-dataproc/protos/google/cloud/dataproc/v1/clusters.proto
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -25,6 +25,8 @@ import "google/longrunning/operations.proto";
import "google/protobuf/duration.proto";
import "google/protobuf/field_mask.proto";
import "google/protobuf/timestamp.proto";
import "google/protobuf/wrappers.proto";
import "google/type/interval.proto";

option go_package = "cloud.google.com/go/dataproc/v2/apiv1/dataprocpb;dataprocpb";
option java_multiple_files = true;
Expand Down Expand Up @@ -671,22 +673,20 @@ message InstanceGroupConfig {
// Platform](https://cloud.google.com/dataproc/docs/concepts/compute/dataproc-min-cpu).
string min_cpu_platform = 9 [(google.api.field_behavior) = OPTIONAL];

// Optional. The minimum number of instances to create.
// If min_num_instances is set, min_num_instances is used for a criteria to
// decide the cluster. Cluster creation will be failed by being an error state
// if the total number of instances created is less than the
// min_num_instances.
// For example, given that num_instances = 5 and min_num_instances = 3,
// * if 4 instances are created and then registered successfully but one
// instance is failed, the failed VM will be deleted and the cluster will be
// resized to 4 instances in running state.
// * if 2 instances are created successfully and 3 instances are failed,
// the cluster will be in an error state and does not delete failed VMs for
// debugging.
// * if 2 instance are created and then registered successfully but 3
// instances are failed to initialize, the cluster will be in an error state
// and does not delete failed VMs for debugging.
// NB: This can only be set for primary workers now.
// Optional. The minimum number of primary worker instances to create.
// If `min_num_instances` is set, cluster creation will succeed if
// the number of primary workers created is at least equal to the
// `min_num_instances` number.
//
// Example: Cluster creation request with `num_instances` = `5` and
// `min_num_instances` = `3`:
//
// * If 4 VMs are created and 1 instance fails,
// the failed VM is deleted. The cluster is
// resized to 4 instances and placed in a `RUNNING` state.
// * If 2 instances are created and 3 instances fail,
// the cluster in placed in an `ERROR` state. The failed VMs
// are not deleted.
int32 min_num_instances = 12 [(google.api.field_behavior) = OPTIONAL];

// Optional. Instance flexibility Policy allowing a mixture of VM shapes and
Expand Down Expand Up @@ -843,12 +843,12 @@ message NodeGroup {
pattern: "projects/{project}/regions/{region}/clusters/{cluster}/nodeGroups/{node_group}"
};

// Node group roles.
// Node pool roles.
enum Role {
// Required unspecified role.
ROLE_UNSPECIFIED = 0;

// Job drivers run on the node group.
// Job drivers run on the node pool.
DRIVER = 1;
}

Expand Down Expand Up @@ -1531,6 +1531,25 @@ message DiagnoseClusterRequest {

// Required. The cluster name.
string cluster_name = 2 [(google.api.field_behavior) = REQUIRED];

// Optional. The output Cloud Storage directory for the diagnostic
// tarball. If not specified, a task-specific directory in the cluster's
// staging bucket will be used.
string tarball_gcs_dir = 4 [(google.api.field_behavior) = OPTIONAL];

// Optional. Time interval in which diagnosis should be carried out on the
// cluster.
google.type.Interval diagnosis_interval = 6
[(google.api.field_behavior) = OPTIONAL];

// Optional. Specifies a list of jobs on which diagnosis is to be performed.
// Format: projects/{project}/regions/{region}/jobs/{job}
repeated string jobs = 10 [(google.api.field_behavior) = OPTIONAL];

// Optional. Specifies a list of yarn applications on which diagnosis is to be
// performed.
repeated string yarn_application_ids = 11
[(google.api.field_behavior) = OPTIONAL];
}

// The location of diagnostic output.
Expand Down
Loading