Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

incresaed cleanup operator time #184

Merged
merged 18 commits into from
Oct 31, 2021
Merged

incresaed cleanup operator time #184

Show file tree
Hide file tree
Changes from 1 commit
Commits
Show all changes
18 commits
Select commit Hold shift + click to select a range
609ea78
incresaed cleanup operator time
kafkasl Oct 29, 2021
c3581c4
exiting 0 when activity error happens
kafkasl Oct 29, 2021
d04b596
removed try catch changes
kafkasl Oct 30, 2021
b7474ea
added taskName to qc message outer structure
kafkasl Oct 30, 2021
c3c04c5
added event exporter deployment
kafkasl Oct 30, 2021
61ceec1
added correct endpoint and temporary layout
kafkasl Oct 30, 2021
75da79d
remove layout from kubernetes events
kafkasl Oct 30, 2021
ba7e487
changed var subs in yamls
kafkasl Oct 30, 2021
e1aa2a4
removed namespace creation from kubernetes event exporter
kafkasl Oct 30, 2021
b313927
logLevel info
kafkasl Oct 30, 2021
5a4f234
only chek namespace
kafkasl Oct 30, 2021
2ab6da6
fine-grained matching
kafkasl Oct 30, 2021
f18100e
added more matchers
kafkasl Oct 30, 2021
51b8720
removed cleanup operator
kafkasl Oct 30, 2021
c34eb6a
changed config formatting
kafkasl Oct 30, 2021
4600c57
changed service account
kafkasl Oct 30, 2021
4c34d48
reverted back to event-exporter service account
kafkasl Oct 31, 2021
20d9933
filtering only by pod events
kafkasl Oct 31, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion chart-infra/templates/kube-cleanup-operator-deployment.yaml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -30,7 +30,8 @@ spec:
# delete orphaned pods after 1s, so successfully run
# workers wont linger around
- --legacy-mode=false
- --delete-orphaned-pods-after=1s
# wait up to 6 minutes to avoid removing pods which are in containerCreating but are already assigned
- --delete-orphaned-pods-after=3600s
image: quay.io/lwolf/kube-cleanup-operator
imagePullPolicy: Always
name: cleanup-operator
Expand Down
1 change: 1 addition & 0 deletions pipeline-runner/R/gem2s-7-upload_to_aws.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -29,6 +29,7 @@ upload_to_aws <- function(input, pipeline_config, prev_out) {
fpath <- file.path(tempdir(), 'experiment.rds')
saveRDS(scdata, fpath, compress = FALSE)

# can only upload up to 50Gb because part numbers can be any number from 1 to 10,000, inclusive.
put_object_in_s3_multipart(pipeline_config,
bucket = pipeline_config$source_bucket,
object = fpath,
Expand Down
2 changes: 1 addition & 1 deletion pipeline-runner/init.R
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -346,7 +346,7 @@ init <- function() {
cause = error_txt
)

send_pipeline_fail_update(pipeline_config, input_parse$experimentId, input_parse$processName, "Error message placeholder")
send_pipeline_fail_update(pipeline_config, input_parse$experimentId, input_parse$processName, error_txt)

message("Sent task failure to state machine task: ", taskToken)
message("recovered from error:", e$message)
Expand Down