Prepare monorepo import eapolinario flytepropeller #3
This pull request is big! We’re only showing the most recent 250 commits.
Commits on Jul 2, 2021
-
Do not add downstream links for sub-branch-nodes (flyteorg#285)
Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Jul 7, 2021
-
FlytePropeller should ignore if Admin is already in terminal state #p…
…atch (flyteorg#281) * FlytePropeller should ignore if Admin is already in terminal state Signed-off-by: Ketan Umare <[email protected]> * Lint fixes Signed-off-by: Ketan Umare <[email protected]>
Commits on Jul 9, 2021
Commits on Jul 13, 2021
-
Plugins error document origin (flyteorg#289)
* bump Signed-off-by: wild-endeavor <[email protected]> * tidy Signed-off-by: wild-endeavor <[email protected]>
Commits on Jul 14, 2021
-
Construct subnode DataDir to be under parent's node OutputDir to keep…
… behavior consistent across (flyteorg#292)
Commits on Jul 15, 2021
-
Revert "Construct subnode DataDir to be under parent's node OutputDir…
… to keep behavior consistent across (flyteorg#292)" (flyteorg#293) This reverts commit 9ba5beb. Signed-off-by: Haytham Abuelfutuh <[email protected]>
-
Fix case in which downstream nodes may incorrectly fail because of pa…
…rentNodeID #minor (flyteorg#288) * Branch canexecute fix Signed-off-by: Ketan Umare <[email protected]> * tests Signed-off-by: Ketan Umare <[email protected]> * more unit test fixes Signed-off-by: Ketan Umare <[email protected]> * fixed tests Signed-off-by: Ketan Umare <[email protected]> * more documentation Signed-off-by: Ketan Umare <[email protected]>
Commits on Jul 17, 2021
-
Update Boilerplate (flyteorg#287)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Jul 20, 2021
-
Update code of conduct (flyteorg#296)
* Update code of conduct Signed-off-by: Samhita Alla <[email protected]> * boilerplate Signed-off-by: Samhita Alla <[email protected]>
Commits on Jul 21, 2021
-
Simplify GetNodeExecutionStatus (flyteorg#294)
* Revert "Construct subnode DataDir to be under parent's node OutputDir to keep behavior consistent across (flyteorg#292)" (flyteorg#293) This reverts commit 9ba5beb. Signed-off-by: Haytham Abuelfutuh <[email protected]> * Simplify GetNodeExecutionStatus Signed-off-by: Haytham Abuelfutuh <[email protected]> * SetDataDir in cached DynamicWorkflow case Signed-off-by: Haytham Abuelfutuh <[email protected]> * Fix case in which downstream nodes may incorrectly fail because of parentNodeID #minor (flyteorg#288) * Branch canexecute fix Signed-off-by: Ketan Umare <[email protected]> * tests Signed-off-by: Ketan Umare <[email protected]> * more unit test fixes Signed-off-by: Ketan Umare <[email protected]> * fixed tests Signed-off-by: Ketan Umare <[email protected]> * more documentation Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> * [wip] Working for ev1 to be merged into flyteorg#294 (flyteorg#295) Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> * bump for DCO Signed-off-by: Haytham Abuelfutuh <[email protected]> * Fix DataDir of child nodes Signed-off-by: Haytham Abuelfutuh <[email protected]> * Fix mocks for tests Signed-off-by: Haytham Abuelfutuh <[email protected]> * fixed unit test Signed-off-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
3 people authoredJul 21, 2021
Commits on Aug 9, 2021
Commits on Aug 10, 2021
Commits on Aug 11, 2021
-
Bump flyteplugins to pull in sidecar resources fix (flyteorg#300)
Signed-off-by: Jeev B <[email protected]>
Commits on Aug 12, 2021
-
Update flyteplugins to 0.5.64 (flyteorg#301)
Signed-off-by: Jeev B <[email protected]>
Commits on Aug 13, 2021
-
Update Boilerplate (flyteorg#302)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Aug 16, 2021
Commits on Aug 19, 2021
-
Upgrade flyteplugins to v0.5.67 (flyteorg#307)
* Upgrade flyteplugins to v0.5.67 Signed-off-by: Jeev B <[email protected]> * Tidy up unused versions Signed-off-by: Jeev B <[email protected]>
Commits on Aug 20, 2021
-
Fix pod creation issue when having multiple secret from same secret g…
…roup (flyteorg#306) * Fix pod creation issue when having multiple secret from same secret group Signed-off-by: Pradithya Aria <[email protected]> * Add new line Signed-off-by: Pradithya Aria <[email protected]> * Remove garbage Signed-off-by: Pradithya Aria <[email protected]>
Commits on Aug 21, 2021
-
bump flytestdlib version (flyteorg#308)
* bump flytestdlib version Signed-off-by: Yuvraj <[email protected]>
Commits on Aug 23, 2021
Commits on Aug 24, 2021
-
Update flyteplugins to v0.5.69 (flyteorg#312)
Signed-off-by: Jeev B <[email protected]>
Commits on Aug 26, 2021
Commits on Aug 30, 2021
-
Update flyteplugins to v0.5.72 (flyteorg#316)
Signed-off-by: Jeev B <[email protected]>
Commits on Aug 31, 2021
Commits on Sep 1, 2021
Commits on Sep 2, 2021
Commits on Sep 7, 2021
-
changed node timeout logging to info level instead of error (flyteorg…
…#318) Signed-off-by: Daniel Rammer <[email protected]>
Commits on Sep 9, 2021
-
#minor Revert "Adopt flyteidl's ordered variable map change" (flyteor…
…g#321) * Revert "Adopt flyteidl's ordered variable map change (flyteorg#305)" This reverts commit 23f809c. Signed-off-by: Sean Lin <[email protected]>
Commits on Sep 13, 2021
-
Better resrouce formatting (flyteorg#313)
Signed-off-by: Haytham Abuelfutuh <[email protected]>
-
Order UpstreamNodeIds to ensure consistent compiler output (flyteorg#323
) * Order UpstreamNodeIds to ensure consistent compiler output Signed-off-by: Haytham Abuelfutuh <[email protected]> * Unit test & lint Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Sep 14, 2021
-
Use PluginCleanupPolicy from flyteorg/flyteplugins#203 (flyteorg#311)
* use PluginCleanupPolicy if it exists Signed-off-by: Claire McGinty <[email protected]> * add unit test Signed-off-by: Claire McGinty <[email protected]> * cleanup test code Signed-off-by: Claire McGinty <[email protected]> * assert OnAbort is not attempted by default Signed-off-by: Claire McGinty <[email protected]> * update plugin override interface Signed-off-by: Claire McGinty <[email protected]> * lint Signed-off-by: Claire McGinty <[email protected]> * Apply PR suggestions Signed-off-by: Claire McGinty <[email protected]> * update flyteplugins lib Signed-off-by: Claire McGinty <[email protected]>
Commits on Sep 15, 2021
-
Start-Node Output file copied to output result (flyteorg#317)
* Start-Node outputUri copied in input and output result Signed-off-by: Prafulla Mahindrakar <[email protected]> * Test fix Signed-off-by: Prafulla Mahindrakar <[email protected]> * Linter fix Signed-off-by: Prafulla Mahindrakar <[email protected]> * Incorporated the feedback Signed-off-by: Prafulla Mahindrakar <[email protected]> * Only adding outputUri in the result for start-node Signed-off-by: Prafulla Mahindrakar <[email protected]> * linter issues Signed-off-by: Prafulla Mahindrakar <[email protected]>
Commits on Sep 17, 2021
Commits on Sep 24, 2021
-
Update flyteplugins to 0.6.3 to pickup WebApi fix (flyteorg#327)
Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Oct 6, 2021
-
Added snowflake plugin (flyteorg#319)
Signed-off-by: Kevin Su <[email protected]>
Commits on Oct 7, 2021
-
Fix panic in Pod Webhook (flyteorg#330)
Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Oct 8, 2021
-
Replace v1.ResourceList in backoff controller with a thread-safe impl… (
flyteorg#331) * Replace v1.ResourceList in backoff controller with a thread-safe implementation Signed-off-by: Haytham Abuelfutuh <[email protected]> * Unit tests Signed-off-by: Haytham Abuelfutuh <[email protected]> * Lint Signed-off-by: Haytham Abuelfutuh <[email protected]> * Avoid lock copying Signed-off-by: Haytham Abuelfutuh <[email protected]> * PR Comments and fix benchmark test Signed-off-by: Haytham Abuelfutuh <[email protected]> * Rename github action Signed-off-by: Haytham Abuelfutuh <[email protected]>
-
Switch to using encoder in pluginmachinery (flyteorg#334)
Signed-off-by: Sean Lin <[email protected]>
Commits on Oct 11, 2021
Commits on Oct 12, 2021
Commits on Oct 13, 2021
Commits on Oct 19, 2021
-
Implements native k8s backend plugin for Kubeflow MPI operator. (flyt…
…eorg#314) * Implements native k8s backend plugin for Kubeflow MPI operator. Signed-off-by: Yuvraj <[email protected]>
-
Do not run container as root #patch (flyteorg#336)
* Do not run container as root Signed-off-by: Fredrik Sannholm <[email protected]> * Fix command Signed-off-by: Fredrik Sannholm <[email protected]>
-
Kubeclient has a filter to prevent unnecessary calls to KubeAPI #minor (
flyteorg#339) * [wip] Kubeclient has a filter to prevent unnecessary calls to KubeAPI - Filter is existence check and needs to provide zero false positives, false negatives are ok - Every object that was created or deleted is recorded, so that in subsequent rounds, if needed to be created or deleted can be skipped Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * Added a test Signed-off-by: Ketan Umare <[email protected]> * fixed Signed-off-by: Ketan Umare <[email protected]> * lint fix Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * comment updated Signed-off-by: Ketan Umare <[email protected]> * comment updatd Signed-off-by: Ketan Umare <[email protected]> * updated size Signed-off-by: Ketan Umare <[email protected]> * removed erroneous print Signed-off-by: Ketan Umare <[email protected]> * Improved error message Signed-off-by: Ketan Umare <[email protected]>
Commits on Oct 20, 2021
-
replacing managed fields with a list of one empty instance to clear c…
…rd managed fields (flyteorg#344) Signed-off-by: Daniel Rammer <[email protected]>
Commits on Oct 21, 2021
-
Fix invalid Kubernetes pod names (flyteorg#325)
Signed-off-by: Sean Lin <[email protected]>
-
Add missing omitempty (flyteorg#347)
Signed-off-by: Ketan Umare <[email protected]>
-
Task parallelism updated. (flyteorg#348)
Signed-off-by: Ketan Umare <[email protected]>
-
Status compression: Clear the status for a node when the node was suc…
…cessful (flyteorg#349) * Try clear all status fields on terminal status Signed-off-by: Ketan Umare <[email protected]> * lint fix Signed-off-by: Ketan Umare <[email protected]> * Fix linter issues Signed-off-by: Ketan Umare <[email protected]>
-
Create a flyteadmin event sink filter (flyteorg#341)
* moved events to top-level and merged with flyteidl events Signed-off-by: Daniel Rammer <[email protected]> * fixed events imports on tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * added error message length validation and truncating if necessary. Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * added admin event error message truncating Signed-off-by: Daniel Rammer <[email protected]> * began adding oppo bloom filter to admin event sink Signed-off-by: Daniel Rammer <[email protected]> * removed truncating error message - didn't actually fix anything Signed-off-by: Daniel Rammer <[email protected]> * added oppobloom filter to admin event sink Signed-off-by: Daniel Rammer <[email protected]> * moved EventRecorder and EventSink mocks to the mocks package Signed-off-by: Daniel Rammer <[email protected]> * updated tests to use fastcheck.Filter mock Signed-off-by: Daniel Rammer <[email protected]> * resolved merge conflicts with go.mod and go.sum Signed-off-by: Daniel Rammer <[email protected]> * moved idFromMessage to separate func Signed-off-by: Daniel Rammer <[email protected]> * updated go.mod with newest flytestdlib version to include filter mocks Signed-off-by: Daniel Rammer <[email protected]> * added another level of prometheus scope to better clarify metric Signed-off-by: Daniel Rammer <[email protected]> * exported IDFromMessage and added explicit unit tests for it Signed-off-by: Daniel Rammer <[email protected]> * added comment on IDFromMessage function Signed-off-by: Daniel Rammer <[email protected]>
-
Vault Secret Manager #patch (flyteorg#343)
* Start adding Vaul Secret manager Signed-off-by: Tim Bauer <[email protected]> * Auto-update enumer Signed-off-by: Tim Bauer <[email protected]> * Make verbose Signed-off-by: Tim Bauer <[email protected]> * Revert to print Signed-off-by: Tim Bauer <[email protected]> * Mark debug statements Signed-off-by: Tim Bauer <[email protected]> * Remove prints, simplify vault Signed-off-by: Tim Bauer <[email protected]> * Test format env var, print more Signed-off-by: Tim Bauer <[email protected]> * Check annotations Signed-off-by: Tim Bauer <[email protected]> * Try to retrieve annotations Signed-off-by: Tim Bauer <[email protected]> * Attempt append annotation Signed-off-by: Tim Bauer <[email protected]> * Test annotation injection Signed-off-by: Tim Bauer <[email protected]> * Pre-populate only Signed-off-by: Tim Bauer <[email protected]> * Utils func for vault secret annotation Signed-off-by: Tim Bauer <[email protected]> * Add shorter id to avoid 63 char limit Signed-off-by: Tim Bauer <[email protected]> * Rm print Signed-off-by: Tim Bauer <[email protected]> * Set vault role from config Signed-off-by: Tim Bauer <[email protected]> * Add tests Signed-off-by: Tim Bauer <[email protected]> * Name coreIdl import Signed-off-by: Tim Bauer <[email protected]> * Rm duplicate import Signed-off-by: Tim Bauer <[email protected]> * Update documentation and naming Signed-off-by: Tim Bauer <[email protected]> * Update pkg/webhook/utils.go Co-authored-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Tim Bauer <[email protected]> * Update pkg/webhook/vault_secret_manager.go There is a [workaround](https://www.vaultproject.io/docs/platform/k8s/injector/examples#environment-variable-example) which involves mounting a template formatted file that contains `export API_KEY="{{ .Data.data.api_key }}"` and then sourcing this file as an extra step. But unless the user takes this extra sourcing step, this is still file mounting. So I would go with this Error message since the user should be warned that the expected result from requesting Env var will not be achieved with this. Co-authored-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Tim Bauer <[email protected]> * Update pkg/webhook/vault_secret_manager.go Co-authored-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Tim Bauer <[email protected]> * Fix naming and indent Signed-off-by: Tim Bauer <[email protected]> * Add handling of different kv version and test Signed-off-by: Tim Bauer <[email protected]> * Remove print Signed-off-by: Tim Bauer <[email protected]> * Add enumer for KV version Signed-off-by: Tim Bauer <[email protected]> * Correct kvversion type Signed-off-by: Tim Bauer <[email protected]> * Rm newlines from vault secret template Signed-off-by: Tim Bauer <[email protected]> * Apply suggestions from code review Co-authored-by: Ketan Umare <[email protected]> Signed-off-by: Tim Bauer <[email protected]> * Add docstring Signed-off-by: Tim Bauer <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
3 people authoredOct 21, 2021
Commits on Oct 25, 2021
-
Truncate admin event error messages (flyteorg#352)
* truncating admin event error messages to 100KB Signed-off-by: Daniel Rammer <[email protected]> * added event recorder unit tests Signed-off-by: Daniel Rammer <[email protected]> * removed unnecessary message creation loop in event recorder testing Signed-off-by: Daniel Rammer <[email protected]>
Commits on Oct 26, 2021
-
Make default backoff MaxDuration 20sec (flyteorg#354)
Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Oct 28, 2021
-
CRD validation on flytepropeller project structure (flyteorg#328)
* CRD validation Signed-off-by: Kevin Su <[email protected]> * CRD validation Signed-off-by: Kevin Su <[email protected]> * Revert json tag name Signed-off-by: Kevin Su <[email protected]> * Address comment Signed-off-by: Kevin Su <[email protected]> * Address comment Signed-off-by: Kevin Su <[email protected]> * Fixed test Signed-off-by: Kevin Su <[email protected]> * Rebase Signed-off-by: Kevin Su <[email protected]> * Updated CRD Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]>
-
Revert "CRD validation on flytepropeller project structure (flyteorg#328
)" (flyteorg#355) This reverts commit 0cac412. Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Nov 1, 2021
-
Adding shard, project, and domain labels to crd (flyteorg#357)
* added shard, project, and domain labels to crd Signed-off-by: Daniel Rammer <[email protected]> * fixed tests Signed-off-by: Daniel Rammer <[email protected]> * removed test debugging log output Signed-off-by: Daniel Rammer <[email protected]> * setting project / domain labels based on workflow identifer (in case of dynamic workflow) Signed-off-by: Daniel Rammer <[email protected]> * fixed dynamic workflow test composedPBStore write size Signed-off-by: Daniel Rammer <[email protected]> * fixing random pr comments Signed-off-by: Daniel Rammer <[email protected]> * fixed dynamic workflow test composedPBStore write size ... again Signed-off-by: Daniel Rammer <[email protected]> * moved ShardKeyspaceSize definition to v1alpha1 package Signed-off-by: Daniel Rammer <[email protected]>
-
* plugins v0.7.2 Signed-off-by: Miguel <[email protected]> * go mod tidy Signed-off-by: Miguel <[email protected]>
Commits on Nov 2, 2021
-
Update Boilerplate (flyteorg#359)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Nov 10, 2021
-
Bump flytestdlib to v0.4.4 (flyteorg#361)
* Updated stdlib version Signed-off-by: Kevin Su <[email protected]> * Revert Makefile Signed-off-by: Kevin Su <[email protected]>
Commits on Nov 12, 2021
-
Fix violated uniqueness constraint on task events (flyteorg#364)
* bug fix Signed-off-by: Ketan Umare <[email protected]> * node exec event should have retry group Signed-off-by: Ketan Umare <[email protected]>
-
Update Boilerplate (flyteorg#365)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Nov 18, 2021
-
bump: flyteplugins v0.7.5 (flyteorg#366)
Signed-off-by: Yuvraj <[email protected]>
Commits on Nov 23, 2021
-
Run endtoend tests in PRs against sandbox releases (flyteorg#350)
* Added flytectl in sandbox Signed-off-by: Yuvraj <[email protected]> * wip: Added manual change in boilerplate script Signed-off-by: Yuvraj <[email protected]> * flytectl config fix Signed-off-by: Yuvraj <[email protected]> * Revert registry name in ci Signed-off-by: Yuvraj <[email protected]>
Commits on Nov 24, 2021
-
#minor Intratask checkpointing in FlytePropeller (flyteorg#360)
* Intratask checkpointing in FlytePropeller Signed-off-by: Ketan Umare <[email protected]> * fixed tests Signed-off-by: Ketan Umare <[email protected]> * addressed comments Signed-off-by: Ketan Umare <[email protected]> * merged from master Signed-off-by: Ketan Umare <[email protected]> * updated config Signed-off-by: Ketan Umare <[email protected]> * plugins updated Signed-off-by: Ketan Umare <[email protected]>
Commits on Dec 1, 2021
-
Cache Serialize API (flyteorg#326)
* added cache reservation logic to task execution Signed-off-by: Daniel Rammer <[email protected]> * setting reservation heartbeat to workflow reeval duration Signed-off-by: Daniel Rammer <[email protected]> * releasing reservation on completion Signed-off-by: Daniel Rammer <[email protected]> * fixed lint errors Signed-off-by: Daniel Rammer <[email protected]> * updated task metadata field to DiscoverySerializable in accordance with name change Signed-off-by: Daniel Rammer <[email protected]> * not allowing cache serialization without specifying cachable as well Signed-off-by: Daniel Rammer <[email protected]> * added datacatalog unit tests Signed-off-by: Daniel Rammer <[email protected]> * added handler unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins versions - change before merging Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugin version to reflect latest changes Signed-off-by: Daniel Rammer <[email protected]> * remove flyteidl replace in go.mod and updating to latest version Signed-off-by: Daniel Rammer <[email protected]> * moving release reservation to node finalize function Signed-off-by: Daniel Rammer <[email protected]> * updated go.mod and added missing docs on public types Signed-off-by: Daniel Rammer <[email protected]>
Commits on Dec 2, 2021
-
bump flytestdlib to 0.4.7 (flyteorg#371)
Signed-off-by: Babis Kiosidis <[email protected]> Co-authored-by: Babis Kiosidis <[email protected]>
Commits on Dec 3, 2021
-
Updated flyteplugins to support global pod security groups #patch (fl…
…yteorg#369) * Updated flyteplugins to support global pod security groups Signed-off-by: Ketan Umare <[email protected]> * testing Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]>
-
Scale out with propeller manager and workflow sharding (flyteorg#351)
* added 'manager' command Signed-off-by: Daniel Rammer <[email protected]> * using go routine and timer for manager loop Signed-off-by: Daniel Rammer <[email protected]> * moved manager loop out of cmd and into pkg directory Signed-off-by: Daniel Rammer <[email protected]> * detecting missing replicas Signed-off-by: Daniel Rammer <[email protected]> * moved extracting replica from pod name to new function Signed-off-by: Daniel Rammer <[email protected]> * creating managed flytepropeller pods Signed-off-by: Daniel Rammer <[email protected]> * refactored configuration Signed-off-by: Daniel Rammer <[email protected]> * removed regex parsing for replica - checking for existance with fully qualified pod name Signed-off-by: Daniel Rammer <[email protected]> * mocked out shard strategy abstraction Signed-off-by: Daniel Rammer <[email protected]> * adding arguments to podspec for ConsistentHashingShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * updated import naming Signed-off-by: Daniel Rammer <[email protected]> * moved manager to a top-level package Signed-off-by: Daniel Rammer <[email protected]> * added shard strategy to manager configuration Signed-off-by: Daniel Rammer <[email protected]> * setting shard key label selector on managed propeller instances Signed-off-by: Daniel Rammer <[email protected]> * fixed random lint issues Signed-off-by: Daniel Rammer <[email protected]> * split pod name generate to separate function to ease future auto-scaler implementation Signed-off-by: Daniel Rammer <[email protected]> * cleaned up pod label selector Signed-off-by: Daniel Rammer <[email protected]> * delete pods on shutdown Signed-off-by: Daniel Rammer <[email protected]> * added prometheus metric reporting Signed-off-by: Daniel Rammer <[email protected]> * updated manager run loop to use k8s wait.UntilWithContext Signed-off-by: Daniel Rammer <[email protected]> * moved getKubeConfig into a shared package Signed-off-by: Daniel Rammer <[email protected]> * assigning shard and namespace labels on FlyteWorkflow Signed-off-by: Daniel Rammer <[email protected]> * implement NamespaceShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * implemented NamespaceShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * fixed shard label Signed-off-by: Daniel Rammer <[email protected]> * added comments Signed-off-by: Daniel Rammer <[email protected]> * checking for existing pods on startup Signed-off-by: Daniel Rammer <[email protected]> * handling delete of non-existent pod Signed-off-by: Daniel Rammer <[email protected]> * changes ConsistentHashing name to Random - because that's what it really is Signed-off-by: Daniel Rammer <[email protected]> * implemented EnableUncoveredReplica configuration option Signed-off-by: Daniel Rammer <[email protected]> * added leader election to manager using existing propeller config Signed-off-by: Daniel Rammer <[email protected]> * fixed disable leader election in managed propeller pods Signed-off-by: Daniel Rammer <[email protected]> * removed listPods function Signed-off-by: Daniel Rammer <[email protected]> * added leader election to mitigate concurrent modification issues Signed-off-by: Daniel Rammer <[email protected]> * enabled pprof to profile resource metrics Signed-off-by: Daniel Rammer <[email protected]> * added 'manager' target to Makefile to start manager in development mode (similar to existing server) Signed-off-by: Daniel Rammer <[email protected]> * added shard strategy test for computing key ranges Signed-off-by: Daniel Rammer <[email protected]> * fixed key range computation Signed-off-by: Daniel Rammer <[email protected]> * implemented project and domain shard types Signed-off-by: Daniel Rammer <[email protected]> * returning error on out of range podIndex during UpdatePodSpec call on shard strategy Signed-off-by: Daniel Rammer <[email protected]> * fixed random lint issues Signed-off-by: Daniel Rammer <[email protected]> * added manager tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * added doc comments on exported types and functions Signed-off-by: Daniel Rammer <[email protected]> * exporting ComputeKeyRange function and changed adding addLabelSelector function name to addLabelSelectorIfExists to better reflect functionality Signed-off-by: Daniel Rammer <[email protected]> * adding pod template resource version and shard config hash annotations to fuel automatic pod management on updates Signed-off-by: Daniel Rammer <[email protected]> * removed pod deletion on manager shutdown Signed-off-by: Daniel Rammer <[email protected]> * cleaned up unit tests and lint Signed-off-by: Daniel Rammer <[email protected]> * updated getContainer function to retrive flytepropeller container from pod spec using container name instead of command Signed-off-by: Daniel Rammer <[email protected]> * removed addLabelSelectorIfExists function call Signed-off-by: Daniel Rammer <[email protected]> * changed bytes.Buffer from a var to declaring with new Signed-off-by: Daniel Rammer <[email protected]> * created a new shardstrategy package Signed-off-by: Daniel Rammer <[email protected]> * generating mocks for ShardStrategy to decouple manager package tests from shardstrategy package tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * changed shard configuration defintions and added support for wildcard id in EnvironmentShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * updated documentation Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * setting managed pod owner references Signed-off-by: Daniel Rammer <[email protected]> * updated documentation Signed-off-by: Daniel Rammer <[email protected]> * fixed a few nits Signed-off-by: Daniel Rammer <[email protected]> * delete pods with failed state Signed-off-by: Daniel Rammer <[email protected]> * changed ShardType type to int instead of string Signed-off-by: Daniel Rammer <[email protected]> * removed default values in manager config Signed-off-by: Daniel Rammer <[email protected]> * updated config_flags with pflags generation Signed-off-by: Daniel Rammer <[email protected]>
Commits on Dec 9, 2021
-
Update flyteplugins to fix pickup k8s array plugin fix for caching (f…
…lyteorg#375) * Use latest plugins Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * wip Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteplugins dependency Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Dec 16, 2021
-
Binding Panic when SDK produces an empty binding (flyteorg#376)
* Better Error Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix unit test Signed-off-by: Haytham Abuelfutuh <[email protected]> * Add defensive nil checks Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Dec 21, 2021
-
Added script for checking diff of generated code (flyteorg#372)
* Added ci check for go generate Signed-off-by: Yuvraj <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Co-authored-by: Katrina Rogan <[email protected]>
Commits on Dec 31, 2021
-
Gracefully fail when NodeEvent exceeds gRPC size limit (flyteorg#377)
* failing node when grpc message is too large Signed-off-by: Daniel Rammer <[email protected]> * calling IdempotentRecordEvent to update the UI node status and error Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * removed TODO message Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jan 4, 2022
Commits on Jan 7, 2022
-
Create codeql-analysis.yml (flyteorg#374)
* Scale out with propeller manager and workflow sharding (flyteorg#351) * added 'manager' command Signed-off-by: Daniel Rammer <[email protected]> * using go routine and timer for manager loop Signed-off-by: Daniel Rammer <[email protected]> * moved manager loop out of cmd and into pkg directory Signed-off-by: Daniel Rammer <[email protected]> * detecting missing replicas Signed-off-by: Daniel Rammer <[email protected]> * moved extracting replica from pod name to new function Signed-off-by: Daniel Rammer <[email protected]> * creating managed flytepropeller pods Signed-off-by: Daniel Rammer <[email protected]> * refactored configuration Signed-off-by: Daniel Rammer <[email protected]> * removed regex parsing for replica - checking for existance with fully qualified pod name Signed-off-by: Daniel Rammer <[email protected]> * mocked out shard strategy abstraction Signed-off-by: Daniel Rammer <[email protected]> * adding arguments to podspec for ConsistentHashingShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * updated import naming Signed-off-by: Daniel Rammer <[email protected]> * moved manager to a top-level package Signed-off-by: Daniel Rammer <[email protected]> * added shard strategy to manager configuration Signed-off-by: Daniel Rammer <[email protected]> * setting shard key label selector on managed propeller instances Signed-off-by: Daniel Rammer <[email protected]> * fixed random lint issues Signed-off-by: Daniel Rammer <[email protected]> * split pod name generate to separate function to ease future auto-scaler implementation Signed-off-by: Daniel Rammer <[email protected]> * cleaned up pod label selector Signed-off-by: Daniel Rammer <[email protected]> * delete pods on shutdown Signed-off-by: Daniel Rammer <[email protected]> * added prometheus metric reporting Signed-off-by: Daniel Rammer <[email protected]> * updated manager run loop to use k8s wait.UntilWithContext Signed-off-by: Daniel Rammer <[email protected]> * moved getKubeConfig into a shared package Signed-off-by: Daniel Rammer <[email protected]> * assigning shard and namespace labels on FlyteWorkflow Signed-off-by: Daniel Rammer <[email protected]> * implement NamespaceShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * implemented NamespaceShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * fixed shard label Signed-off-by: Daniel Rammer <[email protected]> * added comments Signed-off-by: Daniel Rammer <[email protected]> * checking for existing pods on startup Signed-off-by: Daniel Rammer <[email protected]> * handling delete of non-existent pod Signed-off-by: Daniel Rammer <[email protected]> * changes ConsistentHashing name to Random - because that's what it really is Signed-off-by: Daniel Rammer <[email protected]> * implemented EnableUncoveredReplica configuration option Signed-off-by: Daniel Rammer <[email protected]> * added leader election to manager using existing propeller config Signed-off-by: Daniel Rammer <[email protected]> * fixed disable leader election in managed propeller pods Signed-off-by: Daniel Rammer <[email protected]> * removed listPods function Signed-off-by: Daniel Rammer <[email protected]> * added leader election to mitigate concurrent modification issues Signed-off-by: Daniel Rammer <[email protected]> * enabled pprof to profile resource metrics Signed-off-by: Daniel Rammer <[email protected]> * added 'manager' target to Makefile to start manager in development mode (similar to existing server) Signed-off-by: Daniel Rammer <[email protected]> * added shard strategy test for computing key ranges Signed-off-by: Daniel Rammer <[email protected]> * fixed key range computation Signed-off-by: Daniel Rammer <[email protected]> * implemented project and domain shard types Signed-off-by: Daniel Rammer <[email protected]> * returning error on out of range podIndex during UpdatePodSpec call on shard strategy Signed-off-by: Daniel Rammer <[email protected]> * fixed random lint issues Signed-off-by: Daniel Rammer <[email protected]> * added manager tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * added doc comments on exported types and functions Signed-off-by: Daniel Rammer <[email protected]> * exporting ComputeKeyRange function and changed adding addLabelSelector function name to addLabelSelectorIfExists to better reflect functionality Signed-off-by: Daniel Rammer <[email protected]> * adding pod template resource version and shard config hash annotations to fuel automatic pod management on updates Signed-off-by: Daniel Rammer <[email protected]> * removed pod deletion on manager shutdown Signed-off-by: Daniel Rammer <[email protected]> * cleaned up unit tests and lint Signed-off-by: Daniel Rammer <[email protected]> * updated getContainer function to retrive flytepropeller container from pod spec using container name instead of command Signed-off-by: Daniel Rammer <[email protected]> * removed addLabelSelectorIfExists function call Signed-off-by: Daniel Rammer <[email protected]> * changed bytes.Buffer from a var to declaring with new Signed-off-by: Daniel Rammer <[email protected]> * created a new shardstrategy package Signed-off-by: Daniel Rammer <[email protected]> * generating mocks for ShardStrategy to decouple manager package tests from shardstrategy package tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * changed shard configuration defintions and added support for wildcard id in EnvironmentShardStrategy Signed-off-by: Daniel Rammer <[email protected]> * updated documentation Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * setting managed pod owner references Signed-off-by: Daniel Rammer <[email protected]> * updated documentation Signed-off-by: Daniel Rammer <[email protected]> * fixed a few nits Signed-off-by: Daniel Rammer <[email protected]> * delete pods with failed state Signed-off-by: Daniel Rammer <[email protected]> * changed ShardType type to int instead of string Signed-off-by: Daniel Rammer <[email protected]> * removed default values in manager config Signed-off-by: Daniel Rammer <[email protected]> * updated config_flags with pflags generation Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> * Create codeql-analysis.yml Signed-off-by: Haytham Abuelfutuh <[email protected]> * Handle code quality issue Signed-off-by: Haytham Abuelfutuh <[email protected]> * check boundaries Signed-off-by: Haytham Abuelfutuh <[email protected]> * 0 is ok Signed-off-by: Haytham Abuelfutuh <[email protected]> * Use ParseUint instead Signed-off-by: Haytham Abuelfutuh <[email protected]> * bump for DCO Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
-
Update Boilerplate (flyteorg#368)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
-
Update Flyteidl version (flyteorg#252)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Jan 11, 2022
-
Add structured dataset type checker (flyteorg#384)
Signed-off-by: Kevin Su <[email protected]>
Commits on Jan 23, 2022
-
Added workflow for patch release (flyteorg#387)
Signed-off-by: Yuvraj <[email protected]>
Commits on Jan 25, 2022
-
Fix e2e sandbox for upgrade (flyteorg#388)
Signed-off-by: Yuvraj <[email protected]>
-
Bump flyteplugins for merged pod plugin (flyteorg#386)
* updating flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * changed flyteplugins version to merged release Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jan 26, 2022
-
Add RetryAttempt and Phase to Task Event Metadata (flyteorg#381)
* transforming TaskInfo ExternalResources to event ExternalResourceInfo to correctly map resource phase Signed-off-by: Daniel Rammer <[email protected]> * fixing lint issues Signed-off-by: Daniel Rammer <[email protected]> * populating ExternalResource proto Index field Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins versions Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jan 29, 2022
-
Update Flyteidl version (flyteorg#390)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Feb 1, 2022
-
Enable datacatalog client to use admin's auth option (flyteorg#389)
* Enable datacatalog client to use admin's auth option Signed-off-by: Sean Lin <[email protected]> * goimport Signed-off-by: Sean Lin <[email protected]> * Update flyteidl version Signed-off-by: Sean Lin <[email protected]> * update idl version Signed-off-by: Sean Lin <[email protected]>
Commits on Feb 8, 2022
-
A new metrics scope per plugin is created (flyteorg#393)
* A new metrics scope per plugin is created Signed-off-by: Ketan Umare <[email protected]> * updated setup context Signed-off-by: Ketan Umare <[email protected]> * added unit test Signed-off-by: Ketan Umare <[email protected]>
-
Kenny/type metadata (flyteorg#370)
* feat: strip annotation from node template Signed-off-by: Kenny Workman <[email protected]> * feat: strip annotation tests Signed-off-by: Kenny Workman <[email protected]> * fix: strip outputs + test Signed-off-by: Kenny Workman <[email protected]> * fix: temp flyteidl for tests Signed-off-by: Kenny Workman <[email protected]> * fix: tmp update flyteidl Signed-off-by: Kenny Workman <[email protected]> * fix: range nil ok Signed-off-by: Kenny Workman <[email protected]> * fix: strip annotation in type compare Signed-off-by: Kenny Workman <[email protected]> * fix: rm .swp Signed-off-by: Kenny Workman <[email protected]> * fix: new flyteidl Signed-off-by: Kenny Workman <[email protected]>
Commits on Feb 10, 2022
-
Added reuseable workflow (flyteorg#391)
* Added reuseable workflow Signed-off-by: Yuvraj <[email protected]>
Commits on Feb 15, 2022
-
Refactor for single binary (flyteorg#399)
* default config for events Signed-off-by: Ketan Umare <[email protected]> * Refactor flytepropeller to allow external start Signed-off-by: Ketan Umare <[email protected]> * refactor lint fix Signed-off-by: Ketan Umare <[email protected]> * entrypoint Signed-off-by: Ketan Umare <[email protected]>
-
Implement GetGeneratedNameWith (flyteorg#400)
* Implement GetGeneratedNameWith Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update plugins version Signed-off-by: Haytham Abuelfutuh <[email protected]> * Unit test Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Feb 16, 2022
-
fix missing error handling when creating mutatingwebhookconfiguration (…
…flyteorg#401) Signed-off-by: akumor <[email protected]>
-
Add BigQuery plugin (flyteorg#394)
* Add bigquery plugin Signed-off-by: Kevin Su <[email protected]> * rerun tests Signed-off-by: Kevin Su <[email protected]> * Use go 1.16 Signed-off-by: Kevin Su <[email protected]>
-
StructuredDataset without column should be castable to StructuredData…
…set with column (flyteorg#395) Signed-off-by: Kevin Su <[email protected]>
Commits on Feb 17, 2022
-
Added reuseable functional test (flyteorg#397)
* Added reuseable workflow Signed-off-by: Yuvraj <[email protected]>
Commits on Feb 18, 2022
-
Update boilerplate version (flyteorg#402)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: evalsocket <[email protected]>
Commits on Mar 2, 2022
-
Hotfix docker image cache key in end2end test (flyteorg#408)
* Added hotfix for end2end test Signed-off-by: Yuvraj <[email protected]>
Commits on Mar 3, 2022
-
Take literal hash into account during cache key calculation (flyteorg…
…#406) * Point to flyteidl branch Signed-off-by: Eduardo Apolinario <[email protected]> * Set tag Signed-off-by: Eduardo Apolinario <[email protected]> * Add replaceHashInLiteral function Signed-off-by: Eduardo Apolinario <[email protected]> * Changes to propeller-config.yaml to run locally Signed-off-by: Eduardo Apolinario <[email protected]> * Pick up latest flyteidl@add-hash-to-literal Signed-off-by: Eduardo Apolinario <[email protected]> * Rename variables Signed-off-by: Eduardo Apolinario <[email protected]> * Add unit tests Signed-off-by: Eduardo Apolinario <[email protected]> * s/replaceHashInLiteral/hashify/g Signed-off-by: Eduardo Apolinario <[email protected]> * s/literalMapCopy/hashifiedLiteralMap/g Signed-off-by: Eduardo Apolinario <[email protected]> * Bump flyteidl Signed-off-by: Eduardo Apolinario <[email protected]> * Use flyteidl 0.23.0 Signed-off-by: Eduardo Apolinario <[email protected]> * Revert "Changes to propeller-config.yaml to run locally" This reverts commit 983ee6c580b6b98a2150de8ac1a1f272367659bd. Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
Commits on Mar 4, 2022
-
Update boilerplate version (flyteorg#409)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: evalsocket <[email protected]>
Commits on Mar 8, 2022
-
Use PodTemplate metadata in managed Pods (flyteorg#411)
* overriding base PodTemplate with manager-specific updates Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * using the ObjectMeta from the PodTemplateSpec instead of the PodTemplate resource Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 10, 2022
-
Only require default plugin configuration for task types with multipl…
…e registered plugins (flyteorg#405) * task types with only one registered plugin use it - otherwise declare default plugin in config Signed-off-by: Daniel Rammer <[email protected]> * fixed test and lint issues Signed-off-by: Daniel Rammer <[email protected]> * updated task-config flags Signed-off-by: Daniel Rammer <[email protected]> * converted to named return values Signed-off-by: Daniel Rammer <[email protected]> * failing to initialize plugins on nil TaskConfig Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins to merged version Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 14, 2022
-
Fix map task with empty inputs (flyteorg#403)
* bumping flyteplugins and flytestdlib Signed-off-by: Daniel Rammer <[email protected]> * updated to latest flytestlib and flyteplugins PRs Signed-off-by: Daniel Rammer <[email protected]> * updated flytestdlib and flyteplugins versions Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 16, 2022
-
Support default PodTemplate for k8s plugin (flyteorg#404)
* updated flyteplugins to use local repo Signed-off-by: Daniel Rammer <[email protected]> * initializing default pod template informer Signed-off-by: Daniel Rammer <[email protected]> * moved k8s plugin configuration Signed-off-by: Daniel Rammer <[email protected]> * updated to reflect per-namespace PodTemplate support Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * fixes lint issue Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 18, 2022
-
bump latest flyteplugin (flyteorg#413)
Signed-off-by: Yuvraj <[email protected]>
Commits on Mar 22, 2022
-
Add Union Type validation (flyteorg#373)
* Add Union Type validation * Update to use string tags * Fix go.mod + update typing tests * Restore ambiguity checking, add binding tests * Add union type promise binding tests * Do not use NONE as any type * Ditto * Implement changed design * Bump flyteidl to 0.24.0 Signed-off-by: Eduardo Apolinario <[email protected]> * Fix merge mistake in typing.go Signed-off-by: Eduardo Apolinario <[email protected]> * Add tests to bindings_test.go back Signed-off-by: Eduardo Apolinario <[email protected]> * Set `val` variable Signed-off-by: Eduardo Apolinario <[email protected]> * Lint Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
Commits on Mar 24, 2022
-
Remove extra SetMetricKeys (flyteorg#416)
Signed-off-by: Ketan Umare <[email protected]>
Commits on Mar 31, 2022
-
Create FlyteWorkflow CRD on FlytePropeller startup (flyteorg#414)
* creating FlyteWorkflow CRD on FlytePropeller startup Signed-off-by: Daniel Rammer <[email protected]> * checking for CRD existence before creating Signed-off-by: Daniel Rammer <[email protected]> * added CreateFlyteWorkflowCRD configuration option Signed-off-by: Daniel Rammer <[email protected]> * fixed CreateFlyteWorkflowCRD configuration description Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 1, 2022
Commits on Apr 8, 2022
-
Reporting more complete metadata for map task subtasks (flyteorg#417)
* fixed potential nil pointer Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * added CacheStatus and Logs fields to ExternalResourceInfo proto Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * bumping flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * setting TaskExecutionEvent version to 1 Signed-off-by: Daniel Rammer <[email protected]> * bumped flyteidl and flyteplugins version Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 11, 2022
-
Write certificate to local file (flyteorg#421)
* Make MetricsBindAddress configurable Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * Expose init cert Signed-off-by: Kevin Su <[email protected]> * Expose init cert Signed-off-by: Kevin Su <[email protected]> * Address webhook issue Signed-off-by: Kevin Su <[email protected]> * Fixed tests Signed-off-by: Kevin Su <[email protected]> * Make controller manager externally callable Signed-off-by: Kevin Su <[email protected]> * Lint Signed-off-by: Kevin Su <[email protected]> * comment Signed-off-by: Kevin Su <[email protected]>
-
Supporting interruptible for map tasks (flyteorg#415)
* added GetInterruptibleFailureThreshold function to nodeExecMetadata Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * Update to released flyteplugins Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
-
Move secrets util package to flyteplugins (flyteorg#420)
* removed the secrets package to flyteplugins and updated dependencies Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * Update deps Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
Commits on Apr 12, 2022
-
Moving all plugins to a common package, for easy loading (flyteorg#423)
* Moving all plugins to a common package, for easy loading Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]>
-
Build flytepropeller binary in github release
Signed-off-by: Haytham Abuelfutuh <[email protected]>
-
Use the correct path to main.go
Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Apr 13, 2022
-
Signed-off-by: Ketan Umare <[email protected]>
-
Bump boilerplate (flyteorg#426)
Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
Commits on Apr 14, 2022
Commits on Apr 15, 2022
-
Fix the Garbage collection problem (flyteorg#419)
* change label selector op Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * lint error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * test error Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * revert para name and keep old function Signed-off-by: Lisa <[email protected]> Signed-off-by: aeioulisa <[email protected]> * remove strconv.Itoa Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * two lebal Signed-off-by: aeioulisa <[email protected]> * reset config and add split function Signed-off-by: aeioulisa <[email protected]>
Commits on Apr 25, 2022
-
GC: Fix condition for finding completed workflows #major (flyteorg#430)
* Fix condition for finding completed workflows Signed-off-by: Haytham Abuelfutuh <[email protected]> * PR Comments Signed-off-by: Haytham Abuelfutuh <[email protected]> * More PR feedback Signed-off-by: Haytham Abuelfutuh <[email protected]> * unit test fixes Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update versions Signed-off-by: Haytham Abuelfutuh <[email protected]> * Unit test Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Apr 27, 2022
Commits on May 2, 2022
-
Apply interruptible override from workflow execution config #minor (f…
…lyteorg#429) * Workflows can now be marked as interruptible via execution config Allows for override of interruptible flag for a single execution, NodeDefaults still apply if not set in config Fixed minor typos in NodeSpec/nodeExecMetadata Signed-off-by: Nick Müller <[email protected]> * Updated golden files Signed-off-by: Nick Müller <[email protected]> * Interruptible override from execution config is now optional Allows for workflow interruptible flag to be overwritten to false as well (task settings still take preference) Signed-off-by: Nick Müller <[email protected]> * Use released flyteidl version Signed-off-by: Nick Müller <[email protected]>
Commits on May 5, 2022
Commits on May 17, 2022
-
Added grpc_client_handed_total prometheus metric for datacatalog (fly…
…teorg#437) * Added grpc_client_handed_total prometheus metric for datacatalog Signed-off-by: Prafulla Mahindrakar <[email protected]> * fixed lint issues Signed-off-by: Prafulla Mahindrakar <[email protected]>
-
deleting pod finalizers regardless of InjectFinalizer configuration (f…
…lyteorg#438) Signed-off-by: Daniel Rammer <[email protected]>
Commits on May 24, 2022
-
Strip Type Metadata (flyteorg#442)
* Strip Type Metadata Signed-off-by: Haytham Abuelfutuh <[email protected]> * Strip variable descriptions Signed-off-by: Haytham Abuelfutuh <[email protected]> * Strip Description Signed-off-by: Haytham Abuelfutuh <[email protected]> * Use a fixed time for datetime field literals Signed-off-by: Haytham Abuelfutuh <[email protected]> * Move logic to strip metadata to k8s transformer Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update code documentation Signed-off-by: Haytham Abuelfutuh <[email protected]> * PR Comments Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on May 27, 2022
-
Restrict garbage collection with sharded propeller (flyteorg#444)
* refactored to isolate computing sharded label selector requirements Signed-off-by: Daniel Rammer <[email protected]> * restricting garbage collection with sharded label selector requirements Signed-off-by: Daniel Rammer <[email protected]> * fix lint issue Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 1, 2022
-
Make metric keys configurable (flyteorg#446)
* making metric keys configurable Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * go generated to update config flags Signed-off-by: Daniel Rammer <[email protected]>
-
Update boilerplate version (flyteorg#445)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Jun 2, 2022
-
Upgrade flyteidl, preload lb policy and add default service config fo…
…r catalog client (flyteorg#447)
Commits on Jun 6, 2022
-
calling abort rather than finalize on permanent failure (flyteorg#449)
Signed-off-by: Daniel Rammer <[email protected]>
-
checking for NotFound grpc code instead of nil launchplan (flyteorg#441)
Signed-off-by: Daniel Rammer <[email protected]>
-
Graceful failure on ExecutionNotFound error (flyteorg#439)
* updated ExecutionNotFound to check failed attempts rather than workflow state Signed-off-by: Daniel Rammer <[email protected]> * added unit test to allow continue until maxRetries Signed-off-by: Daniel Rammer <[email protected]> * added dropping ExecutionNotFound error when aborting workflow Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 9, 2022
-
Fix prometheus metric panic when using additional labels (flyteorg#451)
* updated flytestdlib Signed-off-by: Daniel Rammer <[email protected]> * set flytestdlib to new release Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 16, 2022
-
Add support optional input (flyteorg#452)
* Support optional input Signed-off-by: Kevin Su <[email protected]> * updates Signed-off-by: Kevin Su <[email protected]> * updates Signed-off-by: Kevin Su <[email protected]> * update Signed-off-by: Kevin Su <[email protected]>
Commits on Jun 22, 2022
-
* Add deck_uri in NodeExecutionEvent Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * lint fix Signed-off-by: Kevin Su <[email protected]> * Updated idl Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Fix tests Signed-off-by: Kevin Su <[email protected]> * Updated idl Signed-off-by: Kevin Su <[email protected]> * address comment Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * updates Signed-off-by: Kevin Su <[email protected]> * updates Signed-off-by: Kevin Su <[email protected]> * More cleanup Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint Signed-off-by: Haytham Abuelfutuh <[email protected]> * PR comments Signed-off-by: Haytham Abuelfutuh <[email protected]> * Unit tests Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Kevin Su <[email protected]>
Commits on Jul 8, 2022
-
Delete stale map task interface altering code. (flyteorg#456)
Signed-off-by: Yee Hing Tong <[email protected]>
Commits on Jul 13, 2022
-
Add fold_logs.py script to summarize workflow execution by parsing lo…
…gs (flyteorg#434) * resurrected fold-logs.py script Signed-off-by: Daniel Rammer <[email protected]> * printing more info Signed-off-by: Daniel Rammer <[email protected]> * formatting output Signed-off-by: Daniel Rammer <[email protected]> * removed queue tracking Signed-off-by: Daniel Rammer <[email protected]> * added cache logs Signed-off-by: Daniel Rammer <[email protected]> * cleaning up block definitions for uniformity Signed-off-by: Daniel Rammer <[email protected]> * added comments Signed-off-by: Daniel Rammer <[email protected]> * added argparse Signed-off-by: Daniel Rammer <[email protected]> * tracking workflow enqueues on node updates Signed-off-by: Daniel Rammer <[email protected]> * moved fold-logs.py to a script directory Signed-off-by: Daniel Rammer <[email protected]> * parse gcp formatted logs (flyteorg#435) Co-authored-by: Babis Kiosidis <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Babis Kiosidis <[email protected]> Co-authored-by: Babis Kiosidis <[email protected]>
3 people authoredJul 13, 2022
Commits on Jul 15, 2022
-
update config.go (flyteorg#457)
* update config.go [Slack Conversation](https://flyte-org.slack.com/archives/C01P3B761A6/p1657755908722469) The timeouts for nodes/workflows are configured using different config.go files, depending on the deadlines (such as NodeExecution, NodeActive, and WorkflowActive deadlines). Specify that the times set by user includes only the time spent running and queued respectively. Signed-off-by: [email protected] Signed-off-by: SmritiSatyanV [email protected] Signed-off-by: SmritiSatyanV <[email protected]> * Make generate Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]>
-
[Snyk] Security upgrade alpine from 3.15 to 3.16 (flyteorg#455)
* fix: Dockerfile to reduce vulnerabilities Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update builder image as well Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
-
Update flyteidl version (flyteorg#454)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Jul 21, 2022
-
Add flag to export prometheus grpc latency metrics (flyteorg#460)
* add flag to enable latency metrics Signed-off-by: Sonja Ericsson <[email protected]> * fix Signed-off-by: Sonja Ericsson <[email protected]> * add import Signed-off-by: Sonja Ericsson <[email protected]> * moved import for lint Signed-off-by: Daniel Rammer <[email protected]> * updated config generation Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Sonja Ericsson <[email protected]>
Commits on Jul 26, 2022
-
Support external deletion of non-terminal map task subtasks (flyteorg…
…#462) * updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * bumping flyteplugins version Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jul 27, 2022
-
remove welcome bot from boilerplate (flyteorg#463)
Signed-off-by: Samhita Alla <[email protected]>
Commits on Jul 30, 2022
-
only writting subworkflow inputs once (flyteorg#464)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 5, 2022
-
Add nil check for output reader (flyteorg#465)
Signed-off-by: Kevin Su <[email protected]>
-
Enabled workqueue subqueue additions (flyteorg#466)
* enabled workqueue subqueue additions Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 15, 2022
-
Bump FlytePlugins version (flyteorg#469)
* Bump FlytePlugins version Signed-off-by: Kevin Su <[email protected]> * use ZZZ_DeprecatedClusterName instead Signed-off-by: Kevin Su <[email protected]> * lint Signed-off-by: Kevin Su <[email protected]> * lint Signed-off-by: Kevin Su <[email protected]> * go generate Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * update workflow name Signed-off-by: Kevin Su <[email protected]> * add comment Signed-off-by: Kevin Su <[email protected]> * Address comment Signed-off-by: Kevin Su <[email protected]> * fix test Signed-off-by: Kevin Su <[email protected]> * remove cluster name Signed-off-by: Kevin Su <[email protected]> * remove cluster name Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * test Signed-off-by: Kevin Su <[email protected]> * fix tests Signed-off-by: Kevin Su <[email protected]> * set timeout Signed-off-by: Kevin Su <[email protected]> * update viper Signed-off-by: Kevin Su <[email protected]> * Revert "update viper" This reverts commit 0ea7a9fdbc3120adaeb42fabca07f56c6cf454ff. * ping viper 0.9.0 Signed-off-by: Kevin Su <[email protected]> * updated stdlib Signed-off-by: Kevin Su <[email protected]> * nit Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]>
Commits on Aug 17, 2022
-
Handle WorkflowClosure from storage (flyteorg#459)
* add data reference fields Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * combine crd parts into one object/location Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * create and pass static obj storage to propeller handler Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * read static blob at the beginning and clear fields before updating the status Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * pass tests Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * remove terminated wfs' blob obj from cache Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * no return on remove blob method Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * test happy offloaded spec scenario Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * set static fields on every streak Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added crdOffloadStore interface Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * made crdoffloadstore configurable Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * add metrics/formatting Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * load static workflow data outside streak loop Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * cleaned up metric reporting Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * renamed inmemory to active Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added lruCRDOffloadStore unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added activeCRDOffloadStore unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added unit test for offloading crd error on handle Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * handle offloaded WorkflowClosure instead of parts of the crd Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * cache wf crd fields instead of wf closure Signed-off-by: Babis Kiosidis <[email protected]> * Update pkg/controller/controller.go Co-authored-by: Ketan Umare <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * reading workflow closure directly from data store rather than cache Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * added prometheus metric Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * updated comments Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Babis Kiosidis <[email protected]> Co-authored-by: Daniel Rammer <[email protected]> Co-authored-by: Ketan Umare <[email protected]>
4 people authoredAug 17, 2022 -
Lower-case mount path for file secrets (flyteorg#472)
* lower mount path Signed-off-by: Yee Hing Tong <[email protected]> * test Signed-off-by: Yee Hing Tong <[email protected]> * lower case the group also Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]>
Commits on Aug 25, 2022
-
avoid interacting with nil mutatedWfs (flyteorg#477)
* avoid interacting with nil mutatedWfs Signed-off-by: Babis Kiosidis <[email protected]> * only strip crd fields from mutated wfs Signed-off-by: Babis Kiosidis <[email protected]> * test failure in executor Signed-off-by: Babis Kiosidis <[email protected]> Signed-off-by: Babis Kiosidis <[email protected]>
Commits on Sep 1, 2022
-
Tracking terminated workflows in LRU cache (flyteorg#450)
* added lru filter to track terminated workflows Signed-off-by: Daniel Rammer <[email protected]> * updated unit tests Signed-off-by: Daniel Rammer <[email protected]> * add test on terminated filter Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Sep 8, 2022
-
Abort subworkflow on subnode failure (flyteorg#468)
* using 'failing' state to handle subworkflow aborts Signed-off-by: Daniel Rammer <[email protected]> * propogating node failure is subworkflow to subworkflow failure message in ui Signed-off-by: Daniel Rammer <[email protected]> * working with other failure scenarios Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * updated error message to match Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
-
Use correct node ID when recovery subworkflow nodes (flyteorg#481)
* using fully qualified node id on recovery Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Sep 9, 2022
-
Add structure attribute back to the CRD (flyteorg#479)
* Add structure attribute back to the CRD Signed-off-by: Kevin Su <[email protected]> * Add tests Signed-off-by: Kevin Su <[email protected]> * validate input before stripping metadata Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]>
-
Use fnv64 to create a hash for child workflow executions (flyteorg#476)
* Prefix sub-lp exec id with the parent exec-id Signed-off-by: Haytham Abuelfutuh <[email protected]> * cleanup Signed-off-by: Haytham Abuelfutuh <[email protected]> * cleanup Signed-off-by: Haytham Abuelfutuh <[email protected]> * Use a CRD-level version instead Signed-off-by: Haytham Abuelfutuh <[email protected]> * cleanup Signed-off-by: Haytham Abuelfutuh <[email protected]> * Use fnv64 to create a hash for child workflow executions Signed-off-by: Haytham Abuelfutuh <[email protected]> * Move DefinitionVersion to Status of the CRD Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update to the released flyteplugins Signed-off-by: Haytham Abuelfutuh <[email protected]> * Regenerate Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix unit tests Signed-off-by: Haytham Abuelfutuh <[email protected]> * more fixes Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix old unit test Signed-off-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Sep 16, 2022
-
changed default webhook failure policy to fail (flyteorg#473)
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Sep 19, 2022
-
Enable leader election on managed instances in a sharded deployment (f…
…lyteorg#461) * injecting leader election name if configured Signed-off-by: Daniel Rammer <[email protected]> * cleanup Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
Commits on Sep 20, 2022
-
Skip the structured dataset format check when the expected format is …
…empty (flyteorg#483) Signed-off-by: Kevin Su <[email protected]>
Commits on Sep 23, 2022
-
Fixing additional truncation of Error (flyteorg#478)
* Fixing additional truncation of Error Signed-off-by: Ketan Umare <[email protected]> * added trunc message Signed-off-by: Ketan Umare <[email protected]> * updated Signed-off-by: Ketan Umare <[email protected]> * lint fix Signed-off-by: Ketan Umare <[email protected]> * fixed generate Signed-off-by: Ketan Umare <[email protected]> Signed-off-by: Ketan Umare <[email protected]>
Commits on Sep 26, 2022
-
updated flyteplugins version (flyteorg#484)
Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Sep 29, 2022
-
Updated flyteplugins version for snowflake plugin (flyteorg#487)
* Update flyteplugin Signed-off-by: Kevin Su <[email protected]> * Update flyteplugin Signed-off-by: Kevin Su <[email protected]> * update endpoint Signed-off-by: Kevin Su <[email protected]> * update plugins Signed-off-by: Kevin Su <[email protected]> * bump flyteplugins Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]>
Commits on Sep 30, 2022
-
Update flyteidl version (flyteorg#467)
* Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Fix build break Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
3 people authoredSep 30, 2022
Commits on Oct 5, 2022
-
Make k8s secrets optional (flyteorg#482)
* make secrets optional - to skip mounting missing values Signed-off-by: Daniel Rammer <[email protected]> * add env var secrets to optional values Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Oct 6, 2022
-
Allow checkpoint resume when recovering a workflow (flyteorg#486)
* Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Fix build break Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Save/restore CheckpointUri from NodeExecution Signed-off-by: Andrew Dye <[email protected]> * Lints, generate Signed-off-by: Andrew Dye <[email protected]> * Fix log line Signed-off-by: Andrew Dye <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Andrew Dye <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
4 people authoredOct 6, 2022
Commits on Oct 11, 2022
-
disable k8s controller-runtime manager metrics server (flyteorg#492)
* setting MetricsBindAddress to 0 to disable controller-runtime manager metrics server Signed-off-by: Daniel Rammer <[email protected]> * and now in the webhook Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]>
Commits on Oct 12, 2022
-
fix: Add servicename in certs (flyteorg#491)
Before: A hardcoded string was used for setting the secret namespace After: The value for the secret namespace for settings is grabbed dynamically. Signed-off-by: Francisco J. Solis <[email protected]> Signed-off-by: Francisco J. Solis <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Oct 24, 2022
-
[core-Feature] Set deadlines to zero (flyteorg#493)
* Update config.go Set the default values to 0 Signed-off-by: LN <[email protected]> Signed-off-by: Ln11211 <[email protected]> * disable k8s controller-runtime manager metrics server (flyteorg#492) * setting MetricsBindAddress to 0 to disable controller-runtime manager metrics server Signed-off-by: Daniel Rammer <[email protected]> * and now in the webhook Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Ln11211 <[email protected]> * fix: Add servicename in certs (flyteorg#491) Before: A hardcoded string was used for setting the secret namespace After: The value for the secret namespace for settings is grabbed dynamically. Signed-off-by: Francisco J. Solis <[email protected]> Signed-off-by: Francisco J. Solis <[email protected]> Co-authored-by: Dan Rammer <[email protected]> Signed-off-by: Ln11211 <[email protected]> * Update config.go Removed DefaultDeadlines Signed-off-by: Ln11211 <[email protected]> Signed-off-by: LN <[email protected]> Signed-off-by: Ln11211 <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Francisco J. Solis <[email protected]> Co-authored-by: Dan Rammer <[email protected]> Co-authored-by: Francisco J. Solis <[email protected]>
3 people authoredOct 24, 2022 -
Move workflowstore logic to correct abstraction (flyteorg#496)
* Move logic to clear workflow CRD managed fields to passthrough workflowstore Signed-off-by: Daniel Shuy <[email protected]> * Move logic to track terminated workflows to new TerminatedTrackingStore workflowstore Signed-off-by: Daniel Shuy <[email protected]> Signed-off-by: Daniel Shuy <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Nov 1, 2022
-
Persist flyte deck during workflow recovery (flyteorg#495)
Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Nov 3, 2022
-
bump flyteplugin (flyteorg#499)
Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Nov 8, 2022
Commits on Nov 28, 2022
-
updating end2end tests from boilerplate (flyteorg#504)
Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
-
Bump flyteplugins to v1.0.20 (flyteorg#503)
Signed-off-by: live-wire <[email protected]> Signed-off-by: live-wire <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Nov 30, 2022
-
check deck uri exists before checking for datastore existance (flyteo…
…rg#506) Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
Commits on Dec 1, 2022
-
Add support for GateNode with signal and sleep condition (flyteorg#436)
* Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * Fix build break Signed-off-by: Haytham Abuelfutuh <[email protected]> * Update flyteidl version Signed-off-by: Flyte-Bot <[email protected]> * added GateNode to compiler Signed-off-by: Daniel Rammer <[email protected]> * added gate node handler Signed-off-by: Daniel Rammer <[email protected]> * enable reading and setting gate node state Signed-off-by: Daniel Rammer <[email protected]> * gate nodes working Signed-off-by: Daniel Rammer <[email protected]> * changed Conditional to Condition in proto naming Signed-off-by: Daniel Rammer <[email protected]> * passing admin client to gate node handler Signed-off-by: Daniel Rammer <[email protected]> * using signal service client to check for signal in admin and write output Signed-off-by: Daniel Rammer <[email protected]> * updated comments Signed-off-by: Daniel Rammer <[email protected]> * completed implementation Signed-off-by: Daniel Rammer <[email protected]> * added unit tests for gate node Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests with missing signal mocks Signed-off-by: Daniel Rammer <[email protected]> * added docs on gate node handler Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * updating flyteidl dependency Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * added output variable name to signal condition Signed-off-by: Daniel Rammer <[email protected]> * using last attempt started at timestamp on node context rather than tracking in gate node status Signed-off-by: Daniel Rammer <[email protected]> * updated GateNodeStatus mocks Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl deps Signed-off-by: Daniel Rammer <[email protected]> * update flyteidl deps Signed-off-by: Daniel Rammer <[email protected]> * added interface validation for approve condition Signed-off-by: Daniel Rammer <[email protected]> * added approve condition unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed missed merge conflict updating to slice of dial options Signed-off-by: Daniel Rammer <[email protected]> * update generated mocks Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
3 people authoredDec 1, 2022
Commits on Dec 5, 2022
Commits on Dec 8, 2022
-
Updated default subqueue base delay to 0 (#497)
* updated subqueue base delay to 0 Signed-off-by: Daniel Rammer <[email protected]> * increases rate and capacit of queue and subqueue Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
Commits on Dec 13, 2022
-
Fixed nil panic on gate node sleep check (flyteorg#507)
* fixed nil panic on gate node sleep check Signed-off-by: Dan Rammer <[email protected]> * lint issue Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
Commits on Dec 19, 2022
-
Add support Databricks plugin (flyteorg#510)
* update idl Signed-off-by: Kevin Su <[email protected]> * update idl Signed-off-by: Kevin Su <[email protected]> * update loader Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * wip Signed-off-by: Kevin Su <[email protected]> * Updare plugin Signed-off-by: Kevin Su <[email protected]> * Updare plugin Signed-off-by: Kevin Su <[email protected]> * Updare plugin Signed-off-by: Kevin Su <[email protected]> * Updare plugin Signed-off-by: Kevin Su <[email protected]> * Updare plugin Signed-off-by: Kevin Su <[email protected]> * update idl Signed-off-by: Kevin Su <[email protected]> * update Signed-off-by: Kevin Su <[email protected]> * updateidl Signed-off-by: Kevin Su <[email protected]> * go import Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]>
Commits on Dec 27, 2022
-
Fix issue where k8s plugin default labels override those set on pod (f…
…lyteorg#511) * fixing issue where k8s plugin default labels override those set on pod Signed-off-by: Dan Rammer <[email protected]> * updated flyteplugins dependency Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
Commits on Jan 3, 2023
-
Fix gate node approve condition node interface binding validation (fl…
…yteorg#512) * added random print statements Signed-off-by: Dan Rammer <[email protected]> * random print statement Signed-off-by: Dan Rammer <[email protected]> * infering gate node approve condition input and output variables from upstream nodes Signed-off-by: Dan Rammer <[email protected]> * removed print statements Signed-off-by: Dan Rammer <[email protected]> * fixed unit tests Signed-off-by: Dan Rammer <[email protected]> Signed-off-by: Dan Rammer <[email protected]>
Commits on Jan 5, 2023
-
Update flyteplugin version to v1.0.27 (flyteorg#513)
Signed-off-by: Ankit Goyal <[email protected]> Signed-off-by: Ankit Goyal <[email protected]>
Commits on Jan 7, 2023
-
* Add dask plugin Signed-off-by: Bernhard Stadlbauer <[email protected]> * Use `ToK8sResourceRequirements` from flyteplugins Signed-off-by: Bernhard Stadlbauer <[email protected]> * Run go mod tidy after merge Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update flyteplugins to 1.0.28 Signed-off-by: Bernhard Stadlbauer <[email protected]> * Fix linting error Signed-off-by: Bernhard Stadlbauer <[email protected]> Signed-off-by: Bernhard Stadlbauer <[email protected]>
Commits on Jan 9, 2023
-
Publish multi-architecture image (flyteorg#514)
Signed-off-by: Jeev B <[email protected]>
Commits on Jan 10, 2023
-
Add TensorFlow to loader.go (flyteorg#515)
* Add tensorflow to loader.go Signed-off-by: Kevin Su <[email protected]> * lint Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Kevin Su <[email protected]>
Commits on Jan 11, 2023
-
use different perm (flyteorg#517)
Signed-off-by: Yee Hing Tong <[email protected]> Signed-off-by: Yee Hing Tong <[email protected]>
Commits on Jan 19, 2023
-
Update flyteidl version (#490)
Signed-off-by: Flyte-Bot <[email protected]> Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]>
Commits on Feb 7, 2023
-
Updating PodTemplateStore API (flyteorg#516)
* updated pod template store handling Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * updating flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins versions Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * bumping go version to 1.19 Signed-off-by: Daniel Rammer <[email protected]> * fixed linter Signed-off-by: Daniel Rammer <[email protected]> * bumping setup-go action Signed-off-by: Daniel Rammer <[email protected]> * using different flytetools branch Signed-off-by: Daniel Rammer <[email protected]> * fixed go generate Signed-off-by: Daniel Rammer <[email protected]> * using master branch of flytetools Signed-off-by: Daniel Rammer <[email protected]> * removed secrets from unit tests Signed-off-by: Daniel Rammer <[email protected]> * maybe surround go-version in quotes Signed-off-by: Daniel Rammer <[email protected]> * specific 1.19.5 version Signed-off-by: Daniel Rammer <[email protected]> * moved go-version before secrets Signed-off-by: Daniel Rammer <[email protected]> * OMG Signed-off-by: Daniel Rammer <[email protected]> * updated flyte deps Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Feb 9, 2023
-
Inline input data for execution events (flyteorg#521)
* wip Signed-off-by: Katrina Rogan <[email protected]> * recovery Signed-off-by: Katrina Rogan <[email protected]> * test compile Signed-off-by: Katrina Rogan <[email protected]> * GH ur my fave diff tool Signed-off-by: Katrina Rogan <[email protected]> * revert styling changes Signed-off-by: Katrina Rogan <[email protected]> * more test Signed-off-by: Katrina Rogan <[email protected]> * goimports Signed-off-by: Katrina Rogan <[email protected]> * rm log line Signed-off-by: Katrina Rogan <[email protected]> * changes Signed-off-by: Katrina Rogan <[email protected]> * lint Signed-off-by: Katrina Rogan <[email protected]> * review comment Signed-off-by: Katrina Rogan <[email protected]> * go.mod Signed-off-by: Katrina Rogan <[email protected]> * review comment Signed-off-by: Katrina Rogan <[email protected]> --------- Signed-off-by: Katrina Rogan <[email protected]> Signed-off-by: Katrina Rogan <[email protected]> Co-authored-by: Katrina Rogan <[email protected]>
Commits on Feb 18, 2023
-
bump golang.org/x/net from 0.5.0 to 0.7.0 (#526)
Bumps [golang.org/x/net](https://github.com/golang/net) from 0.5.0 to 0.7.0. - [Release notes](https://github.com/golang/net/releases) - [Commits](golang/net@v0.5.0...v0.7.0) --- updated-dependencies: - dependency-name: golang.org/x/net dependency-type: indirect ... Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Commits on Feb 22, 2023
-
Update flyteplugins to 1.0.37 (flyteorg#527)
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Commits on Feb 25, 2023
-
fix recovery of already recovered workflow (flyteorg#530)
Signed-off-by: Felix Ruess <[email protected]>
Commits on Mar 4, 2023
-
Copy dynamicJobSpec from partially completed DynamicTask on recovery (f…
…lyteorg#518) * storing compiled workflow Signed-off-by: Daniel Rammer <[email protected]> * working Signed-off-by: Daniel Rammer <[email protected]> * added unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * recovering inputs of partially complete dynamic task Signed-off-by: Daniel Rammer <[email protected]> * removed dead code Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl version Signed-off-by: Daniel Rammer <[email protected]> * returning inputs from recoverInputs function Signed-off-by: Daniel Rammer <[email protected]> * fixed tests Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 6, 2023
-
bumping flyteplugins (flyteorg#537)
* bumping flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * and one more time Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
-
Extend pod webhook self-signed certs expiration date to 99 years in t…
…he future (flyteorg#535) Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
Commits on Mar 8, 2023
-
Structured Dataset with generic format should be castable to Flyte Sc…
…hema (flyteorg#536) * Structured Dataset with generic format should be castable to Schema Signed-off-by: Kevin Su <[email protected]> * Structured Dataset with generic format should be castable to Schema Signed-off-by: Kevin Su <[email protected]> * update Signed-off-by: Kevin Su <[email protected]> --------- Signed-off-by: Kevin Su <[email protected]>
Commits on Mar 21, 2023
-
bumping flyteplugins to 1.0.42 to pick up external cluster configurat…
…ion for ray (flyteorg#542) Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 24, 2023
-
Add reported_at timestamps to events (flyteorg#529)
* updated flyteidl Signed-off-by: Daniel Rammer <[email protected]> * updated pod timestamp recording and added recorded_at on task event Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins Signed-off-by: Daniel Rammer <[email protected]> * setting occurred_at when node inputs begin resolution Signed-off-by: Daniel Rammer <[email protected]> * added reported_at to node execution events Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins dependencies Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl and flyteplugins deps Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
-
Including all upstream node deps on BranchNode subnode execution (fly…
…teorg#543) * waiting for upstream nodes on branch subnode evaluation Signed-off-by: Daniel Rammer <[email protected]> * removed dead comments Signed-off-by: Daniel Rammer <[email protected]> * added unit test Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 30, 2023
-
persisting k8s plugin state between evaluations (flyteorg#540)
* persisting k8s plugin state between evaluations Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests and linter Signed-off-by: Daniel Rammer <[email protected]> * added docs Signed-off-by: Daniel Rammer <[email protected]> * updating flyteplugins dep Signed-off-by: Daniel Rammer <[email protected]> * added unit tests Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Mar 31, 2023
-
Add support for GCP secret manager (flyteorg#547)
Signed-off-by: Jeev B <[email protected]>
-
bumped flyteplugins (flyteorg#549)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 7, 2023
-
moved controller runtime start out of webhook Run function (flyteorg#546
) Signed-off-by: Daniel Rammer <[email protected]>
-
returning 'Undefined' when recovering a SKIPPED node so that the node…
… is executed (flyteorg#551) Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 11, 2023
-
Remove resource injection on the node for container task (flyteorg#544)
* Remove resource injection on the node for container task Signed-off-by: byhsu <[email protected]> * fix conflict Signed-off-by: byhsu <[email protected]> * fix lint Signed-off-by: byhsu <[email protected]> * downgrade plugin Signed-off-by: byhsu <[email protected]> --------- Signed-off-by: byhsu <[email protected]> Co-authored-by: byhsu <[email protected]>
Commits on Apr 12, 2023
-
Infer GOOS and GOARCH from environment (flyteorg#552)
Signed-off-by: Jeev B <[email protected]>
-
fix makefile to read variables from environment and overrides (flyteo…
…rg#554) Signed-off-by: Jeev B <[email protected]>
Commits on Apr 14, 2023
-
Remove BarrierTick (flyteorg#545)
* removed barrier logic Signed-off-by: Daniel Rammer <[email protected]> * deprecated TransitionTypeBarrier Signed-off-by: Daniel Rammer <[email protected]> * removed barrier tests Signed-off-by: Daniel Rammer <[email protected]> * bumping flyteplugins Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 16, 2023
-
Check for TerminateExecution error and eat Precondition status (flyte…
…org#553) * Check for TerminateExecution error and eat Precondition status Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint Signed-off-by: Haytham Abuelfutuh <[email protected]> --------- Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Apr 17, 2023
-
Setting primaryContainerName by default on Pod plugin (flyteorg#555)
* updated flyteplugins version Signed-off-by: Daniel Rammer <[email protected]> * go mod tidy Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 19, 2023
-
Implement ability to specify additional/override annotations when usi…
…ng Vault Secret Manager (flyteorg#556) * Implement ability to specify additional annotations when using Vault secret manager Signed-off-by: Pradithya Aria <[email protected]> * Infer GOOS and GOARCH from environment (flyteorg#552) Signed-off-by: Jeev B <[email protected]> Signed-off-by: Pradithya Aria <[email protected]> * fix makefile to read variables from environment and overrides (flyteorg#554) Signed-off-by: Jeev B <[email protected]> Signed-off-by: Pradithya Aria <[email protected]> * Remove BarrierTick (flyteorg#545) * removed barrier logic Signed-off-by: Daniel Rammer <[email protected]> * deprecated TransitionTypeBarrier Signed-off-by: Daniel Rammer <[email protected]> * removed barrier tests Signed-off-by: Daniel Rammer <[email protected]> * bumping flyteplugins Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Pradithya Aria <[email protected]> * Check for TerminateExecution error and eat Precondition status (flyteorg#553) * Check for TerminateExecution error and eat Precondition status Signed-off-by: Haytham Abuelfutuh <[email protected]> * lint Signed-off-by: Haytham Abuelfutuh <[email protected]> --------- Signed-off-by: Haytham Abuelfutuh <[email protected]> Signed-off-by: Pradithya Aria <[email protected]> * Rename to annotation Signed-off-by: Pradithya Aria <[email protected]> * Inline merging annotations Signed-off-by: Pradithya Aria <[email protected]> --------- Signed-off-by: Pradithya Aria <[email protected]> Signed-off-by: Jeev B <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Haytham Abuelfutuh <[email protected]> Co-authored-by: Jeev B <[email protected]> Co-authored-by: Dan Rammer <[email protected]> Co-authored-by: Haytham Abuelfutuh <[email protected]>
4 people authoredApr 19, 2023
Commits on Apr 20, 2023
-
Maintaining Interruptible and OverwriteCache for reference launchplans (
flyteorg#557) * maintaining Interruptible and OverwriteCache for reference launchplans Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Apr 28, 2023
-
Added support for aborting task nodes reported as failures (flyteorg#541
) * added CleanupOnFailure support for TaskNodeStatus to support aborting failed task nodes Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins and generated Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on May 3, 2023
-
Added support for EnvironmentVariables on ExecutionConfig (flyteorg#558)
* added environment variables to ExecutionConfig and propagated to taskExecutionMetadata Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * updated flyteplugins Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on May 5, 2023
-
Fast fail if task resource requests exceed k8s resource limits (flyte…
…org#488) * checking if task resource requests exceed k8s limits Signed-off-by: Daniel Rammer <[email protected]> * added better message to task failure Signed-off-by: Daniel Rammer <[email protected]> * added request checks Signed-off-by: Daniel Rammer <[email protected]> * added tests for checking resource eligibility Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * updated comment Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on May 11, 2023
-
Use group attribute for KV version and add DB engine support (flyteor…
…g#539) * Continue supporting config kvversion as default arg Signed-off-by: Tim Bauer <[email protected]> * Remove obsolete test Signed-off-by: Tim Bauer <[email protected]> * Add deprecation information Signed-off-by: Tim Bauer <[email protected]> * Remove comment Signed-off-by: Tim Bauer <[email protected]> * Remove comments Signed-off-by: Tim Bauer <[email protected]> --------- Signed-off-by: Tim Bauer <[email protected]>
Commits on May 16, 2023
-
Add grpc plugin to loader.go (flyteorg#562)
Signed-off-by: Kevin Su <[email protected]>
Commits on May 18, 2023
-
Use correct k8 client (flyteorg#563)
* Use correct k8 client Signed-off-by: Sonja Ericsson <[email protected]> * Use correct k8 client Signed-off-by: Sonja Ericsson <[email protected]> --------- Signed-off-by: Sonja Ericsson <[email protected]>
Commits on May 24, 2023
-
bumped flyteplugins (flyteorg#566)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on May 27, 2023
-
Bump flyteplugins to v1.0.63 (flyteorg#568)
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Commits on Jun 7, 2023
-
Not stripping structure from literal types (flyteorg#571)
* not stripping structure from literal types Signed-off-by: Daniel Rammer <[email protected]> * fixed new lint issues Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
-
Register gRPC plugin after reading configmap (flyteorg#564)
Signed-off-by: Kevin Su <[email protected]>
Commits on Jun 9, 2023
-
Update flyteplugins to v1.0.67 (flyteorg#575)
Signed-off-by: Bernhard Stadlbauer <[email protected]>
Commits on Jun 12, 2023
-
Correctly validating error code on aborting terminal launchplans (fly…
…teorg#574) * correctly checking error code Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issue Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 13, 2023
-
Use GetExecutionData instead (flyteorg#573)
* Use GetExecutionData instead Signed-off-by: Haytham Abuelfutuh <[email protected]> * bump Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix unit tests Signed-off-by: Haytham Abuelfutuh <[email protected]> * more fixes Signed-off-by: Haytham Abuelfutuh <[email protected]> * fix more Signed-off-by: Haytham Abuelfutuh <[email protected]> * Regenerate Signed-off-by: Haytham Abuelfutuh <[email protected]> * Only call GetExecutionData when the workflow succeeds Signed-off-by: Haytham Abuelfutuh <[email protected]> --------- Signed-off-by: Haytham Abuelfutuh <[email protected]>
Commits on Jun 26, 2023
-
propagating environment variables through launchplans (flyteorg#576)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 28, 2023
-
upgrade dockerfile go to 1.19 (flyteorg#580)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jun 30, 2023
-
Bump flyteplugins version (flyteorg#581)
Signed-off-by: Kevin Su <[email protected]>
-
Support for cross-project secrets for GCP (flyteorg#582)
* Support for cross-project secrets for GCP Signed-off-by: Jeev B <[email protected]> * Bump flyteplugins version (flyteorg#581) Signed-off-by: Kevin Su <[email protected]> Signed-off-by: Jeev B <[email protected]> --------- Signed-off-by: Jeev B <[email protected]> Signed-off-by: Kevin Su <[email protected]> Co-authored-by: Kevin Su <[email protected]>
Commits on Jul 5, 2023
-
Update
flyteplugins(flyteorg#578)* Update `flyteplugins` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Update `flytepropeller` Signed-off-by: Bernhard Stadlbauer <[email protected]> * Fix imports Signed-off-by: Bernhard Stadlbauer <[email protected]> * Bump flyteplugins to v1.1.7 Signed-off-by: Bernhard Stadlbauer <[email protected]> * Return `err` instead of `nil` Signed-off-by: Bernhard Stadlbauer <[email protected]> --------- Signed-off-by: Bernhard Stadlbauer <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
-
Fix usage of archive replacements (flyteorg#586)
Signed-off-by: eduardo apolinario <[email protected]> Co-authored-by: eduardo apolinario <[email protected]>
Commits on Jul 6, 2023
-
Added IsFailurePermanent flag on DynamicTaskStatus (flyteorg#567)
* added IsFailurePermanent flag on DynamicTaskStatus Signed-off-by: Daniel Rammer <[email protected]> * fixed linter Signed-off-by: Daniel Rammer <[email protected]> * cleaned up retryable vs permanent reporting Signed-off-by: Daniel Rammer <[email protected]> * make generate Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Jul 7, 2023
-
updating flyteplugins (flyteorg#587)
Signed-off-by: Daniel Rammer <[email protected]>
-
Bump flytestdlib (flyteorg#588)
Signed-off-by: eduardo apolinario <[email protected]> Co-authored-by: eduardo apolinario <[email protected]>
Commits on Jul 15, 2023
-
Update boilerplate version (flyteorg#591)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: eapolinario <[email protected]>
Commits on Jul 26, 2023
-
fixing max parallelism (flyteorg#594)
* fixing max parallelism Signed-off-by: Daniel Rammer <[email protected]> * Run subset of functional tests Signed-off-by: eduardo apolinario <[email protected]> * Comment basics.deck.wf test out Signed-off-by: eduardo apolinario <[email protected]> * Use flytetools@master in e2e tests and bring decks test back Signed-off-by: eduardo apolinario <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: eduardo apolinario <[email protected]> Co-authored-by: eduardo apolinario <[email protected]>
-
Updated flytestdlib 1.0.20 (flyteorg#593)
* updated flytestdlib Signed-off-by: Daniel Rammer <[email protected]> * fixing unit tests - using log events rather than unavailable admin Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
-
Update boilerplate version (flyteorg#597)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
3 people authoredJul 26, 2023
Commits on Jul 31, 2023
-
Instrument ArrayNode (flyteorg#550)
* updated flyteidl to local to get ArrayNode Signed-off-by: Daniel Rammer <[email protected]> * added boilerplate to support ArrayNode Signed-off-by: Daniel Rammer <[email protected]> * pushing forward Signed-off-by: Daniel Rammer <[email protected]> * refactored node executor interfaces to fix dependency cycle Signed-off-by: Daniel Rammer <[email protected]> * refactoring almost complete Signed-off-by: Daniel Rammer <[email protected]> * refactor complete Signed-off-by: Daniel Rammer <[email protected]> * supporting environment variables Signed-off-by: Daniel Rammer <[email protected]> * minimum viable product Signed-off-by: Daniel Rammer <[email protected]> * update print statements for debugging Signed-off-by: Daniel Rammer <[email protected]> * massive refactor fixing NodeExecutionContext override for ArrayNode Signed-off-by: Daniel Rammer <[email protected]> * refactoring TODOs Signed-off-by: Daniel Rammer <[email protected]> * subnode retries working Signed-off-by: Daniel Rammer <[email protected]> * parallelism working Signed-off-by: Daniel Rammer <[email protected]> * cache and cache_serialize working - first new functionality in maptask Signed-off-by: Daniel Rammer <[email protected]> * adding implementation notes Signed-off-by: Daniel Rammer <[email protected]> * removed eventing from subtasks Signed-off-by: Daniel Rammer <[email protected]> * adding correct requirements Signed-off-by: Daniel Rammer <[email protected]> * working end-2-end with flytekit Signed-off-by: Daniel Rammer <[email protected]> * reporting output directory on success Signed-off-by: Daniel Rammer <[email protected]> * fixed output directory append Signed-off-by: Daniel Rammer <[email protected]> * mocking TaskTemplate interface to enable caching Signed-off-by: Daniel Rammer <[email protected]> * capture failure reasons Signed-off-by: Daniel Rammer <[email protected]> * wrapped up abort and finalize functionality Signed-off-by: Daniel Rammer <[email protected]> * mocking initialization events Signed-off-by: Daniel Rammer <[email protected]> * sending all events Signed-off-by: Daniel Rammer <[email protected]> * minor refactoring of debug prints and formatting Signed-off-by: Daniel Rammer <[email protected]> * intratask checkpointing working Signed-off-by: Daniel Rammer <[email protected]> * support for and Signed-off-by: Daniel Rammer <[email protected]> * setting node log ids correctly Signed-off-by: Daniel Rammer <[email protected]> * reporting cache status Signed-off-by: Daniel Rammer <[email protected]> * correctly setting subnode abort phase Signed-off-by: Daniel Rammer <[email protected]> * removing dead code Signed-off-by: Daniel Rammer <[email protected]> * cleaned up most random TODO items Signed-off-by: Daniel Rammer <[email protected]> * refactored into new files Signed-off-by: Daniel Rammer <[email protected]> * refactoring for ArrayNode unit tests Signed-off-by: Daniel Rammer <[email protected]> * refactored for unit testing to allow creation of NodeExecutor in array package Signed-off-by: Daniel Rammer <[email protected]> * first unit test for handling ArrayNodePhaseNone Signed-off-by: Daniel Rammer <[email protected]> * most of executing unit tests completed Signed-off-by: Daniel Rammer <[email protected]> * finished executing unit tests Signed-off-by: Daniel Rammer <[email protected]> * finished succeeding unit tests Signed-off-by: Daniel Rammer <[email protected]> * wrote failing phase unit tests Signed-off-by: Daniel Rammer <[email protected]> * moving towards complete unit_test success Signed-off-by: Daniel Rammer <[email protected]> * unit tests passing Signed-off-by: Daniel Rammer <[email protected]> * fixed lint issues Signed-off-by: Daniel Rammer <[email protected]> * updated flyteidl dep Signed-off-by: Daniel Rammer <[email protected]> * added unit tests for Abort Signed-off-by: Daniel Rammer <[email protected]> * adding unit test for Finalize Signed-off-by: Daniel Rammer <[email protected]> * added utils unit tests Signed-off-by: Daniel Rammer <[email protected]> * moved state structs to handler package Signed-off-by: Daniel Rammer <[email protected]> * added docs Signed-off-by: Daniel Rammer <[email protected]> * cleaned up abort event reporting Signed-off-by: Daniel Rammer <[email protected]> * fixed RecordNodeEvent unit tests Signed-off-by: Daniel Rammer <[email protected]> * removed taskEventRecorder from nodes package Signed-off-by: Daniel Rammer <[email protected]> * adding interface checking for arraynode Signed-off-by: Daniel Rammer <[email protected]> * added transform unit test Signed-off-by: Daniel Rammer <[email protected]> * fixed input bindings issue Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * go generate Signed-off-by: Daniel Rammer <[email protected]> * addressing random TODO Signed-off-by: Daniel Rammer <[email protected]> * fixed unit tests Signed-off-by: Daniel Rammer <[email protected]> * addressing pr comments Signed-off-by: Daniel Rammer <[email protected]> --------- Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 3, 2023
-
make singular unions castable to their underlying type (flyteorg#599)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 4, 2023
-
fixed correct propagation of launchplan start error (flyteorg#598)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 6, 2023
-
Bumping flytestdlib and stow versions (flyteorg#602)
* go mod Signed-off-by: Gopal K. Vashishtha <[email protected]> * updating go mod Signed-off-by: Gopal K. Vashishtha <[email protected]> * bumping version Signed-off-by: Gopal K. Vashishtha <[email protected]> * some commenting Signed-off-by: Gopal K. Vashishtha <[email protected]> * make singular unions castable to their underlying type (flyteorg#599) Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Gopal K. Vashishtha <[email protected]> * fixed correct propagation of launchplan start error (flyteorg#598) Signed-off-by: Daniel Rammer <[email protected]> Signed-off-by: Gopal K. Vashishtha <[email protected]> * bumping flytestdlib Signed-off-by: Gopal K. Vashishtha <[email protected]> --------- Signed-off-by: Gopal K. Vashishtha <[email protected]> Signed-off-by: Daniel Rammer <[email protected]> Co-authored-by: Dan Rammer <[email protected]>
Commits on Aug 8, 2023
-
Update boilerplate version (flyteorg#601)
Signed-off-by: Flyte-Bot <[email protected]> Co-authored-by: flyte-bot <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
3 people authoredAug 8, 2023 -
Bump flyteplugins to 1.1.15 (flyteorg#603)
* Bump flyteplugins to 1.1.15 Signed-off-by: Eduardo Apolinario <[email protected]> * Drop replace directive from go.mod Signed-off-by: Eduardo Apolinario <[email protected]> * go mod tidy Signed-off-by: Eduardo Apolinario <[email protected]> --------- Signed-off-by: Eduardo Apolinario <[email protected]> Co-authored-by: Eduardo Apolinario <[email protected]>
-
updated flyteplugins to 1.1.16 (flyteorg#604)
Signed-off-by: Daniel Rammer <[email protected]>
Commits on Aug 9, 2023
-
Prepare monorepo - import eapolinario-flytepropeller
Signed-off-by: Eduardo Apolinario <[email protected]>
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.