Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Prepare monorepo import eapolinario flytepropeller #3

Merged
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
592 commits
Select commit Hold shift + click to select a range
1c001ad
Do not add downstream links for sub-branch-nodes (#285)
EngHabu Jul 2, 2021
ff38e72
FlytePropeller should ignore if Admin is already in terminal state #p…
kumare3 Jul 7, 2021
6d99721
Add pod spec validation (#286)
Jul 9, 2021
cbf4405
Plugins error document origin (#289)
wild-endeavor Jul 13, 2021
9ba5beb
Construct subnode DataDir to be under parent's node OutputDir to keep…
EngHabu Jul 14, 2021
979eb2f
Revert "Construct subnode DataDir to be under parent's node OutputDir…
EngHabu Jul 15, 2021
1ee42d8
Fix case in which downstream nodes may incorrectly fail because of pa…
kumare3 Jul 15, 2021
19ae988
Update Boilerplate (#287)
flyte-bot Jul 17, 2021
05a09db
Update code of conduct (#296)
samhita-alla Jul 20, 2021
72525a2
Implement workflow execution recovery (#290)
Jul 20, 2021
73ac022
Simplify GetNodeExecutionStatus (#294)
EngHabu Jul 21, 2021
65bfdd6
Bump plugins to pick up ephemeral storage changes (#297)
Aug 9, 2021
c857c24
Bump idl version for ephemeral storage (#298)
Aug 9, 2021
c8b22b8
Add ephemeral storage to node resource requirements (#299)
Aug 10, 2021
20769f5
Bump flyteplugins to pull in sidecar resources fix (#300)
jeevb Aug 11, 2021
23f41a2
Update flyteplugins to 0.5.64 (#301)
jeevb Aug 12, 2021
c3dd4f5
Update Boilerplate (#302)
flyte-bot Aug 13, 2021
da345bd
Fix copy paste bug in recovering node exec inputs (#303)
Aug 16, 2021
fbb24fa
Upgrade flyteplugins to v0.5.67 (#307)
jeevb Aug 19, 2021
bab3c23
Fix pod creation issue when having multiple secret from same secret g…
pradithya Aug 20, 2021
c921349
bump flytestdlib version (#308)
yindia Aug 21, 2021
2cdb0e5
Bump flyteplugins to v0.5.68 (#309)
jeevb Aug 23, 2021
9378444
Update flyteplugins to v0.5.69 (#312)
jeevb Aug 24, 2021
1fc56b7
Add option to send raw output data in events (#304)
Aug 26, 2021
6e9629f
Update flyteplugins to v0.5.72 (#316)
jeevb Aug 30, 2021
8c854d5
Update README (#315)
pingsutw Aug 31, 2021
6ae7428
Resolve resource requirements in propeller at execution time (#310)
Sep 1, 2021
23f809c
Adopt flyteidl's ordered variable map change (#305)
mayitbeegh Sep 2, 2021
34fe469
changed node timeout logging to info level instead of error (#318)
hamersaw Sep 7, 2021
98dbaf9
#minor Revert "Adopt flyteidl's ordered variable map change" (#321)
mayitbeegh Sep 9, 2021
248f4cd
Better resrouce formatting (#313)
EngHabu Sep 13, 2021
7370e4d
Order UpstreamNodeIds to ensure consistent compiler output (#323)
EngHabu Sep 13, 2021
b705844
Use PluginCleanupPolicy from flyteorg/flyteplugins#203 (#311)
clairemcginty Sep 14, 2021
b802235
Start-Node Output file copied to output result (#317)
pmahindrakar-oss Sep 15, 2021
89a09c8
gracefully failing workflow on ExecutionNotFound error (#320)
hamersaw Sep 17, 2021
251b281
Update flyteplugins to 0.6.3 to pickup WebApi fix (#327)
EngHabu Sep 24, 2021
cde512f
Added snowflake plugin (#319)
pingsutw Oct 6, 2021
f20cbe3
Fix panic in Pod Webhook (#330)
EngHabu Oct 7, 2021
d869056
Replace v1.ResourceList in backoff controller with a thread-safe impl…
EngHabu Oct 8, 2021
e00d3fd
Switch to using encoder in pluginmachinery (#334)
mayitbeegh Oct 8, 2021
2140de7
Pick up latest plugins resource resolution & implement changes (#333)
Oct 11, 2021
cf1db85
Add Slack button to README (#337)
samhita-alla Oct 12, 2021
9a41c03
update slack URL (#340)
samhita-alla Oct 13, 2021
64820e6
Implements native k8s backend plugin for Kubeflow MPI operator. (#314)
yindia Oct 19, 2021
9242dd8
Do not run container as root #patch (#336)
frsann Oct 19, 2021
1372e2a
Kubeclient has a filter to prevent unnecessary calls to KubeAPI #mino…
kumare3 Oct 19, 2021
bd32547
replacing managed fields with a list of one empty instance to clear c…
hamersaw Oct 20, 2021
486632d
Fix invalid Kubernetes pod names (#325)
mayitbeegh Oct 21, 2021
37408ed
Add missing omitempty (#347)
kumare3 Oct 21, 2021
d5ed64e
Task parallelism updated. (#348)
kumare3 Oct 21, 2021
99c4dbc
Status compression: Clear the status for a node when the node was suc…
kumare3 Oct 21, 2021
7b09010
Create a flyteadmin event sink filter (#341)
hamersaw Oct 21, 2021
9e83dee
Vault Secret Manager #patch (#343)
bimtauer Oct 21, 2021
c8bf86c
Visibility: Better error message when resource quota is exceeded (#353)
kumare3 Oct 25, 2021
3c94c36
Truncate admin event error messages (#352)
hamersaw Oct 25, 2021
d875a22
Make default backoff MaxDuration 20sec (#354)
EngHabu Oct 26, 2021
0cac412
CRD validation on flytepropeller project structure (#328)
pingsutw Oct 28, 2021
3d16575
Revert "CRD validation on flytepropeller project structure (#328)" (#…
EngHabu Oct 28, 2021
85f5531
Adding shard, project, and domain labels to crd (#357)
hamersaw Nov 1, 2021
e238f9b
plugins v0.7.2 (#358)
migueltol22 Nov 1, 2021
5bcc859
Update Boilerplate (#359)
flyte-bot Nov 2, 2021
c517793
Bump flytestdlib to v0.4.4 (#361)
pingsutw Nov 10, 2021
32bfa8d
Fix violated uniqueness constraint on task events (#364)
kumare3 Nov 12, 2021
97d1775
Update Boilerplate (#365)
flyte-bot Nov 12, 2021
9db54f4
bump: flyteplugins v0.7.5 (#366)
yindia Nov 18, 2021
4d3a602
Run endtoend tests in PRs against sandbox releases (#350)
yindia Nov 23, 2021
dc5d314
#minor Intratask checkpointing in FlytePropeller (#360)
kumare3 Nov 24, 2021
709bb00
Cache Serialize API (#326)
hamersaw Dec 1, 2021
b5e533c
bump flytestdlib to 0.4.7 (#371)
ckiosidis Dec 2, 2021
83e93b9
Updated flyteplugins to support global pod security groups #patch (#369)
kumare3 Dec 3, 2021
32ef1ba
Scale out with propeller manager and workflow sharding (#351)
hamersaw Dec 3, 2021
7fe2d31
Update flyteplugins to fix pickup k8s array plugin fix for caching (#…
EngHabu Dec 9, 2021
679c305
Binding Panic when SDK produces an empty binding (#376)
EngHabu Dec 16, 2021
dca08df
Added script for checking diff of generated code (#372)
yindia Dec 21, 2021
5a7b696
Gracefully fail when NodeEvent exceeds gRPC size limit (#377)
hamersaw Dec 31, 2021
e446ac4
Update README.md (#380)
aeioulisa Jan 4, 2022
b135183
Handle incompatible cluster events errors (#378)
Jan 4, 2022
472621f
Create codeql-analysis.yml (#374)
EngHabu Jan 7, 2022
06e212c
Update Boilerplate (#368)
flyte-bot Jan 7, 2022
9db3ae4
Update Flyteidl version (#252)
flyte-bot Jan 7, 2022
5c927e7
Add structured dataset type checker (#384)
pingsutw Jan 11, 2022
0868be2
Added workflow for patch release (#387)
yindia Jan 23, 2022
090df53
Fix e2e sandbox for upgrade (#388)
yindia Jan 25, 2022
a3f73be
Bump flyteplugins for merged pod plugin (#386)
hamersaw Jan 25, 2022
3c90527
Add RetryAttempt and Phase to Task Event Metadata (#381)
hamersaw Jan 26, 2022
5047abf
Update Flyteidl version (#390)
flyte-bot Jan 29, 2022
3950f4a
Enable datacatalog client to use admin's auth option (#389)
mayitbeegh Feb 1, 2022
a6be7f5
A new metrics scope per plugin is created (#393)
kumare3 Feb 8, 2022
77b265a
Kenny/type metadata (#370)
kennyworkman Feb 8, 2022
7625cf3
Pick latest flytestdlib with metric name sanitization (#398)
kumare3 Feb 10, 2022
40adcd1
Added reuseable workflow (#391)
yindia Feb 10, 2022
83f5d32
Refactor for single binary (#399)
kumare3 Feb 15, 2022
6016844
Implement GetGeneratedNameWith (#400)
EngHabu Feb 15, 2022
5db62a0
fix missing error handling when creating mutatingwebhookconfiguration…
akumor Feb 16, 2022
c9e2e3f
Add BigQuery plugin (#394)
pingsutw Feb 16, 2022
72e8d0a
StructuredDataset without column should be castable to StructuredData…
pingsutw Feb 16, 2022
9adf523
Added reuseable functional test (#397)
yindia Feb 17, 2022
2cc2784
Update boilerplate version (#402)
flyte-bot Feb 18, 2022
b28526a
Hotfix docker image cache key in end2end test (#408)
yindia Mar 2, 2022
97d3bff
Take literal hash into account during cache key calculation (#406)
eapolinario Mar 3, 2022
50f0fd5
Update boilerplate version (#409)
flyte-bot Mar 4, 2022
bd1f108
Use PodTemplate metadata in managed Pods (#411)
hamersaw Mar 8, 2022
e30da6f
Only require default plugin configuration for task types with multipl…
hamersaw Mar 10, 2022
37b2e32
Fix map task with empty inputs (#403)
hamersaw Mar 14, 2022
8a7f2dc
Support default PodTemplate for k8s plugin (#404)
hamersaw Mar 16, 2022
4fc7a8f
bump latest flyteplugin (#413)
yindia Mar 18, 2022
e4bfe7c
Add Union Type validation (#373)
maximsmol Mar 22, 2022
cf88449
Remove extra SetMetricKeys (#416)
kumare3 Mar 24, 2022
15d2eab
Create FlyteWorkflow CRD on FlytePropeller startup (#414)
hamersaw Mar 31, 2022
f475c6c
Send IsParent & IsDynamic in node execution events (#418)
katrogan Apr 1, 2022
c4553bb
Reporting more complete metadata for map task subtasks (#417)
hamersaw Apr 8, 2022
ff9b4e8
Write certificate to local file (#421)
pingsutw Apr 11, 2022
f9e165b
Supporting interruptible for map tasks (#415)
hamersaw Apr 11, 2022
027fedb
Move secrets util package to flyteplugins (#420)
hamersaw Apr 11, 2022
3d48153
Moving all plugins to a common package, for easy loading (#423)
kumare3 Apr 12, 2022
1ece976
Build flytepropeller binary in github release
EngHabu Apr 12, 2022
0ca89ee
Use the correct path to main.go
EngHabu Apr 12, 2022
564c899
Pass missing execution params in ExecutionCreateRequest (#425)
EngHabu Apr 12, 2022
e694822
fixed webhook (#427)
kumare3 Apr 13, 2022
ed64dc1
Bump boilerplate (#426)
eapolinario Apr 13, 2022
3e361e8
update flyteplugins to v0.10.24 (#428)
katrogan Apr 14, 2022
76bb2c8
Fix the Garbage collection problem (#419)
aeioulisa Apr 15, 2022
363689e
GC: Fix condition for finding completed workflows #major (#430)
EngHabu Apr 25, 2022
499a01b
added myself to the CODEOWNERS file for default review on PRs (#432)
hamersaw Apr 27, 2022
a643548
Apply interruptible override from workflow execution config #minor (#…
MorpheusXAUT May 2, 2022
c9db886
Empty lists/maps should match list/maps of any types (#433)
EngHabu May 5, 2022
1655417
Added grpc_client_handed_total prometheus metric for datacatalog (#437)
pmahindrakar-oss May 17, 2022
af5f750
deleting pod finalizers regardless of InjectFinalizer configuration (…
hamersaw May 17, 2022
bc060ea
Strip Type Metadata (#442)
EngHabu May 24, 2022
774c457
Restrict garbage collection with sharded propeller (#444)
hamersaw May 27, 2022
bd4f6e2
Make metric keys configurable (#446)
hamersaw Jun 1, 2022
5b14d99
Update boilerplate version (#445)
flyte-bot Jun 1, 2022
66b0d90
Upgrade flyteidl, preload lb policy and add default service config fo…
honnix Jun 2, 2022
d0a8c57
calling abort rather than finalize on permanent failure (#449)
hamersaw Jun 6, 2022
88f7e3d
checking for NotFound grpc code instead of nil launchplan (#441)
hamersaw Jun 6, 2022
a0db3e4
Graceful failure on ExecutionNotFound error (#439)
hamersaw Jun 6, 2022
c712b32
Fix prometheus metric panic when using additional labels (#451)
hamersaw Jun 9, 2022
98667d4
Add support optional input (#452)
pingsutw Jun 16, 2022
dcf0344
Decks proposal (#453)
EngHabu Jun 22, 2022
6cd3279
Delete stale map task interface altering code. (#456)
wild-endeavor Jul 8, 2022
d7fb99b
Add fold_logs.py script to summarize workflow execution by parsing lo…
hamersaw Jul 13, 2022
0031707
update config.go (#457)
SmritiSatyanV Jul 15, 2022
994a707
[Snyk] Security upgrade alpine from 3.15 to 3.16 (#455)
snyk-bot Jul 15, 2022
7133dd5
Update flyteidl version (#454)
flyte-bot Jul 15, 2022
ac6d941
Add flag to export prometheus grpc latency metrics (#460)
hamersaw Jul 21, 2022
c2a41d3
Support external deletion of non-terminal map task subtasks (#462)
hamersaw Jul 26, 2022
725789d
remove welcome bot from boilerplate (#463)
samhita-alla Jul 27, 2022
7a920e0
only writting subworkflow inputs once (#464)
hamersaw Jul 30, 2022
07fc65a
Add nil check for output reader (#465)
pingsutw Aug 5, 2022
6f74890
Enabled workqueue subqueue additions (#466)
hamersaw Aug 5, 2022
a1c8d74
Bump FlytePlugins version (#469)
pingsutw Aug 15, 2022
e1649db
Handle WorkflowClosure from storage (#459)
ckiosidis Aug 17, 2022
5b28a98
Lower-case mount path for file secrets (#472)
wild-endeavor Aug 17, 2022
031a005
avoid interacting with nil mutatedWfs (#477)
ckiosidis Aug 25, 2022
7727851
Tracking terminated workflows in LRU cache (#450)
hamersaw Sep 1, 2022
98ee0f6
Abort subworkflow on subnode failure (#468)
hamersaw Sep 8, 2022
31c6efa
Use correct node ID when recovery subworkflow nodes (#481)
hamersaw Sep 8, 2022
47f5fdb
Add structure attribute back to the CRD (#479)
pingsutw Sep 9, 2022
60a71e3
Use fnv64 to create a hash for child workflow executions (#476)
EngHabu Sep 9, 2022
3036ace
changed default webhook failure policy to fail (#473)
hamersaw Sep 16, 2022
b2c0b97
Enable leader election on managed instances in a sharded deployment (…
hamersaw Sep 19, 2022
0d961dc
Skip the structured dataset format check when the expected format is …
pingsutw Sep 20, 2022
f3b0248
Fixing additional truncation of Error (#478)
kumare3 Sep 23, 2022
3fa9c10
updated flyteplugins version (#484)
hamersaw Sep 26, 2022
9d08c18
Updated flyteplugins version for snowflake plugin (#487)
pingsutw Sep 29, 2022
acfcc7c
Update flyteidl version (#467)
flyte-bot Sep 30, 2022
506cee1
Make k8s secrets optional (#482)
hamersaw Oct 5, 2022
7d6d09d
Allow checkpoint resume when recovering a workflow (#486)
andrewwdye Oct 6, 2022
20129fa
disable k8s controller-runtime manager metrics server (#492)
hamersaw Oct 11, 2022
b8fb08b
fix: Add servicename in certs (#491)
sisco0 Oct 12, 2022
3cf7fd8
[core-Feature] Set deadlines to zero (#493)
Ln11211 Oct 24, 2022
4db2319
Move workflowstore logic to correct abstraction (#496)
daniel-shuy Oct 24, 2022
fbe6f23
Persist flyte deck during workflow recovery (#495)
pingsutw Nov 1, 2022
47b0c0f
bump flyteplugin (#499)
pingsutw Nov 3, 2022
cbffd45
Skipping of cached task outputs via execution config (#489)
Nov 8, 2022
a536308
updating end2end tests from boilerplate (#504)
hamersaw Nov 28, 2022
19acf71
Bump flyteplugins to v1.0.20 (#503)
live-wire Nov 28, 2022
ca7c744
check deck uri exists before checking for datastore existance (#506)
hamersaw Nov 30, 2022
bf00d9f
Add support for GateNode with signal and sleep condition (#436)
hamersaw Dec 1, 2022
24ce23c
Fixed ExecutionInfo/TaskNodeInfo not being assigned properly for dyna…
Dec 5, 2022
6242e63
Updated default subqueue base delay to 0 (#497)
hamersaw Dec 8, 2022
3af5184
Fixed nil panic on gate node sleep check (#507)
hamersaw Dec 13, 2022
b1c8ae7
Add support Databricks plugin (#510)
pingsutw Dec 19, 2022
50f1aa4
Fix issue where k8s plugin default labels override those set on pod (…
hamersaw Dec 27, 2022
437fe5d
Fix gate node approve condition node interface binding validation (#…
hamersaw Jan 3, 2023
e6cedd2
Update flyteplugin version to v1.0.27 (#513)
goyalankit Jan 5, 2023
bd4bc68
Add dask plugin (#508)
bstadlbauer Jan 7, 2023
579f7e6
Publish multi-architecture image (#514)
jeevb Jan 9, 2023
8d212ab
Add TensorFlow to loader.go (#515)
pingsutw Jan 10, 2023
5da40a3
use different perm (#517)
wild-endeavor Jan 11, 2023
7fd4aea
Update flyteidl version (#490)
flyte-bot Jan 19, 2023
054230e
Updating PodTemplateStore API (#516)
hamersaw Feb 7, 2023
6737df3
Inline input data for execution events (#521)
katrogan Feb 9, 2023
1fe813c
bump golang.org/x/net from 0.5.0 to 0.7.0 (#526)
dependabot[bot] Feb 18, 2023
6e5ad21
Update flyteplugins to 1.0.37 (#527)
bstadlbauer Feb 22, 2023
c5ef97f
fix recovery of already recovered workflow (#530)
flixr Feb 25, 2023
703dbf2
Copy dynamicJobSpec from partially completed DynamicTask on recovery …
hamersaw Mar 4, 2023
bc7616d
bumping flyteplugins (#537)
hamersaw Mar 6, 2023
05288d2
Extend pod webhook self-signed certs expiration date to 99 years in t…
eapolinario Mar 6, 2023
84b755f
Structured Dataset with generic format should be castable to Flyte Sc…
pingsutw Mar 8, 2023
82b9fbf
bumping flyteplugins to 1.0.42 to pick up external cluster configurat…
hamersaw Mar 21, 2023
1c11504
Add reported_at timestamps to events (#529)
hamersaw Mar 24, 2023
6eb7e76
Including all upstream node deps on BranchNode subnode execution (#543)
hamersaw Mar 24, 2023
e48986d
persisting k8s plugin state between evaluations (#540)
hamersaw Mar 30, 2023
849530f
Add support for GCP secret manager (#547)
jeevb Mar 31, 2023
9df98f9
bumped flyteplugins (#549)
hamersaw Mar 31, 2023
355a140
moved controller runtime start out of webhook Run function (#546)
hamersaw Apr 7, 2023
109bfe7
returning 'Undefined' when recovering a SKIPPED node so that the node…
hamersaw Apr 7, 2023
2658724
Remove resource injection on the node for container task (#544)
ByronHsu Apr 11, 2023
3a1d98f
Infer GOOS and GOARCH from environment (#552)
jeevb Apr 12, 2023
36d84f7
fix makefile to read variables from environment and overrides (#554)
jeevb Apr 12, 2023
c828138
Remove BarrierTick (#545)
hamersaw Apr 14, 2023
b23415a
Check for TerminateExecution error and eat Precondition status (#553)
EngHabu Apr 16, 2023
0af9c55
Setting primaryContainerName by default on Pod plugin (#555)
hamersaw Apr 17, 2023
7b0243d
Implement ability to specify additional/override annotations when usi…
Apr 19, 2023
0487891
Maintaining Interruptible and OverwriteCache for reference launchplan…
hamersaw Apr 20, 2023
d84d9df
Added support for aborting task nodes reported as failures (#541)
hamersaw Apr 28, 2023
4faaa32
Added support for EnvironmentVariables on ExecutionConfig (#558)
hamersaw May 3, 2023
ca038b5
Fast fail if task resource requests exceed k8s resource limits (#488)
hamersaw May 5, 2023
05540d9
Use group attribute for KV version and add DB engine support (#539)
bimtauer May 11, 2023
b461ba5
Add grpc plugin to loader.go (#562)
pingsutw May 16, 2023
a1aefb0
Use correct k8 client (#563)
sonjaer May 18, 2023
100a3ba
bumped flyteplugins (#566)
hamersaw May 24, 2023
b8a854c
Bump flyteplugins to v1.0.63 (#568)
bstadlbauer May 27, 2023
f296ba9
Not stripping structure from literal types (#571)
hamersaw Jun 7, 2023
550b052
Register gRPC plugin after reading configmap (#564)
pingsutw Jun 7, 2023
7d091e6
Update flyteplugins to v1.0.67 (#575)
bstadlbauer Jun 9, 2023
977aaab
Correctly validating error code on aborting terminal launchplans (#574)
hamersaw Jun 12, 2023
10efce7
Use GetExecutionData instead (#573)
EngHabu Jun 13, 2023
257d2a0
propagating environment variables through launchplans (#576)
hamersaw Jun 26, 2023
262a43c
upgrade dockerfile go to 1.19 (#580)
hamersaw Jun 28, 2023
467ab2b
Bump flyteplugins version (#581)
pingsutw Jun 30, 2023
c7f894b
Support for cross-project secrets for GCP (#582)
jeevb Jun 30, 2023
c5555fc
Update `flyteplugins` (#578)
bstadlbauer Jul 5, 2023
eda29dd
Fix usage of archive replacements (#586)
eapolinario Jul 5, 2023
7010392
Added IsFailurePermanent flag on DynamicTaskStatus (#567)
hamersaw Jul 6, 2023
e4d16cd
updating flyteplugins (#587)
hamersaw Jul 7, 2023
7ae067c
Bump flytestdlib (#588)
eapolinario Jul 7, 2023
d60d711
Update boilerplate version (#591)
flyte-bot Jul 15, 2023
405bd6f
fixing max parallelism (#594)
hamersaw Jul 26, 2023
4633c4c
Updated flytestdlib 1.0.20 (#593)
hamersaw Jul 26, 2023
577b35e
Update boilerplate version (#597)
flyte-bot Jul 26, 2023
924f110
Instrument ArrayNode (#550)
hamersaw Jul 31, 2023
3bfb3ff
make singular unions castable to their underlying type (#599)
hamersaw Aug 3, 2023
95bdaf7
fixed correct propagation of launchplan start error (#598)
hamersaw Aug 4, 2023
799770c
Bumping flytestdlib and stow versions (#602)
gvashishtha Aug 6, 2023
20d8351
Update boilerplate version (#601)
flyte-bot Aug 8, 2023
c37a7e0
Bump flyteplugins to 1.1.15 (#603)
eapolinario Aug 8, 2023
5ea7dad
updated flyteplugins to 1.1.16 (#604)
hamersaw Aug 8, 2023
c5ba5dc
Prepare monorepo - import eapolinario-flytepropeller
eapolinario Aug 9, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
3 changes: 3 additions & 0 deletions flytepropeller/.dockerignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
vendor/*
bin/*
.github/*
82 changes: 82 additions & 0 deletions flytepropeller/.github/workflows/checks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,82 @@
name: Flytepropeller Checks

on:
workflow_dispatch:
pull_request:
branches:
- master
push:
branches:
- master

jobs:
lint:
name: Lint
uses: flyteorg/flytetools/.github/workflows/lint.yml@master
with:
go-version: "1.19"

tests:
name: Unit Tests
uses: flyteorg/flytetools/.github/workflows/tests.yml@master
with:
go-version: "1.19"
secrets:
FLYTE_BOT_PAT: ${{ secrets.FLYTE_BOT_PAT }}

docker-build:
name: Docker Build Images
uses: flyteorg/flytetools/.github/workflows/docker_build.yml@master

endtoend:
name: End2End Test
needs: [ docker-build ]
uses: flyteorg/flytetools/.github/workflows/end2end.yml@master
with:
cache_key: ${{ needs.docker-build.outputs.cache_key }}
priorities: "P0"

integration:
name: Integration Test
needs: [ docker-build ]
uses: flyteorg/flytetools/.github/workflows/integration.yml@master
with:
cache_key: ${{ needs.docker-build.outputs.cache_key }}
go-version: "1.19"

generate:
name: Check Go Generate
uses: flyteorg/flytetools/.github/workflows/go_generate.yml@master
with:
go-version: "1.19"

bump_version:
name: Bump Version
if: ${{ github.event_name != 'pull_request' }}
needs: [ endtoend, integration, lint, tests, generate ] # Only to ensure it can successfully build
uses: flyteorg/flytetools/.github/workflows/bump_version.yml@master
secrets:
FLYTE_BOT_PAT: ${{ secrets.FLYTE_BOT_PAT }}

goreleaser:
name: Goreleaser
needs: [ bump_version ] # Only to ensure it can successfully build
uses: flyteorg/flytetools/.github/workflows/goreleaser.yml@master
with:
go-version: "1.19"
secrets:
FLYTE_BOT_PAT: ${{ secrets.FLYTE_BOT_PAT }}

push_docker_image:
name: Build & Push Flytepropeller Image
needs: [ bump_version ]
uses: flyteorg/flytetools/.github/workflows/publish.yml@master
with:
version: ${{ needs.bump_version.outputs.version }}
dockerfile: Dockerfile
push: true
repository: ${{ github.repository }}
secrets:
FLYTE_BOT_PAT: ${{ secrets.FLYTE_BOT_PAT }}
FLYTE_BOT_USERNAME: ${{ secrets.FLYTE_BOT_USERNAME }}

74 changes: 74 additions & 0 deletions flytepropeller/.github/workflows/codeql-analysis.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,74 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL"

on:
push:
branches: [ master ]
pull_request:
# The branches below must be a subset of the branches above
branches: [ master ]
schedule:
- cron: '23 3 * * 6'

jobs:
analyze:
name: Analyze
runs-on: ubuntu-latest
permissions:
actions: read
contents: read
security-events: write

strategy:
fail-fast: false
matrix:
language: [ 'go' ]
# CodeQL supports [ 'cpp', 'csharp', 'go', 'java', 'javascript', 'python', 'ruby' ]
# Learn more about CodeQL language support at https://git.io/codeql-language-support

steps:
- name: Checkout repository
uses: actions/checkout@v3

- uses: actions/setup-go@v3
with:
go-version: '1.19'

# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v1
with:
languages: ${{ matrix.language }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# queries: ./path/to/local/query, your-org/your-repo/queries@main

# Autobuild attempts to build any compiled languages (C/C++, C#, or Java).
# If this step fails, then you should remove it and run the build manually (see below)
- name: Autobuild
uses: github/codeql-action/autobuild@v1

# ℹ️ Command-line programs to run using the OS shell.
# 📚 https://git.io/JvXDl

# ✏️ If the Autobuild fails above, remove it and uncomment the following three lines
# and modify them (or add more) to build your code if your project
# uses a compiled language

#- run: |
# make bootstrap
# make release

- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v1
19 changes: 19 additions & 0 deletions flytepropeller/.github/workflows/krew.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
name: releaser
on:
release:
types:
- published

jobs:
release-cli-via-krew:
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v2

- name: Unshallow
run: git fetch --prune --unshallow

- name: Update new version in krew-index
if: "!github.event.release.prerelease"
uses: rajatjindal/[email protected]
21 changes: 21 additions & 0 deletions flytepropeller/.github/workflows/upgrade_automation.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,21 @@
name: Upgrade Automation
on:
workflow_dispatch:
inputs:
component:
description: "Name of Flyte component"
required: true
default: "boilerplate"
type: choice
options:
- boilerplate
- flyteidl
jobs:
trigger-upgrade:
name: ${{ github.event.inputs.component }} Upgrade
uses: flyteorg/flytetools/.github/workflows/flyte_automation.yml@master
with:
component: ${{ github.event.inputs.component }}
go-version: 1.19
secrets:
FLYTE_BOT_PAT: ${{ secrets.FLYTE_BOT_PAT }}
6 changes: 6 additions & 0 deletions flytepropeller/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
.idea
vendor
bin
.DS_Store
_test
boilerplate/lyft/end2end/tmp
30 changes: 30 additions & 0 deletions flytepropeller/.golangci.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,30 @@
# WARNING: THIS FILE IS MANAGED IN THE 'BOILERPLATE' REPO AND COPIED TO OTHER REPOSITORIES.
# ONLY EDIT THIS FILE FROM WITHIN THE 'FLYTEORG/BOILERPLATE' REPOSITORY:
#
# TO OPT OUT OF UPDATES, SEE https://github.com/flyteorg/boilerplate/blob/master/Readme.rst

run:
skip-dirs:
- pkg/client

linters:
disable-all: true
enable:
- deadcode
- errcheck
- gas
- goconst
- goimports
- golint
- gosimple
- govet
- ineffassign
- misspell
- nakedret
- staticcheck
- structcheck
- typecheck
- unconvert
- unparam
- unused
- varcheck
86 changes: 86 additions & 0 deletions flytepropeller/.goreleaser.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
project_name: kubectl-flyte
before:
hooks:
- go mod download
builds:
- id: kubectl-flyte
env:
- CGO_ENABLED=0
main: ./cmd/kubectl-flyte/main.go
binary: kubectl-flyte
goos:
- linux
- windows
- darwin
ldflags:
- -s -w -X github.com/lyft/flytepropeller/version.Version={{.Version}} -X github.com/lyft/flytepropeller/version.Build={{.ShortCommit}} -X github.com/lyft/flytepropeller/version.BuildTime={{.Date}}
- id: flytepropeller
env:
- CGO_ENABLED=0
main: ./cmd/controller/main.go
ldflags:
- -s -w -X github.com/flyteorg/flytestdlib/version.Version={{.Version}} -X github.com/flyteorg/flytestdlib/version.Build={{.ShortCommit}} -X github.com/flyteorg/flytestdlib/version.BuildTime={{.Date}}

binary: flytepropeller
goos:
- linux
- windows
- darwin
archives:
- id: kubectl-flyte-archive
name_template: |-
kubectl-flyte_{{ .Tag }}_{{ .Os }}_
{{- if eq .Arch "amd64" }}x86_64
{{- else if eq .Arch "386" }}i386
{{- else }}{{ .Arch }}{{ end }}
builds:
- kubectl-flyte
format_overrides:
- goos: windows
format: zip
- id: flytepropeller-archive
name_template: |-
flytepropeller_{{ .Tag }}_{{ .Os }}_
{{- if eq .Arch "amd64" }}x86_64
{{- else if eq .Arch "386" }}i386
{{- else }}{{ .Arch }}{{ end }}
builds:
- flytepropeller
format_overrides:
- goos: windows
format: zip

checksum:
name_template: 'checksums.txt'
changelog:
sort: asc
filters:
exclude:
- '^docs:'
- '^test:'
# scoop:
# # Default is "https://github.com/<repo_owner>/<repo_name>/releases/download/{{ .Tag }}/{{ .ArtifactName }}"
# # url_template: "http://github.mycompany.com/foo/bar/releases/{{ .Tag }}/{{ .ArtifactName }}"

# # Repository to push the app manifest to.
# bucket:
# owner: lyft
# name: flytepropeller

# # Git author used to commit to the repository.
# # Defaults are shown.
# commit_author:
# name: goreleaserbot
# email: [email protected]

# # Your app's homepage.
# # Default is empty.
# homepage: "https://godoc.org/github.com/lyft/flytepropeller"

# # Your app's description.
# # Default is empty.
# description: "kubectl-flyte is an command line tool that can be used as an extension to kubectl"

# # Your app's license
# # Default is empty.
# license: Apache-2.0
38 changes: 38 additions & 0 deletions flytepropeller/.krew.yaml
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
apiVersion: krew.googlecontainertools.github.com/v1alpha2
kind: Plugin
metadata:
name: flyte
spec:
version: "{{ .TagName }}"
homepage: https://github.com/flyteorg/flyte
shortDescription: Monitor, launch and manage flyte executions
description: |
Kubectl-flyte can be used to manage workflows in a kubernetes cluster executed
by FlytePropeller. Users can monitor, visualize executing workflows, delete or
terminate executions in bulk, and dive into quota utilization for tenants. The
tool allows users to launch yaml based Flyte workflows without FlyteAdmin.

For more information about Flyte refer to https://flyte.org
platforms:
- selector:
matchLabels:
os: darwin
arch: amd64
{{addURIAndSha "https://github.com/flyteorg/flytepropeller/releases/download/{{ .TagName }}/kubectl-flyte_{{ .TagName }}_darwin_x86_64.tar.gz" .TagName | indent 6}}
files:
- from: kubectl-flyte
to: flyte
- from: LICENSE
to: .
bin: flyte
- selector:
matchLabels:
os: linux
arch: amd64
{{addURIAndSha "https://github.com/flyteorg/flytepropeller/releases/download/{{ .TagName }}/kubectl-flyte_{{ .TagName }}_linux_x86_64.tar.gz" .TagName | indent 6}}
files:
- from: kubectl-flyte
to: flyte
- from: LICENSE
to: .
bin: flyte
3 changes: 3 additions & 0 deletions flytepropeller/CODEOWNERS
Validating CODEOWNERS rules …
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
# These owners will be the default owners for everything in
# the repo. Unless a later match takes precedence.
* @kumare3 @EngHabu @hamersaw
2 changes: 2 additions & 0 deletions flytepropeller/CODE_OF_CONDUCT.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
This project is governed by LF AI Foundation's [code of conduct](https://lfprojects.org/policies/code-of-conduct/).
All contributors and participants agree to abide by its terms.
Loading