Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update Flyte components #7

Open
wants to merge 25 commits into
base: master
Choose a base branch
from
Open

Conversation

eapolinario
Copy link
Owner

Updated flyte deployment

  • Updated GCP Flyte kustomize generated manifest file
  • Updated EKS Flyte kustomize generated manifest file
  • Updated Sandbox Flyte kustomize generated manifest file
  • Updated TEST Flyte kustomize generated manifest file
  • Updated GCP Flyte helm generated manifest file
  • Updated EKS Flyte helm generated manifest file
  • Updated Sandbox Flyte helm generated manifest file
  • Updated TEST Flyte helm generated manifest file
  • Auto-generated by [flyte-bot]

eapolinario and others added 21 commits August 8, 2023 14:17
…inario-flytepropeller

Prepare monorepo  import eapolinario flytepropeller
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Build single-binary using local components
…y_owner-everywhere

Use github.repository_owner everywhere
eapolinario added a commit that referenced this pull request Mar 11, 2024
* use-personal

* tidy

* update port to 30004

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bundle (#7)

* Remove private repos (flyteorg#4078)

Signed-off-by: Yee Hing Tong <[email protected]>

* nit from monorepo merge

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring in idl (flyteorg#4204)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artifacts bring admin monorepo (flyteorg#4203)

Main pr to bring in the boilerplate code for artifact service into the main pr.

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Artf/triggers (flyteorg#4394)

This PR adds the trigger concept.

Other changes:
* Forgot to make the index on artifactkey unique.
* Fix to create artifact, it wasn't pulling back out the artifact key.
* Updated the Artifact service to handle the passing of artifacts created by the events processor to the trigger handling components. Currently, and this is potentially a design flaw, the events processor has no way of contacting the trigger component directly.

Signed-off-by: Yee Hing Tong <[email protected]>

* make trigger key index unique (flyteorg#4410)

Signed-off-by: Yee Hing Tong <[email protected]>

* revert data proxy changes (flyteorg#4411)

Signed-off-by: Yee Hing Tong <[email protected]>

* artf/ fix issue with trigger (flyteorg#4412)

Signed-off-by: Yee Hing Tong <[email protected]>

* sandbox lite dockerfile (flyteorg#4413)

Signed-off-by: Yee Hing Tong <[email protected]>

* More pgconn handling fixes (flyteorg#4420)

Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/switch event (flyteorg#4428)

Most of the changes here are generated proto changes. Actual code changes are:
IDL
* Remove the supplemental fields in the CloudEventTaskExecution object and move them to CloudEventNodeExecution object.
* Remove some fields that the artifact service ended up not using (parent_node_execution and scheduled_at)

in the cloudevent publisher, change the code filling in of the aforementioned supplemental information to happen for node execution events instead of task execution events.
* Remove the deleted fields.

On the event handling side, move the logic to the handling of the node event instead of the task event.
Signed-off-by: Yee Hing Tong <[email protected]>

* Artf/lints (flyteorg#4429)

* lint fixes
* rename sandbox_utils without underscore

Signed-off-by: Yee Hing Tong <[email protected]>

* look for other places where pgconn is cast and add handling (flyteorg#4430)

Add additional handling of pgconn
* Change datacatalog logic to use flytestdlib function instead.
* Change flytestdlib constants to capitalize
* Add handling of the other type to Admin code directly

Signed-off-by: Yee Hing Tong <[email protected]>

* add artifacts branch to buf push (flyteorg#4448)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment (flyteorg#4451)

Signed-off-by: squiishyy <[email protected]>

* add dummy comment again (flyteorg#4453)

Signed-off-by: squiishyy <[email protected]>

* artf/updates to source (flyteorg#4443)


Signed-off-by: Yee Hing Tong <[email protected]>

* add a feature gate (flyteorg#4472)


Signed-off-by: Yee Hing Tong <[email protected]>

* fix bug in execution_manager with lpExpectedInputs, remove the standalone artifact client and move into idl clientset, update local config file, remove a deprecated function (flyteorg#4473)

Signed-off-by: Yee Hing Tong <[email protected]>

* initial deletion of artifact code

Signed-off-by: Yee Hing Tong <[email protected]>

* running make manifests in docker/sandbox-bundled to update generated files

Signed-off-by: Yee Hing Tong <[email protected]>

* now compiles (flyteorg#4481)

Signed-off-by: Yee Hing Tong <[email protected]>

* add a debug line

Signed-off-by: Yee Hing Tong <[email protected]>

* more debugging

Signed-off-by: Yee Hing Tong <[email protected]>

* clean up go mod and switch the order of the input extraction to check for uri first

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings in protos

Signed-off-by: squiishyy <[email protected]>

* testing removal of go mod pin

Signed-off-by: Yee Hing Tong <[email protected]>

* update go mod some more

Signed-off-by: Yee Hing Tong <[email protected]>

* copilot go mod

Signed-off-by: Yee Hing Tong <[email protected]>

* change artifacts http mappings again (flyteorg#4562)

Signed-off-by: squiishyy <[email protected]>

* panic and revert gorm log level change

Signed-off-by: Yee Hing Tong <[email protected]>

* revert local to 5

Signed-off-by: Yee Hing Tong <[email protected]>

* make helm

Signed-off-by: Yee Hing Tong <[email protected]>

* Reclaim space prior to running functional tests in single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Pin flytectl version to 0.8.0

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Pin flytectl version to 0.8.0"

This reverts commit cfa4745.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use flytagent v1.10.2

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use an infinite loop instead of watch (term env var is not set)

Signed-off-by: Eduardo Apolinario <[email protected]>

* Get logs from single-binary pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Debug single-binary pod: use a different way of choosing the pod

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix flytectl version

Signed-off-by: Eduardo Apolinario <[email protected]>

* Use port 5432 to connect to postgres

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Use port 5432 to connect to postgres"

This reverts commit 8ded67b.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Fix flytectl version"

This reverts commit 3968d40.

Signed-off-by: Eduardo Apolinario <[email protected]>

* fix database error

Signed-off-by: Yee Hing Tong <[email protected]>

* revert .github/workflows/single-binary.yml by pulling from master

Signed-off-by: Yee Hing Tong <[email protected]>

* Review feedback

Signed-off-by: Eduardo Apolinario <[email protected]>

---------

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>
Signed-off-by: squiishyy <[email protected]>
Co-authored-by: Joe Eschen <[email protected]>
Co-authored-by: squiishyy <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Eduardo Apolinario <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant