Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove spark #684

Closed
wants to merge 2 commits into from
Closed

Remove spark #684

wants to merge 2 commits into from

Conversation

kumare3
Copy link
Contributor

@kumare3 kumare3 commented Jan 29, 2021

Contour version was very old. I replaced it with v0.11.1

Important note: we cannot directly use the remote yaml because it has
a namespace and kustomize's namespace transformer causes conflicts. So
i manually removed the namespace from the deployment

Important: Now console is simply available at
http://localhost/console if running locally
or the best way to get the console is

❯ kubectl get -n flyte service envoy -o wide
NAME    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
envoy   LoadBalancer   10.96.225.227   localhost     80:32613/TCP,443:30037/TCP   2m18s   app=envoy

You can use the External IP
http://<external-ip>/console

OR

kubectl port-forward -n flyte svc/envoy 80:80

then use
http://localhost/console

Contour version was very old. I replaced it with v0.11.1

Important note: we cannot directly use the remote yaml because it has
a namespace and kustomize's namespace transformer causes conflicts. So
i manually removed the namespace from the deployment

Important: Now console is simply available at
`localhost/console` if running locally
or the best way to get the console is

```bash
❯ kubectl get -n flyte service envoy -o wide
NAME    TYPE           CLUSTER-IP      EXTERNAL-IP   PORT(S)                      AGE     SELECTOR
envoy   LoadBalancer   10.96.225.227   localhost     80:32613/TCP,443:30037/TCP   2m18s   app=envoy
```

You can use the External IP
@kumare3 kumare3 closed this Jan 30, 2021
palchicz pushed a commit to palchicz/flyte that referenced this pull request Dec 23, 2021
* Run 3.9 in CI

Signed-off-by: Eduardo Apolinario <[email protected]>

* Also run 3.9 in plugins tests.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix tests

Signed-off-by: Eduardo Apolinario <[email protected]>

* Account for the different exception type raised in case of wrong types

Signed-off-by: Eduardo Apolinario <[email protected]>

* Exclude spark2

Signed-off-by: Eduardo Apolinario <[email protected]>

* Failed to load json_data to dataclass (flyteorg#684)

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Remove sync singledispatch, add option for top-level only sync (flyteorg#681)

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Failed to transform path string to Literal (flyteorg#689)

Signed-off-by: Kevin Su <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Don't node sync on remote wait (flyteorg#690)

* no node sync

Signed-off-by: Yee Hing Tong <[email protected]>

* add param to wait

Signed-off-by: Yee Hing Tong <[email protected]>
Signed-off-by: Eduardo Apolinario <[email protected]>

* Fix plugin regressions (flyteorg#688)

* fix pandera regression

Signed-off-by: Niels Bantilan <[email protected]>

* install plugin with pip

Signed-off-by: Niels Bantilan <[email protected]>

* fix pandera plugin tests

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* add spark flytekit plugin to papermill test_requires

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* add sqlalchemy to great expectations plugin

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* wip

Signed-off-by: Niels Bantilan <[email protected]>

* plugins plugins plugins!

Signed-off-by: Niels Bantilan <[email protected]>

* lint

Signed-off-by: Niels Bantilan <[email protected]>

* Exclude does not understand lists

Signed-off-by: Eduardo Apolinario <[email protected]>

* Invert python version check

Signed-off-by: Eduardo Apolinario <[email protected]>

* Invert python version check for real this time

Signed-off-by: Eduardo Apolinario <[email protected]>

* Enable 3.10 just for kicks

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add quotes around python versions

Yes, this is needed, please see https://dev.to/hugovk/the-python-3-1-problem-85g.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Add quotes around python versions"

This reverts commit 4d619d5c9e829aa1fdc18d2d87e1dbe4dbf42372.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Revert "Enable 3.10 just for kicks"

This reverts commit bd6d69462cb722afc5e2eb09ce77e35da512d288.

Signed-off-by: Eduardo Apolinario <[email protected]>

* wip - restricted types

Signed-off-by: Eduardo Apolinario <[email protected]>

* wip - restricted types

Signed-off-by: Eduardo Apolinario <[email protected]>

* Comment use of restricted types in get_transformer

Signed-off-by: Eduardo Apolinario <[email protected]>

* Publish 3.9 image

Signed-off-by: Eduardo Apolinario <[email protected]>

* Add python 3.9 to the list of supported languages

Signed-off-by: Eduardo Apolinario <[email protected]>

* Comment RestrictedTypeTransformer and add one test case.

Signed-off-by: Eduardo Apolinario <[email protected]>

* Review feedback

Signed-off-by: Eduardo Apolinario <[email protected]>

* Comment RestrictedTypeTransformer

Signed-off-by: Eduardo Apolinario <[email protected]>

* Handle the TypeError

Signed-off-by: Eduardo Apolinario <[email protected]>

* Remove breakpoint

Signed-off-by: Eduardo Apolinario <[email protected]>

Co-authored-by: Eduardo Apolinario <[email protected]>
Co-authored-by: Kevin Su <[email protected]>
Co-authored-by: Yee Hing Tong <[email protected]>
Co-authored-by: Niels Bantilan <[email protected]>
eapolinario pushed a commit to eapolinario/flyte that referenced this pull request Dec 20, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant