[BUG] Flyte console failed to fetch data

[pingsutw]

Describe the bug
After install minikube, I try to install flyte by
kubectl create -f https://raw.githubusercontent.com/lyft/flyte/master/deployment/sandbox/flyte_generated.yaml
I got the below error.

➜  kobe k get pods --namespace flyte                 
NAME                              READY   STATUS      RESTARTS   AGE
datacatalog-7f8ffdbc84-l74sz      1/1     Running     0          25m
flyteadmin-67dc44977f-g6zlx       2/2     Running     0          25m
flyteconsole-5f6d4d87db-gqs5s     1/1     Running     0          25m
flytepropeller-79659b75b4-7l2fs   1/1     Running     0          25m
minio-868dd88876-qll6b            1/1     Running     0          25m
postgres-c49d6ccf9-mt9dp          1/1     Running     0          25m
redis-0                           1/1     Running     0          25m
syncresources-1594804260-rsjmp    0/1     Completed   0          2m43s
syncresources-1594804320-lwd7t    0/1     Completed   0          103s
syncresources-1594804380-gkg2h    0/1     Completed   0          43s

➜  kobe k get svc --namespace flyte
NAME                     TYPE        CLUSTER-IP       EXTERNAL-IP   PORT(S)                AGE
datacatalog              ClusterIP   10.111.133.53    <none>        88/TCP,89/TCP          25m
flyteadmin               ClusterIP   10.100.10.60     <none>        87/TCP,80/TCP,81/TCP   25m
flyteconsole             NodePort    10.111.124.151   <none>        80:32037/TCP           25m
minio                    ClusterIP   10.109.199.56    <none>        9000/TCP               25m
minio-direct             NodePort    10.106.10.55     <none>        9000:30084/TCP         25m
postgres                 ClusterIP   10.106.46.116    <none>        5432/TCP               25m
postgres-direct          NodePort    10.103.121.49    <none>        5432:30083/TCP         25m
redis-resource-manager   ClusterIP   10.102.218.4     <none>        6379/TCP               25m

Expected behavior

Flyte component

• Overall
• Flyte Setup and Installation scripts
• Flyte Documentation
• Flyte communication (slack/email etc)
• FlytePropeller
• FlyteIDL (Flyte specification language)
• Flytekit (Python SDK)
• FlyteAdmin (Control Plane service)
• FlytePlugins
• DataCatalog
• FlyteStdlib (common libraries)
• FlyteConsole (UI)
• Other

To Reproduce
Steps to reproduce the behavior:

1. minikube start --vm-driver=none --cpus 6 --memory 12288 --disk-size=120g --extra-config=apiserver.authorization-mode=RBAC --extra-config=kubelet.resolv-conf=/run/systemd/resolve/resolv.conf --extra-config kubeadm.ignore-preflight-errors=SystemVerification --kubernetes-version v1.14.3

2. kubectl create -f https://raw.githubusercontent.com/lyft/flyte/master/deployment/sandbox/flyte_generated.yaml

Screenshots
If applicable, add screenshots to help explain your problem.

Environment
Flyte component

• Sandbox (local or on one machine)
• Cloud hosted
  • AWS
  • GCP
  • Azure
• Baremetal
• Other

Additional context
Add any other context about the problem here.

[pingsutw]

Sorry I use the wrong port.
30081 port works well for me

[kumare3]

@pingsutw is this a documentation bug? Any recommendations?

[yindia]

@kumare3 I am also facing same issue

[kumare3]

@evalsocket let me
Ping you on slack, are you using minikube?

[kumare3]

@evalsocket is this on kinD? Is this issue resolved?

[yindia]

It's kind. In minikube it's working fine

[emiliza]

btw: also able to reproduce this by deploying sandbox to GCP

[kumare3]

@emiliza how are you able to see the Console UI - are you port-forwarding?
So Let me explain how this works and this could help resolve the problem

So the console is a React App, when you port-forward the console service (or pod), only the react app is returned.

The React App then by default tries to use its window.location.origin to find the FlyteAdmin server - as explained here https://lyft.github.io/flyte/contributor/components/console.html?highlight=window%20location#environment-variables.
FlyteConsole then tries to query for FlyteAdmin on the same origin URL. This is because we assume that you are using ingress and thus both the FlyteConsole and FlyteAdmin are working on the same domain just on different routes
FlyteConsole on /console & FlyteAdmin on /api/v1/...

When you port-forward your window.location.origin is set to http://localhost:<forwarded port>.
But, on this port, FlyteAdmin is not really available.
AFAIK it is not possible to port-forward different services onto the same port.

So preference is to use GCP Ingress configuration https://cloud.google.com/kubernetes-engine/docs/concepts/ingress
OR, if you want to use port-forward then, you will have to change the following things

1. Add config for FlyteConsole to tell it where to find FlyteAdmin
   https://github.com/lyft/flyte/blob/master/kustomize/base/console/configmap.yaml#L10
   e.g.

ADMIN_API_URL: http://localhost:8089

if FlyteAdmin is port-forwarded to localhost:8089
NOTE: The URL includes the protocol

2. Modify config for FlyteAdmin to allow CORS
   https://github.com/lyft/flyte/blob/master/kustomize/overlays/sandbox/admindeployment/flyteadmin_config.yaml#L10
   Change and append

    allowCors: true
    allowedOrigins:
      - "http://localhost:<console_port>"
    allowedHeaders:
      - "Content-Type"

Also @emiliza Please find me on the Flyte-org slack channel and ping me. I would be more than happy to help :). My handle is ketan

[kumare3]

@emiliza I just created a PR - #477, to make all origins default enabled (CORS) in FlyteAdmin, its WIP

[kumare3]

Thank you for bringing the bug to my attention, we are changing the UI (trying out) to ensure that we can allow * as an acceptable origin for sandbox deployments

[kumare3]

@schottra this change makes it possible to set Wildcard CORs flyteorg/flyteconsole#89. Can we make it configurable when AUTH is disabled for this to be false as well

[kumare3]

FlyteAdmin in sandbox has CORS accept origin set to * and FlyteConsole has by default disabled secure cookies in Sandbox. IF anyone want to use port-forwarding then,

Step 1. Decide forwarded port for FlyteAdmin - e.g. port 8089

Step 2. Update flyte-console-config config.yaml in sandbox to add ADMIN_API_URL: http://localhost:8089. This instructs FlyteConsole to find FlyteAdmin on localhost:8089.

Step 3: Deploy (Ensure FlyteConsole pod has recycled / restarted, as the config is only loaded on startup)

Step 4: Port-forward FlyteAdmin

kubectl -n flyte port-forward svc/flyteadmin 8089:80

Step 5: Port-forward FlyteConsole and navigate to the port. e.g

kubectl -n flyte port-forward svc/flyteconsole 8080:80

Navigate: http://localhost:8080/console