[BUG] RPC error in getting started section of the tutorial

[mfourt]

Hello everyone, we are currently testing flyte in a local environment with this tutorial : https://docs.flyte.org/en/latest/getting_started/index.html

We have an issue regarding the "getting started" section of the documentation.
We have followed each step to install flyte locally (we have used virtualenv to install python packages, cf flytekit).

We have tested successfully the file example.py with this command :

pyflyte run example.py wf --n 500 --mean 42 --sigma 2

Then we launch the demo flyte cluster using flytectl (flytectl demo start), everything has worked so far.

Here his the container launched :

CONTAINER ID   IMAGE                                                                                   COMMAND                  CREATED         STATUS         PORTS                                                                                                                                           NAMES
f9effbce13dc   cr.flyte.org/flyteorg/flyte-sandbox-lite:sha-106a8147446a6f0221162f47a9260ea0a764426e   "tini flyte-entrypoi…"   7 minutes ago   Up 7 minutes   0.0.0.0:30080-30082->30080-30082/tcp, 0.0.0.0:30084->30084/tcp, 0.0.0.0:30086->30086/tcp, 2375-2376/tcp, 0.0.0.0:30088-30090->30088-30090/tcp   flyte-sandbox

But when we try to execute the workflow in the cluster using this command :

pyflyte run --remote example.py wf --n 500 --mean 42 --sigma 2

we have this RPC error :

pyflyte run --remote example.py:wf --n 500 --mean 42 --sigma 2
E0621 09:23:13.543600943   15694 ssl_transport_security.cc:1504] Handshake failed with fatal error SSL_ERROR_SSL: error:100000f7:SSL routines:OPENSSL_internal:WRONG_VERSION_NUMBER.
{"asctime": "2022-06-21 09:23:13,547", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1655796193.546881581\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3128,\"referenced_errors\":[{\"created\":\"@1655796193.546880604\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":163,\"grpc_status\":14}]}\"\n>, sleeping 200ms and retrying"}
{"asctime": "2022-06-21 09:23:13,748", "name": "flytekit.cli", "levelname": "ERROR", "message": "Non-auth RPC error <_InactiveRpcError of RPC that terminated with:\n\tstatus = StatusCode.UNAVAILABLE\n\tdetails = \"failed to connect to all addresses\"\n\tdebug_error_string = \"{\"created\":\"@1655796193.747641936\",\"description\":\"Failed to pick subchannel\",\"file\":\"src/core/ext/filters/client_channel/client_channel.cc\",\"file_line\":3128,\"referenced_errors\":[{\"created\":\"@1655796193.747638015\",\"description\":\"failed to connect to all addresses\",\"file\":\"src/core/lib/transport/error_utils.cc\",\"file_line\":163,\"grpc_status\":14}]}\"\n>, sleeping 400ms and retrying"}
Traceback (most recent call last):
  File "/home/mfourt/.local/bin/pyflyte", line 8, in <module>
    sys.exit(main())
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/home/mfourt/.local/lib/python3.8/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/clis/sdk_in_container/run.py", line 152, in run
    wf = remote.register_script(
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/remote/remote.py", line 517, in register_script
    upload_location, md5_version = fast_register_single_script(
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/tools/script_mode.py", line 118, in fast_register_single_script
    upload_location = create_upload_location_fn(content_md5=md5)
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/clients/friendly.py", line 998, in get_upload_signed_url
    return super(SynchronousFlyteClient, self).create_upload_location(
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/clients/raw.py", line 40, in handler
    return fn(*args, **kwargs)
  File "/home/mfourt/.local/lib/python3.8/site-packages/flytekit/clients/raw.py", line 834, in create_upload_location
    return self._dataproxy_stub.CreateUploadLocation(create_upload_location_request, metadata=self._metadata)
  File "/home/mfourt/.local/lib/python3.8/site-packages/grpc/_channel.py", line 946, in __call__
    return _end_unary_response_blocking(state, call, False, None)
  File "/home/mfourt/.local/lib/python3.8/site-packages/grpc/_channel.py", line 849, in _end_unary_response_blocking
    raise _InactiveRpcError(state)
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
	status = StatusCode.UNAVAILABLE
	details = "failed to connect to all addresses"
	debug_error_string = "{"created":"@1655796194.149292252","description":"Failed to pick subchannel","file":"src/core/ext/filters/client_channel/client_channel.cc","file_line":3128,"referenced_errors":[{"created":"@1655796194.149289095","description":"failed to connect to all addresses","file":"src/core/lib/transport/error_utils.cc","file_line":163,"grpc_status":14}]}"

Can you give us an insight or any kind of clue you would have regarding this issue ?

Here is the version of docker and python we are using

Docker version 20.10.16, build aa7e414
Python 3.8.10

Expected behavior

We are able to lauch a worklow on the local cluster using the sandbox

Additional context to reproduce

No response

Screenshots

No response

Are you sure this issue hasn't been raised already?

• Yes

Have you read the Code of Conduct?

• Yes

[welcome]

Thank you for opening your first issue here! 🛠

[pingsutw]

@mfourt
Do you have a config file in the ~/.flyte directory?
~/.flyte/config

[platform]                                                                                        
url = localhost:30081
insecure = True

or ~/.flyte/config.yaml created by flytectl

admin:                                                                                            
  # For GRPC endpoints you might want to use dns:///flyte.myexample.com
  endpoint: dns:///demo.nuclyde.io:30081
  authType: Pkce
  insecure: true
logger:
  show-source: true
  level: 0

Try to explicitly pass the config.
pyflyte --config <config_file> run --remote example.py wf

[mfourt]

Hello @pingsutw,

it worked with the command you provided (passing the config file directly), thank you very much for your time and your solution