Fail to run "az connectedk8s connect"

[chhtw]

I tried to follow https://docs.microsoft.com/en-us/azure/azure-arc/kubernetes/connect-cluster to connect k8s, but got an error after I ran the command "az connectedk8s connect", not quite sure how to fix it.

The command failed with an unexpected error. Here is the traceback: Expecting value: line 1 column 1 (char 0) Traceback (most recent call last): File "/opt/az/lib/python3.6/site-packages/knack/cli.py", line 215, in invoke cmd_result = self.invocation.execute(args) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 654, in execute raise ex File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 718, in _run_jobs_serially results.append(self._run_job(expanded_arg, cmd_copy)) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 711, in _run_job six.reraise(*sys.exc_info()) File "/opt/az/lib/python3.6/site-packages/six.py", line 703, in reraise raise value File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 688, in _run_job result = cmd_copy(params) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/commands/__init__.py", line 325, in __call__ return self.handler(*args, **kwargs) File "/opt/az/lib/python3.6/site-packages/azure/cli/core/__init__.py", line 545, in default_command_handler return op(**command_args) File "/home/myadmin/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 209, in create_connectedk8s registry_path = os.getenv('HELMREGISTRY') if os.getenv('HELMREGISTRY') else get_helm_registry(profile, location) File "/home/myadmin/.azure/cliextensions/connectedk8s/azext_connectedk8s/custom.py", line 408, in get_helm_registry telemetry.set_exception(exception=str(response.json()), fault_type=Get_HelmRegistery_Path_Fault_Type, File "/opt/az/lib/python3.6/site-packages/requests/models.py", line 897, in json return complexjson.loads(self.text, **kwargs) File "/opt/az/lib/python3.6/json/__init__.py", line 354, in loads return _default_decoder.decode(s) File "/opt/az/lib/python3.6/json/decoder.py", line 339, in decode obj, end = self.raw_decode(s, idx=_w(s, 0).end()) File "/opt/az/lib/python3.6/json/decoder.py", line 357, in raw_decode raise JSONDecodeError("Expecting value", s, err.value) from None json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name
az connectedk8s connect Extension Name: connectedk8s. Version: 0.2.1.

Errors:

Expecting value: line 1 column 1 (char 0)
Traceback (most recent call last):
python3.6/site-packages/knack/cli.py, ln 215, in invoke
    cmd_result = self.invocation.execute(args)
cli/core/commands/__init__.py, ln 654, in execute
    raise ex
cli/core/commands/__init__.py, ln 718, in _run_jobs_serially
    results.append(self._run_job(expanded_arg, cmd_copy))
cli/core/commands/__init__.py, ln 711, in _run_job
    six.reraise(*sys.exc_info())
...
lib/python3.6/json/decoder.py, ln 339, in decode
    obj, end = self.raw_decode(s, idx=_w(s, 0).end())
lib/python3.6/json/decoder.py, ln 357, in raw_decode
    raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)

To Reproduce:

Steps to reproduce the behavior. Note that argument values have been redacted, as they may contain sensitive information.

• Put any pre-requisite steps here...
• az connectedk8s connect --name {} --resource-group {} --debug

Expected Behavior

Environment Summary

Linux-5.3.0-1031-azure-x86_64-with-debian-buster-sid
Python 3.6.10
Installer: DEB

azure-cli 2.8.0

Extensions:
k8sconfiguration 0.1.8
connectedk8s 0.2.1

Additional Context

[ghost]

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @akashkeshari.

[yonzhan]

connectedk8s

[akashkeshari]

@chhtw ,

Can you run the following command and share the output so that we can look into the issue:
'az rest -m post -u https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview --resource https://management.core.windows.net/ --debug'

To unblock you, run the following commands:

1. export HELMREGISTRY=azurearcfork8s.azurecr.io/batch1/stable/azure-arc-k8sagents:0.2.4
2. az connectedk8s connect --name {} --resource-group {}

Let me know if you face any issues following above steps.

Thanks,
Akash

[chhtw]

@akashkeshari , Here is the log for 'az rest -m post -u https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview --resource https://management.core.windows.net/ --debug'

Command arguments: ['rest', '-m', 'post', '-u', 'https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview', '--resource', 'https://management.core.windows.net/', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x7f32a5223e18>, <function OutputProducer.on_global_arguments at 0x7f32a4ff8e18>, <function CLIQuery.on_global_arguments at 0x7f32a4da00d0>]
Event: CommandInvoker.OnPreCommandTableCreate []
Installed command modules ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'aro', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'configure', 'consumption', 'container', 'cosmosdb', 'deploymentmanager', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'iotcentral', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'monitor', 'natgateway', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'util', 'vm']
Loaded module 'acr' in 0.027 seconds.
Loaded module 'acs' in 0.109 seconds.
Loaded module 'advisor' in 0.008 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'ams' in 0.030 seconds.
Loaded module 'apim' in 0.018 seconds.
Loaded module 'appconfig' in 0.019 seconds.
Loaded module 'appservice' in 1.955 seconds.
Loaded module 'aro' in 0.152 seconds.
Loaded module 'backup' in 0.031 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'batch' in 0.245 seconds.
Loaded module 'batchai' in 0.016 seconds.
Loaded module 'billing' in 0.013 seconds.
Loaded module 'botservice' in 0.008 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'cdn' in 0.062 seconds.
Loaded module 'cloud' in 0.008 seconds.
Loaded module 'cognitiveservices' in 0.009 seconds.
Loaded module 'configure' in 0.007 seconds.
Loaded module 'consumption' in 0.013 seconds.
Loaded module 'container' in 0.008 seconds.
Loaded module 'cosmosdb' in 0.033 seconds.
Loaded module 'deploymentmanager' in 0.017 seconds.
Loaded module 'dla' in 0.021 seconds.
Loaded module 'dls' in 0.019 seconds.
Loaded module 'dms' in 0.007 seconds.
Loaded module 'eventgrid' in 0.012 seconds.
Loaded module 'eventhubs' in 0.019 seconds.
Loaded module 'extension' in 0.006 seconds.
Loaded module 'feedback' in 0.001 seconds.
Loaded module 'find' in 0.043 seconds.
Loaded module 'hdinsight' in 0.008 seconds.
Loaded module 'interactive' in 0.007 seconds.
Loaded module 'iot' in 0.022 seconds.
Loaded module 'iotcentral' in 0.008 seconds.
Loaded module 'keyvault' in 0.021 seconds.
Loaded module 'kusto' in 0.008 seconds.
Loaded module 'lab' in 0.014 seconds.
Loaded module 'managedservices' in 0.006 seconds.
Loaded module 'maps' in 0.007 seconds.
Loaded module 'monitor' in 0.032 seconds.
Loaded module 'natgateway' in 0.012 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'netappfiles' in 0.008 seconds.
Loaded module 'network' in 0.061 seconds.
Loaded module 'policyinsights' in 0.013 seconds.
Loaded module 'privatedns' in 0.014 seconds.
Loaded module 'profile' in 0.007 seconds.
Loaded module 'rdbms' in 0.019 seconds.
Loaded module 'redis' in 0.007 seconds.
Loaded module 'relay' in 0.016 seconds.
Loaded module 'reservations' in 0.007 seconds.
Loaded module 'resource' in 0.035 seconds.
Loaded module 'role' in 0.015 seconds.
Loaded module 'search' in 0.007 seconds.
Loaded module 'security' in 0.007 seconds.
Loaded module 'servicebus' in 0.015 seconds.
Loaded module 'servicefabric' in 0.015 seconds.
Loaded module 'signalr' in 0.011 seconds.
Loaded module 'sql' in 0.019 seconds.
Loaded module 'sqlvm' in 0.012 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'storage' in 0.083 seconds.
Loaded module 'util' in 0.006 seconds.
Loaded module 'vm' in 0.071 seconds.
Loaded all modules in 3.510 seconds. (note: there's always an overhead with the first module loaded)
Found 2 extensions: ['k8sconfiguration', 'connectedk8s']
Extension compatibility result: is_compatible=True cli_core_version=2.8.0 min_required=2.3.1 max_required=None
Extensions directory: '/home/myadmin/.azure/cliextensions/k8sconfiguration'
Loaded extension 'k8sconfiguration' in 0.014 seconds.
Extension compatibility result: is_compatible=True cli_core_version=2.8.0 min_required=2.0.67 max_required=None
Extensions directory: '/home/myadmin/.azure/cliextensions/connectedk8s'
Loaded extension 'connectedk8s' in 0.205 seconds.
Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x7f32a4733950>]
az_command_data_logger : command args: rest -m {} -u {} --resource {} --debug
metadata file logging enabled - writing logs to '/home/myadmin/.azure/commands'.
Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument..add_subscription_parameter at 0x7f32a470cae8>]
Event: CommandInvoker.OnPostArgumentLoad []
Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument..add_ids_arguments at 0x7f32a470c950>, <function register_cache_arguments..add_cache_arguments at 0x7f32a470ca60>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x7f32a26c22f0>]
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x7f32a4ff8ea0>, <function CLIQuery.handle_query_parameter at 0x7f32a4da0158>, <function register_ids_argument..parse_ids_arguments at 0x7f32a470c9d8>, <function handler at 0x7f32a251fd08>]
Current cloud config:
AzureCloud
Retrieving token for resource https://management.core.windows.net/
attempting to read file /home/myadmin/.azure/accessTokens.json as utf-8-sig
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - Authority:Performing instance discovery: ...
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - Authority:Performing static instance discovery
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - Authority:Authority validated via static instance discovery
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - TokenRequest:Getting token from cache with refresh if necessary.
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:finding with query keys: {'_clientId': '...', 'userId': '...'}
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Looking for potential cache entries: {'_clientId': '...', 'userId': '...'}
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Found 18 potential entries.
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Resource specific token found.
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Returning token from cache lookup, AccessTokenId: b'z5oG6iCqtN0+XC30N6WdoniYQUwCQk5xnmPsPS+Ww+s=', RefreshTokenId: b'W8KXxe92h2tyD9U/uYeDtsaTHaziyquAseOI5IqWniI='
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Cached token is expired at 2020-06-28 13:58:07.697863. Refreshing
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - TokenRequest:called to refresh a token from the cache
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - TokenRequest:Getting a new token from a refresh token
urllib3.connectionpool : Starting new HTTPS connection (1): login.microsoftonline.com:443
urllib3.connectionpool : https://login.microsoftonline.com:443 "POST /72f988bf-86f1-41af-91ab-2d7cd011db47/oauth2/token HTTP/1.1" 200 12314
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - OAuth2Client:Get Token Server returned this correlation_id: e7a19dcb-fcd1-41de-a22d-a4d495cb70f8
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Created new cache entry from refresh response.
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Removing entry.
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Adding entry AccessTokenId: b'QEhwULlQyTEIRokcqAc81r8iJHgLlzWDZxdKaKhNKSQ=', RefreshTokenId: b'Kp5AcXwMW1CNANQTbcAPMP3hyhRVxx5TL4ANw6Dd3OY='
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Updating 17 cached refresh tokens
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Remove many: 17
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Add many: 17
adal-python : e7a19dcb-fcd1-41de-a22d-a4d495cb70f8 - CacheDriver:Returning token refreshed after expiry.
Request URL: 'https://eastus.dp.kubernetesconfiguration.azure.com/azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview'
Request method: 'POST'
Request headers:
'User-Agent': 'AZURECLI/2.8.0 (DEB)'
'Accept-Encoding': 'gzip, deflate'
'Accept': '/'
'Connection': 'keep-alive'
'x-ms-client-request-id': '6ded1983-5178-41a2-82bd-1c3ad84e288e'
'CommandName': 'rest'
'ParameterSetName': '-m -u --resource --debug'
'Authorization': 'Bearer eyJ0eXAiOiJKV...'
'Content-Length': '0'
Request body:
None
urllib3.connectionpool : Starting new HTTPS connection (1): eastus.dp.kubernetesconfiguration.azure.com:443
urllib3.connectionpool : https://eastus.dp.kubernetesconfiguration.azure.com:443 "POST /azure-arc-k8sagents/GetLatestHelmPackagePath?api-version=2019-11-01-preview HTTP/1.1" 400 239
Response status: 400
Response headers:
'Server': 'openresty/1.15.8.2'
'Date': 'Mon, 29 Jun 2020 07:56:29 GMT'
'Content-Type': 'text/html'
'Content-Length': '239'
'Connection': 'close'
'x-content-type-options': 'nosniff'
Response content:

<title>400 Request Header Or Cookie Too Large</title>

400 Bad Request

Request Header Or Cookie Too Large

---

openresty/1.15.8.2

cli.azure.cli.core.util : Bad Request(

<title>400 Request Header Or Cookie Too Large</title>

400 Bad Request

Request Header Or Cookie Too Large

---

openresty/1.15.8.2 ) Bad Request( <title>400 Request Header Or Cookie Too Large</title>

400 Bad Request

Request Header Or Cookie Too Large

---

openresty/1.15.8.2 ) Event: Cli.PostExecute [] az_command_data_logger : exit code: 1 telemetry.save : Save telemetry record of length 5468 in cache telemetry.check : Returns Positive. telemetry.main : Begin creating telemetry upload process. telemetry.process : Creating upload process: "/opt/az/bin/python3 /opt/az/lib/python3.6/site-packages/azure/cli/telemetry/__init__.py /home/myadmin/.azure" telemetry.process : Return from creating process telemetry.main : Finish creating telemetry upload process. Command ran in 5.773 seconds (init: 0.590, invoke: 5.183)

[yungezz]

interesting it loading all cli command modules

[jiasli]

@yungezz, this is exactly why we need command index (Azure/azure-cli#13294). 😉

[akashkeshari]

@chhtw

This issue has been fixed. Please update the connectedk8s extension to version 0.2.4 and try again:
Update using this cmd: 'az extension update -n connectedk8s'

Let us know if you still face this issue

[RakeshMohanMSFT]

Duplicate of #2077