Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{CI} Skip some tests #30164

Merged
merged 8 commits into from
Oct 25, 2024
Merged

{CI} Skip some tests #30164

merged 8 commits into from
Oct 25, 2024

Conversation

bebound
Copy link
Contributor

@bebound bebound commented Oct 23, 2024

The batched CI suddenly failed with Message: The subscription '00000000-0000-0000-0000-000000000000' could not be found. from Sep 25, 2024.
The Test Deb Packages and Test Rpm Package Azure Linux 3.0 fail. Not sure why distro use Python 3.9 still works.

The error is similar to #17185

For more detail, see Jiashuo's analysis at #18853 (comment) and #18611 (comment)

Temporarily skip these tests to unblock release.

Copy link

azure-client-tools-bot-prd bot commented Oct 23, 2024

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.12
️✔️3.9
️✔️ams
️✔️latest
️✔️3.12
️✔️3.9
️✔️apim
️✔️latest
️✔️3.12
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.12
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️aro
️✔️latest
️✔️3.12
️✔️3.9
️✔️backup
️✔️latest
️✔️3.12
️✔️3.9
️✔️batch
️✔️latest
️✔️3.12
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.12
️✔️3.9
️✔️billing
️✔️latest
️✔️3.12
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.12
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.12
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.12
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️compute_recommender
️✔️latest
️✔️3.12
️✔️3.9
️✔️config
️✔️latest
️✔️3.12
️✔️3.9
️✔️configure
️✔️latest
️✔️3.12
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.12
️✔️3.9
️✔️container
️✔️latest
️✔️3.12
️✔️3.9
️✔️containerapp
️✔️latest
️✔️3.12
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.12
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️dla
️✔️latest
️✔️3.12
️✔️3.9
️✔️dls
️✔️latest
️✔️3.12
️✔️3.9
️✔️dms
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.12
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.12
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.12
️✔️3.9
️✔️find
️✔️latest
️✔️3.12
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.12
️✔️3.9
️✔️identity
️✔️latest
️✔️3.12
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.12
️✔️3.9
️✔️lab
️✔️latest
️✔️3.12
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.12
️✔️3.9
️✔️maps
️✔️latest
️✔️3.12
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.12
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.12
️✔️3.9
️✔️mysql
️✔️latest
️✔️3.12
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.12
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.12
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.12
️✔️3.9
️✔️profile
️✔️latest
️✔️3.12
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.12
️✔️3.9
️✔️redis
️✔️latest
️✔️3.12
️✔️3.9
️✔️relay
️✔️latest
️✔️3.12
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️role
️✔️latest
️✔️3.12
️✔️3.9
️✔️search
️✔️latest
️✔️3.12
️✔️3.9
️✔️security
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.12
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.12
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.12
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.12
️✔️3.9
️✔️sql
️✔️latest
️✔️3.12
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.12
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.12
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9
️✔️util
️✔️latest
️✔️3.12
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.12
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.12
️✔️3.9
️✔️latest
️✔️3.12
️✔️3.9

Copy link

Hi @bebound,
Since the current milestone time is less than 7 days, this pr will be reviewed in the next milestone.

Copy link

azure-client-tools-bot-prd bot commented Oct 23, 2024

️✔️AzureCLI-BreakingChangeTest
️✔️Non Breaking Changes

@yonzhan
Copy link
Collaborator

yonzhan commented Oct 23, 2024

Thank you for your contribution! We will review the pull request and get back to you soon.

@bebound bebound marked this pull request as ready for review October 24, 2024 06:52
zhoxing-ms
zhoxing-ms previously approved these changes Oct 24, 2024
@zhoxing-ms
Copy link
Contributor

ACR: @nhu1997
ACS: @FumingZhang
Iot: @digimaun
Please be aware of this PR~ For more context, please refer to the PR description

yanzhudd
yanzhudd previously approved these changes Oct 24, 2024
@bebound bebound dismissed stale reviews from yanzhudd and zhoxing-ms via 4dc4788 October 24, 2024 08:55
@@ -761,7 +761,7 @@ jobs:

- job: TestRpmPackagesAzureLinux
displayName: Test Rpm Package
timeoutInMinutes: 120
timeoutInMinutes: 180
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Test Rpm Package Azure Linux 3.0 AMD64 fails with
##[error]The job running on agent pool-ubuntu-2004 38 ran longer than the maximum time of 120 minutes. For more information, see https://go.microsoft.com/fwlink/?linkid=2077134

@jiasli
Copy link
Member

jiasli commented Oct 24, 2024

Are those tests not from acr having the same The subscription '00000000-0000-0000-0000-000000000000' could not be found error? What is their error messages. If they are not the same, we should create issues to track them.

@@ -539,6 +539,7 @@ def test_aks_api_server_authorized_ip_ranges(self, resource_group, resource_grou
# delete
self.cmd('aks delete -g {resource_group} -n {name} --yes --no-wait', checks=[self.is_empty()])

@live_only()
Copy link
Member

@FumingZhang FumingZhang Oct 25, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's fine to disable the test, but I would like to know how it failed. Are there any examples of failed CI?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

2024-10-17T03:54:44.2162672Z _ AzureKubernetesServiceScenarioTest.test_aks_create_and_update_with_http_proxy_config _
2024-10-17T03:54:44.2163274Z [gw7] linux -- Python 3.11.8 /opt/az/bin/python3
2024-10-17T03:54:44.2163629Z self = <azure.cli.testsdk.base.ExecutionResult object at 0xffffadd9fbd0>
2024-10-17T03:54:44.2164298Z cli_ctx = <azure.cli.core.mock.DummyCli object at 0xffffb4fc6050>
2024-10-17T03:54:44.2165070Z command = 'aks create --resource-group=clitest000001 --name=cliakstest000002 --http-proxy-config=/opt/az/lib/python3.11/site-pac...0/resourceGroups/clitest000001/providers/Microsoft.Network/virtualNetworks/cliakstest000002/subnets/aks-subnet -o json'
2024-10-17T03:54:44.2165605Z expect_failure = False
2024-10-17T03:54:44.2165721Z 
2024-10-17T03:54:44.2166018Z     def _in_process_execute(self, cli_ctx, command, expect_failure=False):
2024-10-17T03:54:44.2166340Z         from io import StringIO
2024-10-17T03:54:44.2166654Z         from vcr.errors import CannotOverwriteExistingCassetteException
2024-10-17T03:54:44.2166935Z     
2024-10-17T03:54:44.2167255Z         if command.startswith('az '):
2024-10-17T03:54:44.2167540Z             command = command[3:]
2024-10-17T03:54:44.2167778Z     
2024-10-17T03:54:44.2168022Z         stdout_buf = StringIO()
2024-10-17T03:54:44.2168297Z         logging_buf = StringIO()
2024-10-17T03:54:44.2168600Z         try:
2024-10-17T03:54:44.2168913Z             # issue: stderr cannot be redirect in this form, as a result some failure information
2024-10-17T03:54:44.2169260Z             # is lost when command fails.
2024-10-17T03:54:44.2169600Z >           self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
2024-10-17T03:54:44.2169779Z 
2024-10-17T03:54:44.2170172Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:302: 
2024-10-17T03:54:44.2170528Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-17T03:54:44.2170964Z /opt/az/lib/python3.11/site-packages/knack/cli.py:245: in invoke
2024-10-17T03:54:44.2171300Z     exit_code = self.exception_handler(ex)
2024-10-17T03:54:44.2171767Z /opt/az/lib/python3.11/site-packages/azure/cli/core/__init__.py:129: in exception_handler
2024-10-17T03:54:44.2172125Z     return handle_exception(ex)
2024-10-17T03:54:44.2172431Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-17T03:54:44.2172579Z 
2024-10-17T03:54:44.2173245Z ex = ResourceNotFoundError("(SubscriptionNotFound) The subscription '00000000-0000-0000-0000-000000000000' could not be found.\nCode: SubscriptionNotFound\nMessage: The subscription '00000000-0000-0000-0000-000000000000' could not be found.")
2024-10-17T03:54:44.2173736Z args = (), kwargs = {}
2024-10-17T03:54:44.2173849Z 
2024-10-17T03:54:44.2174264Z     def _handle_main_exception(ex, *args, **kwargs):  # pylint: disable=unused-argument
2024-10-17T03:54:44.2174642Z         if isinstance(ex, CannotOverwriteExistingCassetteException):
2024-10-17T03:54:44.2175021Z             # This exception usually caused by a no match HTTP request. This is a product error
2024-10-17T03:54:44.2175371Z             # that is caused by change of SDK invocation.
2024-10-17T03:54:44.2175646Z             raise ex
2024-10-17T03:54:44.2175879Z     
2024-10-17T03:54:44.2176132Z >       raise CliExecutionError(ex)
2024-10-17T03:54:44.2176532Z E       azure.cli.testsdk.exceptions.CliExecutionError: The CLI throws exception ResourceNotFoundError during execution and fails the command.
2024-10-17T03:54:44.2176764Z 
2024-10-17T03:54:44.2177181Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/patches.py:35: CliExecutionError
2024-10-17T03:54:44.2177568Z 
2024-10-17T03:54:44.2177869Z During handling of the above exception, another exception occurred:
2024-10-17T03:54:44.2178033Z 
2024-10-17T03:54:44.2178433Z self = <command_modules.acs.tests.latest.test_aks_commands.AzureKubernetesServiceScenarioTest testMethod=test_aks_create_and_update_with_http_proxy_config>
2024-10-17T03:54:44.2178984Z resource_group = 'clitest000001', resource_group_location = 'westus2'
2024-10-17T03:54:44.2179271Z 
2024-10-17T03:54:44.2179522Z     @AllowLargeResponse()
2024-10-17T03:54:44.2180000Z     @AKSCustomResourceGroupPreparer(random_name_length=17, name_prefix='clitest', location='westus2')
2024-10-17T03:54:44.2180448Z     def test_aks_create_and_update_with_http_proxy_config(self, resource_group, resource_group_location):
2024-10-17T03:54:44.2181041Z         aks_name = self.create_random_name('cliakstest', 16)
2024-10-17T03:54:44.2181355Z         self.kwargs.update({
2024-10-17T03:54:44.2181716Z             'resource_group': resource_group,
2024-10-17T03:54:44.2182060Z             'name': aks_name,
2024-10-17T03:54:44.2182652Z             'http_proxy_path': get_test_data_file_path('httpproxyconfig.json').replace('\\', '\\\\'),
2024-10-17T03:54:44.2183168Z             'custom_data_path': get_test_data_file_path('setup_proxy.sh').replace('\\', '\\\\'),
2024-10-17T03:54:44.2183781Z             'ssh_key_value': self.generate_ssh_keys()
2024-10-17T03:54:44.2184068Z         })
2024-10-17T03:54:44.2184283Z     
2024-10-17T03:54:44.2184618Z         create_vnet_cmd = 'network vnet create \
2024-10-17T03:54:44.2184995Z             --resource-group={resource_group} \
2024-10-17T03:54:44.2185322Z             --name={name} \
2024-10-17T03:54:44.2185674Z             --address-prefixes 10.42.0.0/16 \
2024-10-17T03:54:44.2186029Z             --subnet-name aks-subnet \
2024-10-17T03:54:44.2186377Z             --subnet-prefix 10.42.1.0/24'
2024-10-17T03:54:44.2186625Z     
2024-10-17T03:54:44.2186985Z         create_subnet_cmd = 'network vnet subnet create \
2024-10-17T03:54:44.2187373Z             --resource-group={resource_group} \
2024-10-17T03:54:44.2187713Z             --vnet-name={name} \
2024-10-17T03:54:44.2188052Z             --name proxy-subnet \
2024-10-17T03:54:44.2188406Z             --address-prefix 10.42.3.0/24'
2024-10-17T03:54:44.2188859Z             # --default-outbound false  # disable outbound connection would fail cluster creation
2024-10-17T03:54:44.2189165Z     
2024-10-17T03:54:44.2189503Z         show_subnet_cmd = 'network vnet subnet show \
2024-10-17T03:54:44.2189871Z             --resource-group={resource_group} \
2024-10-17T03:54:44.2190214Z             --vnet-name={name} \
2024-10-17T03:54:44.2190545Z             --name aks-subnet'
2024-10-17T03:54:44.2190784Z     
2024-10-17T03:54:44.2191083Z         # name below MUST match the name used in testcerts for httpproxyconfig.json.
2024-10-17T03:54:44.2191450Z         # otherwise the VM will not present a cert with correct hostname
2024-10-17T03:54:44.2191794Z         # else, change the cert to have the correct hostname (harder)
2024-10-17T03:54:44.2192181Z         create_vm_cmd = 'vm create \
2024-10-17T03:54:44.2192541Z             --resource-group={resource_group} \
2024-10-17T03:54:44.2192893Z             --name=cli-proxy-vm \
2024-10-17T03:54:44.2193345Z             --image Canonical:0001-com-ubuntu-server-focal:20_04-lts:latest \
2024-10-17T03:54:44.2193783Z             --ssh-key-values @{ssh_key_value} \
2024-10-17T03:54:44.2194137Z             --public-ip-address "" \
2024-10-17T03:54:44.2194496Z             --custom-data {custom_data_path} \
2024-10-17T03:54:44.2194845Z             --vnet-name {name} \
2024-10-17T03:54:44.2195172Z             --subnet proxy-subnet'
2024-10-17T03:54:44.2195416Z     
2024-10-17T03:54:44.2195685Z         self.cmd(create_vnet_cmd, checks=[
2024-10-17T03:54:44.2196077Z             self.check('newVNet.provisioningState', 'Succeeded')
2024-10-17T03:54:44.2196353Z         ])
2024-10-17T03:54:44.2196575Z     
2024-10-17T03:54:44.2196832Z         self.cmd(create_subnet_cmd, checks=[
2024-10-17T03:54:44.2197416Z             self.check('provisioningState', 'Succeeded')
2024-10-17T03:54:44.2197700Z         ])
2024-10-17T03:54:44.2197917Z     
2024-10-17T03:54:44.2198199Z         subnet_output = self.cmd(show_subnet_cmd).get_output_in_json()
2024-10-17T03:54:44.2198515Z         subnet_id = subnet_output["id"]
2024-10-17T03:54:44.2198791Z         assert subnet_id is not None
2024-10-17T03:54:44.2199031Z     
2024-10-17T03:54:44.2199271Z         self.cmd(create_vm_cmd)
2024-10-17T03:54:44.2199516Z     
2024-10-17T03:54:44.2199752Z         self.kwargs.update({
2024-10-17T03:54:44.2200088Z             'vnet_subnet_id': subnet_id,
2024-10-17T03:54:44.2200349Z         })
2024-10-17T03:54:44.2200690Z     
2024-10-17T03:54:44.2201001Z         # use custom feature so it does not require subscription to regiter the feature
2024-10-17T03:54:44.2201544Z         create_cmd = 'aks create --resource-group={resource_group} --name={name} --http-proxy-config={http_proxy_path} ' \
2024-10-17T03:54:44.2202129Z                      '--ssh-key-value={ssh_key_value} --enable-managed-identity --yes --vnet-subnet-id {vnet_subnet_id} -o json'
2024-10-17T03:54:44.2202461Z     
2024-10-17T03:54:44.2202720Z >       self.cmd(create_cmd, checks=[
2024-10-17T03:54:44.2203089Z             self.check('httpProxyConfig.httpProxy',
2024-10-17T03:54:44.2205586Z                        'http://cli-proxy-vm:3128/'),
2024-10-17T03:54:44.2206133Z             self.check('httpProxyConfig.httpsProxy',
2024-10-17T03:54:44.2206573Z                        'https://cli-proxy-vm:3129/'),
2024-10-17T03:54:44.2211201Z             self.check('httpProxyConfig.trustedCa', 'LS0tLS1CRUdJTiBDRVJUSUZJQ0FURS0tLS0tCk1JSUZHekNDQXdPZ0F3SUJBZ0lVT1FvajhDTFpkc2Vscjk3cnZJd3g1T0xEc3V3d0RRWUpLb1pJaHZjTkFRRUwKQlFBd0Z6RVZNQk1HQTFVRUF3d01ZMnhwTFhCeWIzaDVMWFp0TUI0WERUSXlNRE13T0RFMk5EUTBOMW9YRFRNeQpNRE13TlRFMk5EUTBOMW93RnpFVk1CTUdBMVVFQXd3TVkyeHBMWEJ5YjNoNUxYWnRNSUlDSWpBTkJna3Foa2lHCjl3MEJBUUVGQUFPQ0FnOEFNSUlDQ2dLQ0FnRUEvTVB0VjVCVFB0NmNxaTRSZE1sbXIzeUlzYTJ1anpjaHh2NGgKanNDMUR0blJnb3M1UzQxUEgwcmkrM3RUU1ZYMzJ5cndzWStyRDFZUnVwbTZsbUU3R2hVNUkwR2k5b3prU0YwWgpLS2FKaTJveXBVL0ZCK1FQcXpvQ1JzTUV3R0NibUtGVmw4VnVoeW5kWEs0YjRrYmxyOWJsL2V1d2Q3TThTYnZ6CldVam5lRHJRc2lJc3J6UFQ0S0FaTHFjdHpEZTRsbFBUN1lLYTMzaGlFUE9mdldpWitkcWthUUE5UDY0eFhTeW4KZkhYOHVWQUozdUJWSmVHeEQwcGtOSjdqT3J5YVV1SEh1Y1U4UzltSWpuS2pBQjVhUGpMSDV4QXM2bG1iMzEyMgp5KzF0bkVBbVhNNTBEK1VvRWpmUzZIT2I1cmRpcVhHdmMxS2JvS2p6a1BDUnh4MmE3MmN2ZWdVajZtZ0FKTHpnClRoRTFsbGNtVTRpemd4b0lNa1ZwR1RWT0xMbjFWRkt1TmhNWkN2RnZLZ25Lb0F2M0cwRlVuZldFYVJSalNObUQKTFlhTURUNUg5WnQycERJVWpVR1N0Q2w3Z1J6TUVuWXdKTzN5aURwZzQzbzVkUnlzVXlMOUpmRS9OaDdUZzYxOApuOGNKL1c3K1FZYllsanVyYXA4cjdRRlNyb2wzVkNoRkIrT29yNW5pK3ZvaFNBd0pmMFVsTXBHM3hXbXkxVUk0ClRGS2ZGR1JSVHpyUCs3Yk53WDVoSXZJeTVWdGd5YU9xSndUeGhpL0pkeHRPcjJ0QTVyQ1c3K0N0Z1N2emtxTkUKWHlyN3ZrWWdwNlk1TFpneTR0VWpLMEswT1VnVmRqQk9oRHBFenkvRkY4dzFGRVZnSjBxWS9yV2NMa0JIRFQ4Ugp2SmtoaW84Q0F3RUFBYU5mTUYwd0Z3WURWUjBSQkJBd0RvSU1ZMnhwTFhCeWIzaDVMWFp0TUJJR0ExVWRFd0VCCi93UUlNQVlCQWY4Q0FRQXdEd1lEVlIwUEFRSC9CQVVEQXdmbmdEQWRCZ05WSFNVRUZqQVVCZ2dyQmdFRkJRY0QKQWdZSUt3WUJCUVVIQXdFd0RRWUpLb1pJaHZjTkFRRUxCUUFEZ2dJQkFBb21qQ3lYdmFRT3hnWUs1MHNYTEIyKwp3QWZkc3g1bm5HZGd5Zmc0dXJXMlZtMTVEaEd2STdDL250cTBkWXkyNE4vVWJHN1VEWHZseUxJSkZxMVhQN25mCnBaRzBWQ2paNjlibXhLbTNaOG0wL0F3TXZpOGU5ZWR5OHY5a05CQ3dMR2tIYkE4WW85Q0lpUWdlbGZwcDF2VWgKYm5OQmhhRCtpdTZDZmlDTHdnSmIvaXc3ZW8vQ3lvWnF4K3RqWGFPMnpYdm00cC8rUUlmQU9ndEdRTEZVOGNmWgovZ1VyVHE1Z0ZxMCtQOUd5V3NBVEpGNnE3TDZXWlpqME91VHNlN2Y0Q1NpajZNbk9NTXhBK0pvYWhKejdsc1NpClRKSEl3RXA1ci9SeWhweWVwUXhGWWNVSDVKSmY5cmFoWExXWmkrOVRqeFNNMll5aHhmUlBzaVVFdUdEb2s3OFEKbS9RUGlDaTlKSmIxb2NtVGpBVjh4RFNob2NpdlhPRnlobjZMbjc3dkxqWStBYXZ0V0RoUXRocHVQeHNMdFZ6bQplMFNIMTFkRUxSdGI3NG1xWE9yTzdmdS8rSUJzM0pxTEUvVSt4dXhRdHZHOHZHMXlES0hIU1pxUzJoL1dzNGw0Ck5pQXNoSGdlaFFEUEJjWTl3WVl6ZkJnWnBPVU16ZERmNTB4K0ZTbFk0M1dPSkp6U3VRaDR5WjArM2t5Z3VDRjgKcm5NTFNjZXlTNGNpNExtSi9LQ1N1R2RmNlhWWXo4QkU5Z2pqanBDUDZxeTBVbFJlZldzL2lnL3djSysyYkYxVApuL1l2KzZnWGVDVEhKNzVxRElQbHA3RFJVVWswZmJNajRiSWthb2dXV2s0emYydThteFpMYTBsZVBLTktaTi9tCkdDdkZ3cjNlaSt1LzhjenA1RjdUCi0tLS0tRU5EIENFUlRJRklDQVRFLS0tLS0K')
2024-10-17T03:54:44.2214308Z         ])
2024-10-17T03:54:44.2214410Z 
2024-10-17T03:54:44.2214880Z /opt/az/lib/python3.11/site-packages/azure/cli/command_modules/acs/tests/latest/test_aks_commands.py:608: 
2024-10-17T03:54:44.2215279Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-17T03:54:44.2215730Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:176: in cmd
2024-10-17T03:54:44.2216139Z     return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
2024-10-17T03:54:44.2216647Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:251: in __init__
2024-10-17T03:54:44.2217042Z     self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)
2024-10-17T03:54:44.2217657Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:314: in _in_process_execute
2024-10-17T03:54:44.2218002Z     raise ex.exception
2024-10-17T03:54:44.2218395Z /opt/az/lib/python3.11/site-packages/knack/cli.py:233: in invoke
2024-10-17T03:54:44.2218738Z     cmd_result = self.invocation.execute(args)
2024-10-17T03:54:44.2219201Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:666: in execute
2024-10-17T03:54:44.2219520Z     raise ex
2024-10-17T03:54:44.2219966Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:733: in _run_jobs_serially
2024-10-17T03:54:44.2220355Z     results.append(self._run_job(expanded_arg, cmd_copy))
2024-10-17T03:54:44.2220824Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:703: in _run_job
2024-10-17T03:54:44.2221163Z     result = cmd_copy(params)
2024-10-17T03:54:44.2221612Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:336: in __call__
2024-10-17T03:54:44.2221967Z     return self.handler(*args, **kwargs)
2024-10-17T03:54:44.2222439Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py:121: in handler
2024-10-17T03:54:44.2222800Z     return op(**command_args)
2024-10-17T03:54:44.2223258Z /opt/az/lib/python3.11/site-packages/azure/cli/command_modules/acs/custom.py:679: in aks_create
2024-10-17T03:54:44.2223802Z     return aks_create_decorator.create_mc(mc)
2024-10-17T03:54:44.2224151Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-17T03:54:44.2224290Z 
2024-10-17T03:54:44.2224643Z self = <azure.cli.command_modules.acs.managed_cluster_decorator.AKSManagedClusterCreateDecorator object at 0xffffad12f190>
2024-10-17T03:54:44.2225121Z mc = <azure.mgmt.containerservice.v2024_08_01.models._models_py3.ManagedCluster object at 0xffffad2d94d0>
2024-10-17T03:54:44.2225315Z 
2024-10-17T03:54:44.2225694Z     def create_mc(self, mc: ManagedCluster) -> ManagedCluster:
2024-10-17T03:54:44.2226041Z         """Send request to create a real managed cluster.
2024-10-17T03:54:44.2226308Z     
2024-10-17T03:54:44.2226575Z         :return: the ManagedCluster object
2024-10-17T03:54:44.2226831Z         """
2024-10-17T03:54:44.2227085Z         self._ensure_mc(mc)
2024-10-17T03:54:44.2227330Z     
2024-10-17T03:54:44.2227611Z         # Due to SPN replication latency, we do a few retries here
2024-10-17T03:54:44.2227901Z         max_retry = 30
2024-10-17T03:54:44.2228155Z         error_msg = ""
2024-10-17T03:54:44.2228419Z         for _ in range(0, max_retry):
2024-10-17T03:54:44.2228673Z             try:
2024-10-17T03:54:44.2228931Z                 cluster = self.put_mc(mc)
2024-10-17T03:54:44.2229206Z                 return cluster
2024-10-17T03:54:44.2229518Z             # CloudError was raised before, but since the adoption of track 2 SDK,
2024-10-17T03:54:44.2229857Z             # HttpResponseError would be raised instead
2024-10-17T03:54:44.2230163Z             except HttpResponseError as ex:
2024-10-17T03:54:44.2230442Z                 error_msg = str(ex)
2024-10-17T03:54:44.2230744Z                 if "not found in Active Directory tenant" in ex.message:
2024-10-17T03:54:44.2231046Z                     time.sleep(3)
2024-10-17T03:54:44.2231480Z                 else:
2024-10-17T03:54:44.2231759Z >                   raise map_azure_error_to_cli_error(ex)
2024-10-17T03:54:44.2232367Z E                   azure.cli.core.azclierror.ResourceNotFoundError: (SubscriptionNotFound) The subscription '00000000-0000-0000-0000-000000000000' could not be found.
2024-10-17T03:54:44.2232780Z E                   Code: SubscriptionNotFound
2024-10-17T03:54:44.2233241Z E                   Message: The subscription '00000000-0000-0000-0000-000000000000' could not be found.

https://dev.azure.com/azclitools/public/_build/results?buildId=197899&view=logs&jobId=53b5e140-cff2-558f-049b-6067e7c3cd03&j=038813f6-b85b-5e37-acf4-30b28352865f&t=48a22e66-ceab-5c7f-d00b-cb2046709dfd

@bebound
Copy link
Contributor Author

bebound commented Oct 25, 2024

Are those tests not from acr having the same The subscription '00000000-0000-0000-0000-000000000000' could not be found error? What is their error messages. If they are not the same, we should create issues to track them.

All of the test has same error, except test_hub_wait, which raises Resource group 'clitest.rg000001' could not be found..

2024-10-23T04:43:47.1447896Z =================================== FAILURES ===================================
2024-10-23T04:43:47.1448284Z ___________________________ IoTHubTest.test_hub_wait ___________________________
2024-10-23T04:43:47.1448837Z self = <azure.cli.testsdk.base.ExecutionResult object at 0x7f021aa96010>
2024-10-23T04:43:47.1449225Z cli_ctx = <azure.cli.core.mock.DummyCli object at 0x7f021a527490>
2024-10-23T04:43:47.1449897Z command = 'iot hub wait -n iot-hub-for-test-11000002 -g clitest.rg000001 --created'
2024-10-23T04:43:47.1450214Z expect_failure = False
2024-10-23T04:43:47.1450323Z 
2024-10-23T04:43:47.1450837Z     def _in_process_execute(self, cli_ctx, command, expect_failure=False):
2024-10-23T04:43:47.1451142Z         from io import StringIO
2024-10-23T04:43:47.1451449Z         from vcr.errors import CannotOverwriteExistingCassetteException
2024-10-23T04:43:47.1451714Z     
2024-10-23T04:43:47.1452034Z         if command.startswith('az '):
2024-10-23T04:43:47.1452455Z             command = command[3:]
2024-10-23T04:43:47.1452672Z     
2024-10-23T04:43:47.1452894Z         stdout_buf = StringIO()
2024-10-23T04:43:47.1453156Z         logging_buf = StringIO()
2024-10-23T04:43:47.1453389Z         try:
2024-10-23T04:43:47.1453694Z             # issue: stderr cannot be redirect in this form, as a result some failure information
2024-10-23T04:43:47.1454030Z             # is lost when command fails.
2024-10-23T04:43:47.1454355Z >           self.exit_code = cli_ctx.invoke(shlex.split(command), out_file=stdout_buf) or 0
2024-10-23T04:43:47.1454525Z 
2024-10-23T04:43:47.1455386Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:302: 
2024-10-23T04:43:47.1455733Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-23T04:43:47.1456153Z /opt/az/lib/python3.11/site-packages/knack/cli.py:245: in invoke
2024-10-23T04:43:47.1456469Z     exit_code = self.exception_handler(ex)
2024-10-23T04:43:47.1456944Z /opt/az/lib/python3.11/site-packages/azure/cli/core/__init__.py:129: in exception_handler
2024-10-23T04:43:47.1457286Z     return handle_exception(ex)
2024-10-23T04:43:47.1457588Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-23T04:43:47.1457718Z 
2024-10-23T04:43:47.1458103Z ex = CLIError("Resource group 'clitest.rg000001' could not be found.")
2024-10-23T04:43:47.1458379Z args = (), kwargs = {}
2024-10-23T04:43:47.1458474Z 
2024-10-23T04:43:47.1458889Z     def _handle_main_exception(ex, *args, **kwargs):  # pylint: disable=unused-argument
2024-10-23T04:43:47.1459249Z         if isinstance(ex, CannotOverwriteExistingCassetteException):
2024-10-23T04:43:47.1459615Z             # This exception usually caused by a no match HTTP request. This is a product error
2024-10-23T04:43:47.1459963Z             # that is caused by change of SDK invocation.
2024-10-23T04:43:47.1460220Z             raise ex
2024-10-23T04:43:47.1460428Z     
2024-10-23T04:43:47.1460663Z >       raise CliExecutionError(ex)
2024-10-23T04:43:47.1461041Z E       azure.cli.testsdk.exceptions.CliExecutionError: The CLI throws exception CLIError during execution and fails the command.
2024-10-23T04:43:47.1461254Z 
2024-10-23T04:43:47.1461688Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/patches.py:35: CliExecutionError
2024-10-23T04:43:47.1461867Z 
2024-10-23T04:43:47.1462141Z During handling of the above exception, another exception occurred:
2024-10-23T04:43:47.1462281Z 
2024-10-23T04:43:47.1462599Z self = <command_modules.iot.tests.latest.test_iot_commands.IoTHubTest testMethod=test_hub_wait>
2024-10-23T04:43:47.1463096Z resource_group = 'clitest.rg000001', resource_group_location = 'westus2'
2024-10-23T04:43:47.1463255Z 
2024-10-23T04:43:47.1463484Z     @AllowLargeResponse()
2024-10-23T04:43:47.1463858Z     @ResourceGroupPreparer(location='westus2')
2024-10-23T04:43:47.1464190Z     def test_hub_wait(self, resource_group, resource_group_location):
2024-10-23T04:43:47.1464664Z         hub = self.create_random_name(prefix='iot-hub-for-test-11', length=32)
2024-10-23T04:43:47.1464982Z         rg = resource_group
2024-10-23T04:43:47.1465200Z     
2024-10-23T04:43:47.1465432Z         # Create hub with no wait
2024-10-23T04:43:47.1465841Z         self.cmd('iot hub create -n {0} -g {1} --no-wait'.format(hub, rg))
2024-10-23T04:43:47.1466109Z     
2024-10-23T04:43:47.1466337Z         # Poll till hub is active
2024-10-23T04:43:47.1466734Z >       self.cmd('iot hub wait -n {0} -g {1} --created'.format(hub, rg))
2024-10-23T04:43:47.1466877Z 
2024-10-23T04:43:47.1467362Z /opt/az/lib/python3.11/site-packages/azure/cli/command_modules/iot/tests/latest/test_iot_commands.py:853: 
2024-10-23T04:43:47.1467870Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-23T04:43:47.1468328Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:176: in cmd
2024-10-23T04:43:47.1468724Z     return execute(self.cli_ctx, command, expect_failure=expect_failure).assert_with_checks(checks)
2024-10-23T04:43:47.1469335Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:251: in __init__
2024-10-23T04:43:47.1469708Z     self._in_process_execute(cli_ctx, command, expect_failure=expect_failure)
2024-10-23T04:43:47.1470230Z /opt/az/lib/python3.11/site-packages/azure/cli/testsdk/base.py:314: in _in_process_execute
2024-10-23T04:43:47.1470548Z     raise ex.exception
2024-10-23T04:43:47.1470955Z /opt/az/lib/python3.11/site-packages/knack/cli.py:233: in invoke
2024-10-23T04:43:47.1471273Z     cmd_result = self.invocation.execute(args)
2024-10-23T04:43:47.1471748Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:666: in execute
2024-10-23T04:43:47.1472069Z     raise ex
2024-10-23T04:43:47.1472548Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:733: in _run_jobs_serially
2024-10-23T04:43:47.1472916Z     results.append(self._run_job(expanded_arg, cmd_copy))
2024-10-23T04:43:47.1473412Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:703: in _run_job
2024-10-23T04:43:47.1473746Z     result = cmd_copy(params)
2024-10-23T04:43:47.1474192Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/__init__.py:336: in __call__
2024-10-23T04:43:47.1474549Z     return self.handler(*args, **kwargs)
2024-10-23T04:43:47.1475027Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py:402: in handler
2024-10-23T04:43:47.1475404Z     return self.wait(command_args, cli_ctx=self.cli_ctx, getter=getter)
2024-10-23T04:43:47.1475916Z /opt/az/lib/python3.11/site-packages/azure/cli/core/commands/command_operation.py:431: in wait
2024-10-23T04:43:47.1476265Z     instance = getter(**command_args)
2024-10-23T04:43:47.1476568Z _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ 
2024-10-23T04:43:47.1476702Z 
2024-10-23T04:43:47.1476986Z cmd = <azure.cli.core.commands.AzCliCommand object at 0x7f0219c97ed0>
2024-10-23T04:43:47.1477348Z client = <azure.mgmt.iothub._iot_hub_client.IotHubClient object at 0x7f0219cf7c50>
2024-10-23T04:43:47.1477852Z hub_name = 'iot-hub-for-test-11000002', resource_group_name = 'clitest.rg000001'
2024-10-23T04:43:47.1478018Z 
2024-10-23T04:43:47.1478293Z     def iot_hub_get(cmd, client, hub_name, resource_group_name=None):
2024-10-23T04:43:47.1478584Z         cli_ctx = cmd.cli_ctx
2024-10-23T04:43:47.1478850Z         if resource_group_name is None:
2024-10-23T04:43:47.1479137Z             return _get_iot_hub_by_name(client, hub_name)
2024-10-23T04:43:47.1479472Z         if not _ensure_resource_group_existence(cli_ctx, resource_group_name):
2024-10-23T04:43:47.1479982Z >           raise CLIError("Resource group '{0}' could not be found.".format(resource_group_name))
2024-10-23T04:43:47.1480497Z E           knack.util.CLIError: Resource group 'clitest.rg000001' could not be found.
2024-10-23T04:43:47.1480659Z 
2024-10-23T04:43:47.1481077Z /opt/az/lib/python3.11/site-packages/azure/cli/command_modules/iot/custom.py:603: CLIError
2024-10-23T04:43:47.1481577Z -------------- generated xml file: /azure_cli_test_result/iot.xml --------------
2024-10-23T04:43:47.1481945Z =========================== short test summary info ============================
2024-10-23T04:43:47.1482333Z FAILED tests/latest/test_iot_commands.py::IoTHubTest::test_hub_wait
2024-10-23T04:43:47.1482708Z =================== 1 failed, 8 passed, 2 skipped in 42.37s ====================

@@ -71,12 +71,14 @@ def _get_resource_group(self, **kwargs):

class ServerMgmtScenarioTest(ScenarioTest):

@live_only()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@nasc17 Please be aware of this change. For more context, please refer to the PR description

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot CI CI
Projects
None yet
Development

Successfully merging this pull request may close these issues.

10 participants