Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

{Core} Improve autocomplete performance by preventing loading all modules #24765

Merged
merged 14 commits into from
May 10, 2023

Conversation

bebound
Copy link
Contributor

@bebound bebound commented Nov 30, 2022

Description

Close #24762, related issue #9157, related pr #13294
If the command is not complete, for example: az account sho, press TAB, cli loads all modules, which takes 1s. It gets worse if user installs many extensions. This make autocomplete slow.
This PR prevents module loading in autocomplete mode.

If command is az acc:

  1. module begin with acc will be returned in CommandIndex.get() .
  2. raw_cmd is 'acc'. Trim last word, it becomes '', return self.command_table.
cli.azure.cli.core: In autocomplete mode, load command begin with: 'acc'
cli.azure.cli.core: Modules found from index for 'acc': ['azure.cli.command_modules.profile', 'azure.cli.command_modules.resource']

cli.azure.cli.core: Could not find a match in the command or command group table for 'acc'
cli.azure.cli.core: In autocomplete mode, try to match trimmed raw cmd: ''
cli.azure.cli.core: Trimmed raw cmd is empty, return command table.

If command is az account sh:

  1. account module is returned in CommandIndex.get() .
  2. raw_cmd is account sh. Trim last word, it becomes account and is in self.command_group_table, return self.command_table.
cli.azure.cli.core: Modules found from index for 'account': ['azure.cli.command_modules.profile', 'azure.cli.command_modules.resource']

cli.azure.cli.core: Could not find a match in the command or command group table for 'account sh'
cli.azure.cli.core: In autocomplete mode, try to match trimmed raw cmd: 'account'
cli.azure.cli.core: Found a match in the command group table for trimmed raw cmd: 'account'.

Testing Guide
time IFS=$'\013' _ARC_DEBUG=1 COMP_LINE='az account s' COMP_POINT=12 COMP_TYPE="" _ARGCOMPLETE=1 _ARGCOMPLETE_SUPPRESS_SPACE=0 az 8>&1 1>&1 2>&1 9>&1
before: 2.364s
after: 0.269s


This checklist is used to make sure that common guidelines for a pull request are followed.

@ghost ghost added Auto-Assign Auto assign by bot Core CLI core infrastructure labels Nov 30, 2022
@ghost ghost requested review from jiasli and yonzhan November 30, 2022 06:25
@ghost ghost assigned jiasli Nov 30, 2022
@ghost ghost added this to the Dec 2022 (2023-01-03) milestone Nov 30, 2022
@bebound bebound changed the title {Core} Improve Autocomplete performance {Core} Improve Autocomplete performance by preventing loading all modules Nov 30, 2022
@bebound bebound changed the title {Core} Improve Autocomplete performance by preventing loading all modules {Core} Improve autocomplete performance by preventing loading all modules Nov 30, 2022
@yonzhan
Copy link
Collaborator

yonzhan commented Nov 30, 2022

autocomplete performance improvement

@bebound bebound marked this pull request as ready for review December 5, 2022 02:05
Comment on lines 571 to 579
if not index_modules_extensions and is_autocomplete():
# If user type `az acco`, command begin with `acco` will be matched.
# If the user just installed a new extension which contains a command named `acco`, he can't get the
# expected suggestion. But it's a rare scenario, and it's okay to ignore.
logger.debug("In autocomplete mode, load command begin with: '%s'", top_command)
index_modules_extensions = []
for command in index:
if command.startswith(top_command):
index_modules_extensions += index[command]
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe put these lines after L597 for simplicity?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I put it here as the following code convert index_modules_extensions to index_builtin_modules and index_extensions

@jiasli jiasli modified the milestones: Feb 2023 (2023-03-07), Backlog Mar 2, 2023
@azure-client-tools-bot-prd
Copy link

azure-client-tools-bot-prd bot commented Mar 13, 2023

️✔️AzureCLI-FullTest
️✔️acr
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️acs
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️advisor
️✔️latest
️✔️3.10
️✔️3.9
️✔️ams
️✔️latest
️✔️3.10
️✔️3.9
️✔️apim
️✔️latest
️✔️3.10
️✔️3.9
️✔️appconfig
️✔️latest
️✔️3.10
️✔️3.9
️✔️appservice
️✔️latest
️✔️3.10
️✔️3.9
️✔️aro
️✔️latest
️✔️3.10
️✔️3.9
️✔️backup
️✔️latest
️✔️3.10
️✔️3.9
️✔️batch
️✔️latest
️✔️3.10
️✔️3.9
️✔️batchai
️✔️latest
️✔️3.10
️✔️3.9
️✔️billing
️✔️latest
️✔️3.10
️✔️3.9
️✔️botservice
️✔️latest
️✔️3.10
️✔️3.9
️✔️cdn
️✔️latest
️✔️3.10
️✔️3.9
️✔️cloud
️✔️latest
️✔️3.10
️✔️3.9
️✔️cognitiveservices
️✔️latest
️✔️3.10
️✔️3.9
️✔️config
️✔️latest
️✔️3.10
️✔️3.9
️✔️configure
️✔️latest
️✔️3.10
️✔️3.9
️✔️consumption
️✔️latest
️✔️3.10
️✔️3.9
️✔️container
️✔️latest
️✔️3.10
️✔️3.9
️✔️core
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️cosmosdb
️✔️latest
️✔️3.10
️✔️3.9
️✔️databoxedge
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️dla
️✔️latest
️✔️3.10
️✔️3.9
️✔️dls
️✔️latest
️✔️3.10
️✔️3.9
️✔️dms
️✔️latest
️✔️3.10
️✔️3.9
️✔️eventgrid
️✔️latest
️✔️3.10
️✔️3.9
️✔️eventhubs
️✔️latest
️✔️3.10
️✔️3.9
️✔️feedback
️✔️latest
️✔️3.10
️✔️3.9
️✔️find
️✔️latest
️✔️3.10
️✔️3.9
️✔️hdinsight
️✔️latest
️✔️3.10
️✔️3.9
️✔️identity
️✔️latest
️✔️3.10
️✔️3.9
️✔️iot
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️keyvault
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️kusto
️✔️latest
️✔️3.10
️✔️3.9
️✔️lab
️✔️latest
️✔️3.10
️✔️3.9
️✔️managedservices
️✔️latest
️✔️3.10
️✔️3.9
️✔️maps
️✔️latest
️✔️3.10
️✔️3.9
️✔️marketplaceordering
️✔️latest
️✔️3.10
️✔️3.9
️✔️monitor
️✔️latest
️✔️3.10
️✔️3.9
️✔️netappfiles
️✔️latest
️✔️3.10
️✔️3.9
️✔️network
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️policyinsights
️✔️latest
️✔️3.10
️✔️3.9
️✔️privatedns
️✔️latest
️✔️3.10
️✔️3.9
️✔️profile
️✔️latest
️✔️3.10
️✔️3.9
️✔️rdbms
️✔️latest
️✔️3.10
️✔️3.9
️✔️redis
️✔️latest
️✔️3.10
️✔️3.9
️✔️relay
️✔️latest
️✔️3.10
️✔️3.9
️✔️resource
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️role
️✔️latest
️✔️3.10
️✔️3.9
️✔️search
️✔️latest
️✔️3.10
️✔️3.9
️✔️security
️✔️latest
️✔️3.10
️✔️3.9
️✔️servicebus
️✔️latest
️✔️3.10
️✔️3.9
️✔️serviceconnector
️✔️latest
️✔️3.10
️✔️3.9
️✔️servicefabric
️✔️latest
️✔️3.10
️✔️3.9
️✔️signalr
️✔️latest
️✔️3.10
️✔️3.9
️✔️sql
️✔️latest
️✔️3.10
️✔️3.9
️✔️sqlvm
️✔️latest
️✔️3.10
️✔️3.9
️✔️storage
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️synapse
️✔️latest
️✔️3.10
️✔️3.9
️✔️telemetry
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9
️✔️util
️✔️latest
️✔️3.10
️✔️3.9
️✔️vm
️✔️2018-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2019-03-01-hybrid
️✔️3.10
️✔️3.9
️✔️2020-09-01-hybrid
️✔️3.10
️✔️3.9
️✔️latest
️✔️3.10
️✔️3.9

if not index_modules_extensions and is_autocomplete():
# If user type `az acco`, command begin with `acco` will be matched.
# If the user just installed a new extension which contains a command named `acco`, he can't get the
# expected suggestion. But it's a rare scenario, and it's okay to ignore.
Copy link
Contributor Author

@bebound bebound Mar 13, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If the user just installed a new extension which contains a command named acco, he can't get the
expected suggestion. But it's a rare scenario, and it's okay to ignore.

This does not happen as CommandIndex().invalidate() is called after extension installation/remove/update.

# If the command is not complete in autocomplete mode, we should match shorter command.
# For example, `account sho` should match `account`.
logger.debug("Could not find a match in the command or command group table for '%s'", raw_cmd)
raw_cmd = ' '.join(raw_cmd.split()[:-1])
Copy link
Member

@jiasli jiasli May 10, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please avoid reusing existing name. Instead, introduce a new name to make the code easier to read, such as raw_cmd_without_last_word.

@bebound bebound merged commit 3e7a96f into Azure:dev May 10, 2023
yash-nisar pushed a commit to yash-nisar/azure-cli that referenced this pull request May 16, 2023
@bebound bebound deleted the load-module branch May 17, 2023 08:54
avgale pushed a commit to avgale/azure-cli that referenced this pull request Aug 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Auto-Assign Auto assign by bot Core CLI core infrastructure
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Autocomplete is slow when command is not complete
3 participants