stat: path too long for Windows - error when using large parameters.json file

[ajklotz]

This is autogenerated. Please review and update as needed.

Describe the bug

Command Name
az group deployment create

Errors:

stat: path too long for Windows

To Reproduce:

Create a parameters.json file that has 490+ lines with name value pairs. The total character count of said JSON file is 37343. Total file size is 37KB. The structure of the json file looks similar this:

{
    "$schema": "https://schema.management.azure.com/schemas/2015-01-01/deploymentParameters.json#",
    "contentVersion": "1.0.0.0",
    "parameters": {
        "applicationSettings": {
            "value": {
                "CI": {
                    "AzureWebJobsStorage": "areallylongvaluethatyoumustputhereinordertoreproducetheproblemsojustaddabunchofletterstoproducetheproblemandthenaddthistoeachfieldletmeknowhowitworks",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                },
                "DEV": {
                    "AzureWebJobsStorage": "",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                },
                "QA": {
                    "AzureWebJobsStorage": "",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                },
                "STG": {
                    "AzureWebJobsStorage": "",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                },
                "TRN": {
                    "AzureWebJobsStorage": "",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                },
                "PROD": {
                    "AzureWebJobsStorage": "",
                    "AzureWebJobsDashboard": "",
                    "AzureWebJobsServiceBus": "",
                    "AzureWebJobsServiceBusSource": ""
                }
            }
        }
    }
}

For each object in applicationSettings, there are 78 name value pairs with a value. When I remove chunks of JSON, the command works. The JSON file is formatted correctly and there are no errors. There appears to be a problem parsing a json file with a lot of content.

• Put any pre-requisite steps here...
• az group deployment create --resource-group WB-CI --template-file .\function-app-orderProcessing.json --parameter @function-app-orderProcessing.parameters.json

Expected Behavior

Environment Summary

Windows-10-10.0.17763-SP0
Python 3.6.6
Shell: powershell.exe

azure-cli 2.0.78

Additional Context

[mmyyrroonn]

@zhoxing-ms Please take a look.

[ajklotz]

Any update?

[zhoxing-ms]

@ajklotz Sorry for the late reply. Could you send the json file of --template-file and --parameter to me by email ([email protected])? If there is sensitive data, you can replace it with encrypted characters, thank you~

[ajklotz]

I sent the attachments to your email @zhoxing-ms

[zhoxing-ms]

@ajklotz I tried to use the parameter file provided by you, and the problem was not reproduced even after I changed the content of the file to a larger size.
I think this problem may be caused by the file path being too deep，Could you please tell me the path of your parameter file?

[ajklotz]

@zhoxing-ms Just curious, are you using Powershell or Bash? In my example provided above, the parameter file resided in the same directory as the current working directory for powershell.

To answer your question, my parameter file resided in a path that looks like this:

C:\codebase\digital\CI\Scripts\AzureCLI\AzureFunctions\OrderProcessing

But my powershell session was set to that path already so it used relative path to access the file.

Edit: I tested command by copying the template and parameter file to a shorter path in Windows and still had the same problem. I moved my files to C:\users\klotz\desktop

[ajklotz]

In the file I emailed to you, do a "replace all" and replace "" with "areallylongvaluethatyoumustputhereinordertoreproducetheproblemsojustaddabunchofletterstoproducetheproblemandthenaddthistoeachfieldletmeknowhowitworks"

[ajklotz]

Were you able to reproduce the problem with values for every key?

[zhoxing-ms]

@a-vishar Because I'm taking Chinese annual leave and the internet condition at home is not good, so I'm sorry to reply so late.
I tried both PowerShell and Bash, and I still can't reproduce the problem after doing a "replace all" and replace "" with "areallylongvaluethatyoumustputhereinordertoreproducetheproblemsojustaddabunchofletterstoproducetheproblemandthenaddthistoeachfieldletmeknowhowitworks"

[a-vishar]

Not sure why I was tagged here.

[yonzhan]

@ajklotz please look at the above comment by @zhoxing-ms

[zhoxing-ms]

@a-vishar Please add --debug parameter to the command (like this: az group deployment create --resource-group WB-CI --template-file .\function-app-orderProcessing.json --parameter @function-app-orderProcessing.parameters.jso --debug) , and then paste the complete exception stack for me to see.

[ajklotz]

PS C:\codebase\digital\CI\ARM> az group deployment create --resource-group WB-CI --template-file .\function-app-orderProcessing.json --parameter @function-app-orderProcessing.parameters.json --debug
Command arguments: ['group', 'deployment', 'create', '--resource-group', 'WB-CI', '--template-file', '.\\function-app-orderProcessing.json', '--parameter', '@function-app-orderProcessing.parameters.json', '--debug']
Event: Cli.PreExecute []
Event: CommandParser.OnGlobalArgumentsCreate [<function CLILogging.on_global_arguments at 0x03E029C0>, <function OutputProducer.on_global_arguments at 0x03EC6660>, <function CLIQuery.on_global_arguments at 0x03F2E738>]
attempting to read file function-app-orderProcessing.parameters.json as utf-8-sig
Event: CommandInvoker.OnPreCommandTableCreate []
Installed command modules ['acr', 'acs', 'advisor', 'ams', 'apim', 'appconfig', 'appservice', 'backup', 'batch', 'batchai', 'billing', 'botservice', 'cdn', 'cloud', 'cognitiveservices', 'configure', 'consumption', 'container', 'cosmosdb', 'deploymentmanager', 'dla', 'dls', 'dms', 'eventgrid', 'eventhubs', 'extension', 'feedback', 'find', 'hdinsight', 'interactive', 'iot', 'iotcentral', 'keyvault', 'kusto', 'lab', 'managedservices', 'maps', 'monitor', 'natgateway', 'netappfiles', 'network', 'policyinsights', 'privatedns', 'profile', 'rdbms', 'redis', 'relay', 'reservations', 'resource', 'role', 'search', 'security', 'servicebus', 'servicefabric', 'signalr', 'sql', 'sqlvm', 'storage', 'vm']
Loaded module 'acr' in 0.018 seconds.
Loaded module 'acs' in 0.014 seconds.
Loaded module 'advisor' in 0.004 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'ams' in 0.013 seconds.
Loaded module 'apim' in 0.005 seconds.
Loaded module 'appconfig' in 0.005 seconds.
Loaded module 'appservice' in 0.018 seconds.
Loaded module 'backup' in 0.007 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'batch' in 0.016 seconds.
Loaded module 'batchai' in 0.006 seconds.
Loaded module 'billing' in 0.005 seconds.
Loaded module 'botservice' in 0.009 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'cdn' in 0.006 seconds.
Loaded module 'cloud' in 0.004 seconds.
Loaded module 'cognitiveservices' in 0.005 seconds.
Loaded module 'configure' in 0.004 seconds.
Loaded module 'consumption' in 0.006 seconds.
Loaded module 'container' in 0.005 seconds.
Loaded module 'cosmosdb' in 0.013 seconds.
Loaded module 'deploymentmanager' in 0.006 seconds.
Loaded module 'dla' in 0.008 seconds.
Loaded module 'dls' in 0.007 seconds.
Loaded module 'dms' in 0.004 seconds.
Loaded module 'eventgrid' in 0.004 seconds.
Loaded module 'eventhubs' in 0.006 seconds.
Loaded module 'extension' in 0.002 seconds.
Loaded module 'feedback' in 0.002 seconds.
Loaded module 'find' in 0.002 seconds.
Loaded module 'hdinsight' in 0.004 seconds.
Loaded module 'interactive' in 0.001 seconds.
Loaded module 'iot' in 0.007 seconds.
Loaded module 'iotcentral' in 0.002 seconds.
Loaded module 'keyvault' in 0.008 seconds.
Loaded module 'kusto' in 0.004 seconds.
Loaded module 'lab' in 0.005 seconds.
Loaded module 'managedservices' in 0.003 seconds.
Loaded module 'maps' in 0.003 seconds.
Loaded module 'monitor' in 0.009 seconds.
Loaded module 'natgateway' in 0.003 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'netappfiles' in 0.006 seconds.
Loaded module 'network' in 0.033 seconds.
Loaded module 'policyinsights' in 0.003 seconds.
Loaded module 'privatedns' in 0.008 seconds.
Loaded module 'profile' in 0.004 seconds.
Loaded module 'rdbms' in 0.012 seconds.
Loaded module 'redis' in 0.005 seconds.
Loaded module 'relay' in 0.007 seconds.
Loaded module 'reservations' in 0.005 seconds.
Loaded module 'resource' in 0.013 seconds.
Loaded module 'role' in 0.008 seconds.
Loaded module 'search' in 0.004 seconds.
Loaded module 'security' in 0.004 seconds.
Loaded module 'servicebus' in 0.009 seconds.
Loaded module 'servicefabric' in 0.004 seconds.
Loaded module 'signalr' in 0.003 seconds.
Loaded module 'sql' in 0.010 seconds.
Loaded module 'sqlvm' in 0.004 seconds.
Event: CommandLoader.OnLoadCommandTable []
Loaded module 'storage' in 0.033 seconds.
Loaded module 'vm' in 0.027 seconds.
Loaded all modules in 0.458 seconds. (note: there's always an overhead with the first module loaded)
Extensions directory: 'C:\Users\aklotz\.azure\cliextensions'
Event: CommandInvoker.OnPreCommandTableTruncate [<function AzCliLogging.init_command_file_logging at 0x03E468E8>]
az_command_data_logger : command args: group deployment create --resource-group {} --template-file {} --parameter {} --debug
metadata file logging enabled - writing logs to 'C:\Users\aklotz\.azure\commands'.
Event: CommandInvoker.OnPreArgumentLoad [<function register_global_subscription_argument.<locals>.add_subscription_parameter at 0x040F8A98>]
Configured default 'WB-CI' for arg resource_group_name
Event: CommandInvoker.OnPostArgumentLoad []
Event: CommandInvoker.OnPostCommandTableCreate [<function register_ids_argument.<locals>.add_ids_arguments at 0x041044F8>, <function register_cache_arguments.<locals>.add_cache_arguments at 0x041045D0>]
Event: CommandInvoker.OnCommandTableLoaded []
Event: CommandInvoker.OnPreParseArgs [<function _documentdb_deprecate at 0x04C87270>]
Event: CommandInvoker.OnPostParseArgs [<function OutputProducer.handle_output_argument at 0x03EC66A8>, <function CLIQuery.handle_query_parameter at 0x03F2E780>, <function register_ids_argument.<locals>.parse_ids_arguments at 0x04104588>, <function handler at 0x04D5E2B8>]
attempting to read file .\function-app-orderProcessing.json as utf-8-sig
cli.azure.cli.core.util : stat: path too long for Windows
stat: path too long for Windows
az_command_data_logger : exit code: 1
telemetry.save : Save telemetry record of length 2502 in cache
telemetry.check : Negative: The C:\Users\aklotz\.azure\telemetry.txt was modified at 2020-02-03 08:35:57.685168, which in less than 600.000000 s
command ran in 1.289 seconds.

[a-vishar]

@zhoxing-ms - Please stop tagging me in this issue, I have nothing to do with it.

[zhoxing-ms]

@ajklotz I found out that the problem is probably caused by the @ of --parameter @function-app-orderProcessing.parameters.json. You can first try whether removing the @ prefix can solve the problem.
I will locate the specific reason and sync it later.

[ajklotz]

@a-vishar You are right, the error doesn't occur when I remove @. I was told to provide the @ prefix in another github issue, but I can omit that. I'm glad to hear you are finding the root cause, thank you so much.

I got the idea to use @ from this github issue #11600

[a-vishar]

@ajklotz Am I being punked? Please stop mentioning me in this random GitHub issue. I have nothing to do with this issue and for some reason you all keep mentioning me.

[ajklotz]

Sorry @a-vishar

[zhoxing-ms]

@ajklotz This problem is also related to the version of Python, so you can reinstall azure-cli by using Python 3.8 first.
I've identified the underlying problem, and I'll submit a PR to fix it in other versions of Python.