AzCopy sync with hash-meta-dir hides original files

[tsvietOK]

Which version of the AzCopy was used?

10.25.1

Note: The version is visible when running AzCopy without any argument

Which platform are you using? (ex: Windows, Mac, Linux)

Windows

What command did you run?

azcopy sync ".\testData\" "https://<storage>.blob.core.windows.net/<container>/<folderName>/?sp=<sig>" --delete-destination true --exclude-path "backup;import;system/caches" --compare-hash "MD5" --put-md5 --cap-mbps 200 --local-hash-storage-mode HiddenFiles --hash-meta-dir "E:\hash-meta"

Note: Please remove the SAS to avoid exposing your credentials. If you cannot remember the exact command, please retrieve it from the beginning of the log file.

What problem was encountered?

Command succeeded, but it created a new folder E:\hash-meta with metadata. After that, I noticed that all the files in the original folder were hidden.

How can we reproduce the problem in the simplest way?

1. Create folder testData with test files
2. Run powershell in the testData folder
3. Run command (change hash meta dir to your place) azcopy sync ".\testData\" "https://<storage>.blob.core.windows.net/<container>/<folderName>/?sp=<sig>" --delete-destination true --exclude-path "backup;import;system/caches" --compare-hash "MD5" --put-md5 --cap-mbps 200 --local-hash-storage-mode HiddenFiles --hash-meta-dir "E:\hash-meta"
4. Check that the data files (in folder testData) are hidden and meta data (folder hash-meta) files are not hidden.

Have you found a mitigation/solution?

No

[tsvietOK]

[tsvietOK]

I guess here should be metaFile:
https://github.com/Azure/azure-storage-azcopy/blob/main/common/hash_data_adapter_hidden_files.go#L88

[ktrace2]

azcopy kept crashing on me when using the hash-meta-dir option. I assumed it was related to this bug.

[ashruti-msft]

azcopy kept crashing on me when using the hash-meta-dir option. I assumed it was related to this bug.

@ktrace2 which version of azcopy are you using when it crashes? Also can you share your debug logs for that particular run?

[adreed-msft]

This looks very much like a bug, yes... I'll add this to our backlog, hopefully we can get to it this release.

azcopy kept crashing on me when using the hash-meta-dir option. I assumed it was related to this bug.

It's possible. Do you have the logs for this crash? What version were you on?

[tsvietOK]

This looks very much like a bug, yes... I'll add this to our backlog, hopefully we can get to it this release.

Thank you.