Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fixed bug where getFileClient and getSubDirectoryClient on DirectoryClient would throw IllegalArgumentException if either resource had special characters. #18542

Merged
merged 3 commits into from
Jan 13, 2021
Merged

Fixed bug where getFileClient and getSubDirectoryClient on DirectoryClient would throw IllegalArgumentException if either resource had special characters. #18542

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
b963566
Fixed bug where special characters would not work in getClient method…
gapra-msft Jan 11, 2021
1d1c9ce
Fixed bug
gapra-msft Jan 11, 2021
b38b315
Added code to check object path
gapra-msft Jan 13, 2021
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion sdk/storage/azure-storage-file-datalake/CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
# Release History

## 12.4.0-beta.2 (Unreleased)

- Fixed bug where getFileClient and getSubDirectoryClient on DirectoryClient would throw IllegalArgumentException if either resource had special characters.

## 12.4.0-beta.1 (2020-12-07)
- Added support to list paths on a directory.
Expand Down
8 changes: 4 additions & 4 deletions ...-datalake/src/main/java/com/azure/storage/file/datalake/DataLakeDirectoryAsyncClient.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -173,8 +173,8 @@ public DataLakeFileAsyncClient getFileAsyncClient(String fileName) {

return new DataLakeFileAsyncClient(getHttpPipeline(),
StorageImplUtils.appendToUrlPath(getPathUrl(), Utility.urlEncode(Utility.urlDecode(fileName))).toString(),
getServiceVersion(), getAccountName(), getFileSystemName(), getObjectPath() + "/"
+ Utility.urlDecode(fileName), blockBlobAsyncClient);
getServiceVersion(), getAccountName(), getFileSystemName(), Utility.urlEncode(getObjectPath() + "/"
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This constructor expects an encoded path

+ Utility.urlDecode(fileName)), blockBlobAsyncClient);
}

/**
Expand Down Expand Up @@ -315,8 +315,8 @@ public DataLakeDirectoryAsyncClient getSubdirectoryAsyncClient(String subdirecto

return new DataLakeDirectoryAsyncClient(getHttpPipeline(),
StorageImplUtils.appendToUrlPath(getPathUrl(), Utility.urlEncode(Utility.urlDecode(subdirectoryName)))
.toString(), getServiceVersion(), getAccountName(), getFileSystemName(), getObjectPath() + "/"
+ Utility.urlDecode(subdirectoryName), blockBlobAsyncClient);
.toString(), getServiceVersion(), getAccountName(), getFileSystemName(),
Utility.urlEncode(getObjectPath() + "/" + Utility.urlDecode(subdirectoryName)), blockBlobAsyncClient);
}

/**
Expand Down
32 changes: 30 additions & 2 deletions ...orage-file-datalake/src/test/java/com/azure/storage/file/datalake/DirectoryAPITest.groovy
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,12 +6,11 @@ import com.azure.core.http.HttpPipelineNextPolicy
import com.azure.core.http.HttpRequest
import com.azure.core.http.policy.HttpPipelinePolicy
import com.azure.core.http.rest.Response
import com.azure.core.test.http.MockHttpResponse
import com.azure.core.util.Context
import com.azure.identity.DefaultAzureCredentialBuilder
import com.azure.storage.blob.BlobUrlParts
import com.azure.storage.blob.models.BlobErrorCode

import com.azure.storage.common.Utility
import com.azure.storage.file.datalake.models.*
import com.azure.storage.file.datalake.options.PathRemoveAccessControlRecursiveOptions
import com.azure.storage.file.datalake.options.PathSetAccessControlRecursiveOptions
Expand Down Expand Up @@ -3001,4 +3000,33 @@ class DirectoryAPITest extends APISpec {
then:
thrown(DataLakeStorageException)
}

@Unroll
def "Get file and subdirectory client"() {
setup:
def dirName = generatePathName()
def subPath = generatePathName()
dc = fsc.getDirectoryClient(resourcePrefix + dirName)

when:
def fileClient = dc.getFileClient(subResourcePrefix + subPath)

then:
notThrown(IllegalArgumentException)
fileClient.getFilePath() == Utility.urlDecode(resourcePrefix) + dirName + "/" + Utility.urlDecode(subResourcePrefix) + subPath

when:
def subDirectoryClient = dc.getSubdirectoryClient(subResourcePrefix + subPath)

then:
notThrown(IllegalArgumentException)
subDirectoryClient.getDirectoryPath() == Utility.urlDecode(resourcePrefix) + dirName + "/" + Utility.urlDecode(subResourcePrefix) + subPath

where:
resourcePrefix | subResourcePrefix || _
"" | "" || _
Utility.urlEncode("%") | "" || _ // Resource has special character
"" | Utility.urlEncode("%") || _ // Sub resource has special character
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should check the path against the expected string to ensure the right amount of encoding/decoding is happening

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I can add that

Utility.urlEncode("%") | Utility.urlEncode("%") || _
}
}
86 changes: 86 additions & 0 deletions ...e/src/test/resources/session-records/DirectoryAPITestgetfileandsubdirectoryclient[0].json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
{
"networkCallRecords" : [ {
"Method" : "PUT",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient0666514fdd6f3a9f15?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "5cc3fb9d-064e-4f74-89bb-e3cdac37eb45"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B672442E282B",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:35 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "640874c0-d01e-003a-2d5b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:35 GMT",
"x-ms-client-request-id" : "5cc3fb9d-064e-4f74-89bb-e3cdac37eb45"
},
"Exception" : null
}, {
"Method" : "PUT",
"Uri" : "https://REDACTED.dfs.core.windows.net/jtfsgetfileandsubdirectoryclient0666514fdd6f3a9f15/javapathgetfileandsubdirectoryclient1242530d3d09019a?resource=directory",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-file-datalake/12.4.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "be540635-f414-4786-88b0-177efde83845"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B67244B0FC45",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:36 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "4c80f854-b01f-005e-465b-e8a640000000",
"Date" : "Mon, 11 Jan 2021 20:48:36 GMT",
"x-ms-client-request-id" : "be540635-f414-4786-88b0-177efde83845"
},
"Exception" : null
}, {
"Method" : "GET",
"Uri" : "https://REDACTED.blob.core.windows.net?prefix=jtfsgetfileandsubdirectoryclient&comp=list",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "c3e0539f-8f88-44f0-9301-b4c2e1a656be"
},
"Response" : {
"Transfer-Encoding" : "chunked",
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"StatusCode" : "200",
"x-ms-request-id" : "640875af-d01e-003a-6f5b-e857d8000000",
"Body" : "<?xml version=\"1.0\" encoding=\"utf-8\"?><EnumerationResults ServiceEndpoint=\"https://gaprahnscanary.blob.core.windows.net/\"><Prefix>jtfsgetfileandsubdirectoryclient</Prefix><Containers><Container><Name>jtfsgetfileandsubdirectoryclient0666514fdd6f3a9f15</Name><Properties><Last-Modified>Mon, 11 Jan 2021 20:48:35 GMT</Last-Modified><Etag>\"0x8D8B672442E282B\"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><DefaultEncryptionScope>$account-encryption-key</DefaultEncryptionScope><DenyEncryptionScopeOverride>false</DenyEncryptionScopeOverride><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container></Containers><NextMarker /></EnumerationResults>",
"Date" : "Mon, 11 Jan 2021 20:48:36 GMT",
"x-ms-client-request-id" : "c3e0539f-8f88-44f0-9301-b4c2e1a656be",
"Content-Type" : "application/xml"
},
"Exception" : null
}, {
"Method" : "DELETE",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient0666514fdd6f3a9f15?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "3792ad7a-f8e0-4459-ac9b-f7de17d54381"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "202",
"x-ms-request-id" : "6408762a-d01e-003a-605b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "3792ad7a-f8e0-4459-ac9b-f7de17d54381"
},
"Exception" : null
} ],
"variables" : [ "jtfsgetfileandsubdirectoryclient0666514fdd6f3a9f15", "javapathgetfileandsubdirectoryclient1242530d3d09019a", "javapathgetfileandsubdirectoryclient268667bca3234f49", "javapathgetfileandsubdirectoryclient310511c81b3604ce", "javapathgetfileandsubdirectoryclient424462a215a0ce94" ]
}
86 changes: 86 additions & 0 deletions ...e/src/test/resources/session-records/DirectoryAPITestgetfileandsubdirectoryclient[1].json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
{
"networkCallRecords" : [ {
"Method" : "PUT",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient092223b7473ffd84a9?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "3689dfa2-9413-4056-ab46-80ae36efe00f"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B672453F54BD",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:37 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "6408765c-d01e-003a-095b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "3689dfa2-9413-4056-ab46-80ae36efe00f"
},
"Exception" : null
}, {
"Method" : "PUT",
"Uri" : "https://REDACTED.dfs.core.windows.net/jtfsgetfileandsubdirectoryclient092223b7473ffd84a9/javapathgetfileandsubdirectoryclient1584631ef808625a?resource=directory",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-file-datalake/12.4.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "053f4dc6-724d-48ee-9f2f-1ce4b9f3fe16"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B67245534FC5",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:37 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "4c80f85a-b01f-005e-4b5b-e8a640000000",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "053f4dc6-724d-48ee-9f2f-1ce4b9f3fe16"
},
"Exception" : null
}, {
"Method" : "GET",
"Uri" : "https://REDACTED.blob.core.windows.net?prefix=jtfsgetfileandsubdirectoryclient&comp=list",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "6471c8a6-d6ab-482c-af56-7fc2391f14e5"
},
"Response" : {
"Transfer-Encoding" : "chunked",
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"StatusCode" : "200",
"x-ms-request-id" : "6408768c-d01e-003a-335b-e857d8000000",
"Body" : "<?xml version=\"1.0\" encoding=\"utf-8\"?><EnumerationResults ServiceEndpoint=\"https://gaprahnscanary.blob.core.windows.net/\"><Prefix>jtfsgetfileandsubdirectoryclient</Prefix><Containers><Container><Name>jtfsgetfileandsubdirectoryclient092223b7473ffd84a9</Name><Properties><Last-Modified>Mon, 11 Jan 2021 20:48:37 GMT</Last-Modified><Etag>\"0x8D8B672453F54BD\"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><DefaultEncryptionScope>$account-encryption-key</DefaultEncryptionScope><DenyEncryptionScopeOverride>false</DenyEncryptionScopeOverride><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container></Containers><NextMarker /></EnumerationResults>",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "6471c8a6-d6ab-482c-af56-7fc2391f14e5",
"Content-Type" : "application/xml"
},
"Exception" : null
}, {
"Method" : "DELETE",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient092223b7473ffd84a9?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "3c7848b2-265e-4a4a-b78a-b8a891066236"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "202",
"x-ms-request-id" : "64087695-d01e-003a-3a5b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "3c7848b2-265e-4a4a-b78a-b8a891066236"
},
"Exception" : null
} ],
"variables" : [ "jtfsgetfileandsubdirectoryclient092223b7473ffd84a9", "javapathgetfileandsubdirectoryclient1584631ef808625a", "javapathgetfileandsubdirectoryclient203832f3c79f3223", "javapathgetfileandsubdirectoryclient37667723039e028c", "javapathgetfileandsubdirectoryclient4297577ed7d29e7e" ]
}
86 changes: 86 additions & 0 deletions ...e/src/test/resources/session-records/DirectoryAPITestgetfileandsubdirectoryclient[2].json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,86 @@
{
"networkCallRecords" : [ {
"Method" : "PUT",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient0986864c201d882b62?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "cb844021-53d8-4480-8236-d3aaed8c7802"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B6724585E075",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:38 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "640876b5-d01e-003a-535b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:37 GMT",
"x-ms-client-request-id" : "cb844021-53d8-4480-8236-d3aaed8c7802"
},
"Exception" : null
}, {
"Method" : "PUT",
"Uri" : "https://REDACTED.dfs.core.windows.net/jtfsgetfileandsubdirectoryclient0986864c201d882b62/javapathgetfileandsubdirectoryclient13026056ef99c5d2?resource=directory",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-file-datalake/12.4.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "ab85e036-cf53-4271-8e94-b79fc59a05a0"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-HDFS/1.0 Microsoft-HTTPAPI/2.0",
"ETag" : "0x8D8B67245BC6741",
"Last-Modified" : "Mon, 11 Jan 2021 20:48:38 GMT",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "201",
"x-ms-request-id" : "4c80f85b-b01f-005e-4c5b-e8a640000000",
"Date" : "Mon, 11 Jan 2021 20:48:38 GMT",
"x-ms-client-request-id" : "ab85e036-cf53-4271-8e94-b79fc59a05a0"
},
"Exception" : null
}, {
"Method" : "GET",
"Uri" : "https://REDACTED.blob.core.windows.net?prefix=jtfsgetfileandsubdirectoryclient&comp=list",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "a83baa00-e225-4eef-a15f-e02ec027afce"
},
"Response" : {
"Transfer-Encoding" : "chunked",
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"StatusCode" : "200",
"x-ms-request-id" : "640876d6-d01e-003a-6d5b-e857d8000000",
"Body" : "<?xml version=\"1.0\" encoding=\"utf-8\"?><EnumerationResults ServiceEndpoint=\"https://gaprahnscanary.blob.core.windows.net/\"><Prefix>jtfsgetfileandsubdirectoryclient</Prefix><Containers><Container><Name>jtfsgetfileandsubdirectoryclient0986864c201d882b62</Name><Properties><Last-Modified>Mon, 11 Jan 2021 20:48:38 GMT</Last-Modified><Etag>\"0x8D8B6724585E075\"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><DefaultEncryptionScope>$account-encryption-key</DefaultEncryptionScope><DenyEncryptionScopeOverride>false</DenyEncryptionScopeOverride><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container></Containers><NextMarker /></EnumerationResults>",
"Date" : "Mon, 11 Jan 2021 20:48:38 GMT",
"x-ms-client-request-id" : "a83baa00-e225-4eef-a15f-e02ec027afce",
"Content-Type" : "application/xml"
},
"Exception" : null
}, {
"Method" : "DELETE",
"Uri" : "https://REDACTED.blob.core.windows.net/jtfsgetfileandsubdirectoryclient0986864c201d882b62?restype=container",
"Headers" : {
"x-ms-version" : "2020-04-08",
"User-Agent" : "azsdk-java-azure-storage-blob/12.10.0-beta.2 (11.0.7; Windows 10; 10.0)",
"x-ms-client-request-id" : "0dbfee6f-76b4-4a02-ad06-4e02ed2479ad"
},
"Response" : {
"x-ms-version" : "2020-04-08",
"Server" : "Windows-Azure-Blob/1.0 Microsoft-HTTPAPI/2.0",
"retry-after" : "0",
"Content-Length" : "0",
"StatusCode" : "202",
"x-ms-request-id" : "640876e7-d01e-003a-7d5b-e857d8000000",
"Date" : "Mon, 11 Jan 2021 20:48:38 GMT",
"x-ms-client-request-id" : "0dbfee6f-76b4-4a02-ad06-4e02ed2479ad"
},
"Exception" : null
} ],
"variables" : [ "jtfsgetfileandsubdirectoryclient0986864c201d882b62", "javapathgetfileandsubdirectoryclient13026056ef99c5d2", "javapathgetfileandsubdirectoryclient2472502d96f50b27", "javapathgetfileandsubdirectoryclient343430a8d4a52cb0", "javapathgetfileandsubdirectoryclient4233732a4ae4270d" ]
}
Loading