Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BlobUpload throws "The specifed resource name contains invalid characters" with Azure.Storage.Blobs #839

Closed
Carael opened this issue Jun 18, 2021 · 21 comments
Assignees
Labels
blob-storage question Further information is requested

Comments

@Carael
Copy link

Carael commented Jun 18, 2021

In Squadron we are using Azurite to spin up a docker container that can be easily used in integration tests.

Recently I discovered, after trying to Migrate from Microsoft.Azure.Storage.Blob to Azure.Storage.Blobs, that BlobClient.UploadAsync() returns every time "The specified resource name contains invalid characters" when I try to use the BlobContainerClient.UploadAsync().

Version of Azure.Storage.Blobs is 12.9.0. The resource Uri is: http://localhost:49205/devstoreaccount1/foo so it doesn't have any invalid character.

Sample code test. Name of the failing test: BlobClient_UploadFile_ContentMatch.

The same code works against Azure Storage and with the old library.

Exception message:
` Message:
Azure.RequestFailedException : The specifed resource name contains invalid characters.
RequestId:75c2151b-c75d-4142-bb6c-33f755c736f3
Time:2021-06-18T10:43:31.100Z
Status: 400 (The specifed resource name contains invalid characters.)
ErrorCode: InvalidResourceName

Content:
<?xml version="1.0" encoding="UTF-8" standalone="yes"?>
<Error>
  <Code>InvalidResourceName</Code>
  <Message>The specifed resource name contains invalid characters.
RequestId:75c2151b-c75d-4142-bb6c-33f755c736f3
Time:2021-06-18T10:43:31.100Z</Message>
</Error>

Headers:
Server: Azurite-Blob/3.13.1
x-ms-error-code: InvalidResourceName
x-ms-request-id: 75c2151b-c75d-4142-bb6c-33f755c736f3
Date: Fri, 18 Jun 2021 10:43:31 GMT
Connection: keep-alive
Keep-Alive: REDACTED
Transfer-Encoding: chunked
Content-Type: application/xml

Stack Trace:
BlockBlobRestClient.UploadAsync(Int64 contentLength, Stream body, Nullable1 timeout, Byte[] transactionalContentMD5, String blobContentType, String blobContentEncoding, String blobContentLanguage, Byte[] blobContentMD5, String blobCacheControl, IDictionary2 metadata, String leaseId, String blobContentDisposition, String encryptionKey, String encryptionKeySha256, Nullable1 encryptionAlgorithm, String encryptionScope, Nullable1 tier, Nullable1 ifModifiedSince, Nullable1 ifUnmodifiedSince, String ifMatch, String ifNoneMatch, String ifTags, String blobTagsString, Nullable1 immutabilityPolicyExpiry, Nullable1 immutabilityPolicyMode, Nullable1 legalHold, CancellationToken cancellationToken) BlockBlobClient.UploadInternal(Stream content, BlobHttpHeaders blobHttpHeaders, IDictionary2 metadata, IDictionary2 tags, BlobRequestConditions conditions, Nullable1 accessTier, IProgress1 progressHandler, String operationName, Boolean async, CancellationToken cancellationToken) <<GetPartitionedUploaderBehaviors>b__0>d.MoveNext() --- End of stack trace from previous location where exception was thrown --- PartitionedUploader2.UploadInternal(Stream content, TServiceSpecificArgs args, IProgress1 progressHandler, Boolean async, CancellationToken cancellationToken) BlobClient.StagedUploadInternal(Stream content, BlobUploadOptions options, Boolean async, CancellationToken cancellationToken) AzureStorageBlobResourceTests.BlobClient_UploadFile_ContentMatch() line 55 --- End of stack trace from previous location where exception was thrown ---

@moraleslevi
Copy link

moraleslevi commented Jun 22, 2021

Wanted to confirm I am also seeing this issue when accessing the Emulator Blob Containers in Azure Storage Explorer v1.19.1.

@blueww
Copy link
Member

blueww commented Jun 23, 2021

@moraleslevi

Would you please attach the Azurite debug log for the failure, so we can see the details and investigate it?

From the test code, it seems you will first create a container with name "foo", then upload a blob with name "test.txt" into it.
You have test case in both old and new SDK.
Would you please confirm will the issue happen both with Azurite + old/new SDK?

And I have tried to run the test case with new SDK on Azurite 3.13.1, and it work well, I can't repro the failure.

@blueww blueww added blob-storage question Further information is requested labels Jun 23, 2021
@blueww blueww self-assigned this Jun 23, 2021
@Carael
Copy link
Author

Carael commented Jun 23, 2021

The same test case works in old sdk and doesn't in the new sdk.

I've was able to reproduce the bug with Azurite 3.13.1 and new sdk (12.9.1). Debug log:

2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: PublicAccessAuthenticator:validate() Start validation against public access. 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 debug: PublicAccessAuthenticator:validate() Getting account properties... 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: foo, blob: 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 debug: PublicAccessAuthenticator:validate() Skip public access authentication. Cannot get public access type for container foo 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication. 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"PUT\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:51d06639-d43e-4de3-a9f5-585bb5d761fe\nx-ms-date:Wed, 23 Jun 2021 10:25:52 GMT\nx-ms-return-client-request-id:true\nx-ms-version:2020-08-04\n/devstoreaccount1/devstoreaccount1/foo\nrestype:container" 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:Mo8+/MZUzSbCyN3h8wYkjiZVQWUvA7GYsKvDKn7i8zw= 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched. 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 verbose: DeserializerMiddleware: Start deserializing... 2021-06-23T10:25:52.782Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: HandlerMiddleware: DeserializedParameters={"options":{"metadata":{},"requestId":"51d06639-d43e-4de3-a9f5-585bb5d761fe"},"restype":"container","version":"2020-08-04"} 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The specified container already exists. ErrorHTTPStatusCode=409 ErrorHTTPStatusMessage=The specified container already exists. ErrorHTTPHeaders={"x-ms-error-code":"ContainerAlreadyExists","x-ms-request-id":"ef2352a6-2058-482d-92cf-fccb6460aec9"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n <Code>ContainerAlreadyExists</Code>\n <Message>The specified container already exists.\nRequestId:ef2352a6-2058-482d-92cf-fccb6460aec9\nTime:2021-06-23T10:25:52.782Z</Message>\n</Error>" ErrorStack="StorageError: The specified container already exists.\n at Function.getContainerAlreadyExists (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:23:16)\n at LokiBlobMetadataStore.createContainer (/opt/azurite/dist/src/blob/persistence/LokiBlobMetadataStore.js:292:49)\n at ContainerHandler.create (/opt/azurite/dist/src/blob/handlers/ContainerHandler.js:35:34)\n at /opt/azurite/dist/src/blob/generated/middleware/HandlerMiddlewareFactory.js:58:18\n at /opt/azurite/dist/src/blob/generated/ExpressMiddlewareFactory.js:77:63\n at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:317:13)\n at /opt/azurite/node_modules/express/lib/router/index.js:284:7\n at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:335:12)\n at next (/opt/azurite/node_modules/express/lib/router/index.js:275:10)" 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set HTTP code: 409 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set HTTP status message: The specified container already exists. 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=ContainerAlreadyExists 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=ef2352a6-2058-482d-92cf-fccb6460aec9 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set content type: application/xml 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n <Code>ContainerAlreadyExists</Code>\n <Message>The specified container already exists.\nRequestId:ef2352a6-2058-482d-92cf-fccb6460aec9\nTime:2021-06-23T10:25:52.782Z</Message>\n</Error>" 2021-06-23T10:25:52.783Z ef2352a6-2058-482d-92cf-fccb6460aec9 info: EndMiddleware: End response. TotalTimeInMS=2 StatusCode=409 StatusMessage=The specified container already exists. Headers={"server":"Azurite-Blob/3.13.1","x-ms-error-code":"ContainerAlreadyExists","x-ms-request-id":"ef2352a6-2058-482d-92cf-fccb6460aec9","content-type":"application/xml"} 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef info: BlobStorageContextMiddleware: RequestMethod=PUT RequestURL=http://localhost/devstoreaccount1/test.txt RequestHeaders:{"host":"localhost:7777","x-ms-blob-type":"BlockBlob","x-ms-version":"2020-08-04","accept":"application/xml","x-ms-client-request-id":"9e56a0d8-c59f-45ae-bee4-a701a1fa19c3","x-ms-return-client-request-id":"true","user-agent":"azsdk-net-Storage.Blobs/12.9.1 (.NET Core 3.1.16; Microsoft Windows 10.0.19042)","x-ms-date":"Wed, 23 Jun 2021 10:25:52 GMT","authorization":"SharedKey devstoreaccount1:pplPB6jFu+KE1ULloGaodfD4GNoeAHJGuYKfJp/IkPg=","content-length":"18","content-type":"application/octet-stream"} ClientIP=172.17.0.1 Protocol=http HTTPVersion=1.1 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The specifed resource name contains invalid characters. ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=The specifed resource name contains invalid characters. ErrorHTTPHeaders={"x-ms-error-code":"InvalidResourceName","x-ms-request-id":"974d1f1b-df04-4012-82b1-d944e14d7cef"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n <Code>InvalidResourceName</Code>\n <Message>The specifed resource name contains invalid characters.\nRequestId:974d1f1b-df04-4012-82b1-d944e14d7cef\nTime:2021-06-23T10:25:52.816Z</Message>\n</Error>" ErrorStack="StorageError: The specifed resource name contains invalid characters.\n at Function.getInvalidResourceName (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:228:16)\n at Object.validateContainerName (/opt/azurite/dist/src/blob/utils/utils.js:109:45)\n at blobStorageContextMiddleware (/opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:69:17)\n at /opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:14:16\n at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:317:13)\n at /opt/azurite/node_modules/express/lib/router/index.js:284:7\n at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:335:12)\n at next (/opt/azurite/node_modules/express/lib/router/index.js:275:10)\n at logger (/opt/azurite/node_modules/morgan/index.js:144:5)" 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set HTTP code: 400 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set HTTP status message: The specifed resource name contains invalid characters. 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=InvalidResourceName 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=974d1f1b-df04-4012-82b1-d944e14d7cef 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set content type: application/xml 2021-06-23T10:25:52.816Z 974d1f1b-df04-4012-82b1-d944e14d7cef error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n <Code>InvalidResourceName</Code>\n <Message>The specifed resource name contains invalid characters.\nRequestId:974d1f1b-df04-4012-82b1-d944e14d7cef\nTime:2021-06-23T10:25:52.816Z</Message>\n</Error>" 2021-06-23T10:25:52.817Z 974d1f1b-df04-4012-82b1-d944e14d7cef info: EndMiddleware: End response. TotalTimeInMS=1 StatusCode=400 StatusMessage=The specifed resource name contains invalid characters. Headers={"server":"Azurite-Blob/3.13.1","x-ms-error-code":"InvalidResourceName","x-ms-request-id":"974d1f1b-df04-4012-82b1-d944e14d7cef","content-type":"application/xml"}

@blueww
Copy link
Member

blueww commented Jun 24, 2021

@Carael

The failed upload Blob request Uri in the debug log doesn't contain the container name "foo":
RequestURL=http://localhost/devstoreaccount1/test.txt

From the Sample code test you have shared, as the Blobclient is created from container client BlobClient textFile = containerClient.GetBlobClient("test.txt");, the Uri should contain the container name. So the issue should not be reproed by the Sample code test you have shared, but by some other code.

Would you please add the container name to the blob Uri in you test, and see if it works?

@isidentical
Copy link

We also are affected by this (happens on 3.13), and we don't even send anything relevant with container names. This happens while azure python client performs a list_containers call, without any prefix etc. Here are the logs. @blueww

@isidentical
Copy link

Here is a short reproducer using the official azure-storage-blob client;

from azure.storage.blob import BlobServiceClient

CONNECTION_STRING = (
    "DefaultEndpointsProtocol=http;"
    "AccountName=devstoreaccount1;"
    "AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSR"
    "Z6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;"
    "BlobEndpoint=http://127.0.0.1:32780/devstoreaccount1;"
)

blob_service_client = BlobServiceClient.from_connection_string(CONNECTION_STRING)
print(list(blob_service_client.list_containers()))

The azurite instance is started through docker

@isidentical
Copy link

The docker logs show only this when I execute the script above;

172.18.0.1 - - [29/Jun/2021:08:40:55 +0000] "GET /devstoreaccount1/?comp=list&include= HTTP/1.1" 400 -

@isidentical
Copy link

It seems like a regression caused in 3.13, since the code above works perfectly fine with azurite 3.12.

@moraleslevi
Copy link

@blueww Sorry for the delay. Here is the debug log from Azurite when I attempt to list the containers in the emulator account via Azure Storage Explorer v1.19.1.

2021-06-29T14:08:46.811Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=undefined Blob=
2021-06-29T14:08:46.811Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de verbose: DispatchMiddleware: Dispatching request...
2021-06-29T14:08:46.816Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: DispatchMiddleware: Operation=Service_GetAccountInfo
2021-06-29T14:08:46.816Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-06-29T14:08:46.816Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: PublicAccessAuthenticator:validate() Start validation against public access.
2021-06-29T14:08:46.816Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de debug: PublicAccessAuthenticator:validate() Getting account properties...
2021-06-29T14:08:46.817Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: undefined, blob:
2021-06-29T14:08:46.817Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: PublicAccessAuthenticator:validate() Skip public access authentication. Container name is undefined.
2021-06-29T14:08:46.817Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-06-29T14:08:46.819Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:70af855d-3abc-4904-b5aa-b83531d391a6\nx-ms-date:Tue, 29 Jun 2021 14:08:46 GMT\nx-ms-version:2019-07-07\n/devstoreaccount1/devstoreaccount1\ncomp:properties\nrestype:account"
2021-06-29T14:08:46.821Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:IcQF9pr365wSZ9T6tYfetgTErbcXsrMpj8zKOw9ZssQ=
2021-06-29T14:08:46.822Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2021-06-29T14:08:46.822Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de verbose: DeserializerMiddleware: Start deserializing...
2021-06-29T14:08:46.825Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: HandlerMiddleware: DeserializedParameters={"restype":"account","comp":"properties","version":"2019-07-07"}
2021-06-29T14:08:46.826Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de verbose: SerializerMiddleware: Start serializing...
2021-06-29T14:08:46.827Z a1d2236c-5ab2-45ca-b9cf-4fffe68d27de info: EndMiddleware: End response. TotalTimeInMS=17 StatusCode=200 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.13.0","x-ms-client-request-id":"70af855d-3abc-4904-b5aa-b83531d391a6","x-ms-request-id":"a1d2236c-5ab2-45ca-b9cf-4fffe68d27de","x-ms-version":"2020-08-04","date":"Tue, 29 Jun 2021 14:08:46 GMT","x-ms-sku-name":"Standard_RAGRS","x-ms-account-kind":"StorageV2"}
2021-06-29T14:08:46.829Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1?restype=account&comp=properties RequestHeaders:{"x-ms-version":"2019-07-07","user-agent":"Microsoft Azure Storage Explorer, 1.19.1, darwin azsdk-js-storageblob/12.4.1 (NODE-VERSION v12.16.3; Darwin 20.5.0)","x-ms-client-request-id":"8c34ebeb-a0e5-4d01-9f2e-b5fbd38926b7","x-ms-date":"Tue, 29 Jun 2021 14:08:46 GMT","authorization":"SharedKey devstoreaccount1:DPp/59R95o04AbgSE9S5dcABu6b1Kg/NziP0ACr8eXQ=","cookie":"","accept":"*/*","host":"127.0.0.1:10000","connection":"keep-alive"} ClientIP=172.27.0.1 Protocol=http HTTPVersion=1.1
2021-06-29T14:08:46.829Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=undefined Blob=
2021-06-29T14:08:46.829Z 48fa3117-826e-4537-9e2a-8d5fae23953a verbose: DispatchMiddleware: Dispatching request...
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: DispatchMiddleware: Operation=Service_GetAccountInfo
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: PublicAccessAuthenticator:validate() Start validation against public access.
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a debug: PublicAccessAuthenticator:validate() Getting account properties...
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: undefined, blob:
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: PublicAccessAuthenticator:validate() Skip public access authentication. Container name is undefined.
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:8c34ebeb-a0e5-4d01-9f2e-b5fbd38926b7\nx-ms-date:Tue, 29 Jun 2021 14:08:46 GMT\nx-ms-version:2019-07-07\n/devstoreaccount1/devstoreaccount1\ncomp:properties\nrestype:account"
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:DPp/59R95o04AbgSE9S5dcABu6b1Kg/NziP0ACr8eXQ=
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a verbose: DeserializerMiddleware: Start deserializing...
2021-06-29T14:08:46.830Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: HandlerMiddleware: DeserializedParameters={"restype":"account","comp":"properties","version":"2019-07-07"}
2021-06-29T14:08:46.831Z 48fa3117-826e-4537-9e2a-8d5fae23953a verbose: SerializerMiddleware: Start serializing...
2021-06-29T14:08:46.831Z 48fa3117-826e-4537-9e2a-8d5fae23953a info: EndMiddleware: End response. TotalTimeInMS=2 StatusCode=200 StatusMessage=undefined Headers={"server":"Azurite-Blob/3.13.0","x-ms-client-request-id":"8c34ebeb-a0e5-4d01-9f2e-b5fbd38926b7","x-ms-request-id":"48fa3117-826e-4537-9e2a-8d5fae23953a","x-ms-version":"2020-08-04","date":"Tue, 29 Jun 2021 14:08:46 GMT","x-ms-sku-name":"Standard_RAGRS","x-ms-account-kind":"StorageV2"}
2021-06-29T14:08:48.453Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1?include=metadata&comp=list RequestHeaders:{"x-ms-version":"2019-07-07","user-agent":"Microsoft Azure Storage Explorer, 1.19.1, darwin azsdk-js-storageblob/12.4.1 (NODE-VERSION v12.16.3; Darwin 20.5.0)","x-ms-client-request-id":"f5c41eb1-aad5-4634-bfda-0bacbbfce7ec","x-ms-date":"Tue, 29 Jun 2021 14:08:48 GMT","authorization":"SharedKey devstoreaccount1:qElCvmDWxedVdpm/Pd0BTPQTIZzICenGEozbbJqbdKw=","cookie":"","accept":"*/*","host":"127.0.0.1:10000","connection":"keep-alive"} ClientIP=172.27.0.1 Protocol=http HTTPVersion=1.1
2021-06-29T14:08:48.453Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=undefined Blob=
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 verbose: DispatchMiddleware: Dispatching request...
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: DispatchMiddleware: Operation=Service_ListContainersSegment
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: PublicAccessAuthenticator:validate() Start validation against public access.
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 debug: PublicAccessAuthenticator:validate() Getting account properties...
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: undefined, blob:
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: PublicAccessAuthenticator:validate() Skip public access authentication. Container name is undefined.
2021-06-29T14:08:48.454Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-06-29T14:08:48.455Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:f5c41eb1-aad5-4634-bfda-0bacbbfce7ec\nx-ms-date:Tue, 29 Jun 2021 14:08:48 GMT\nx-ms-version:2019-07-07\n/devstoreaccount1/devstoreaccount1\ncomp:list\ninclude:metadata"
2021-06-29T14:08:48.455Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:qElCvmDWxedVdpm/Pd0BTPQTIZzICenGEozbbJqbdKw=
2021-06-29T14:08:48.455Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2021-06-29T14:08:48.455Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 verbose: DeserializerMiddleware: Start deserializing...
2021-06-29T14:08:48.455Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: HandlerMiddleware: DeserializedParameters={"options":{"include":"metadata","requestId":"f5c41eb1-aad5-4634-bfda-0bacbbfce7ec"},"comp":"list","version":"2019-07-07"}
2021-06-29T14:08:48.461Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 verbose: SerializerMiddleware: Start serializing...
2021-06-29T14:08:48.467Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 debug: Serializer: Raw response body string is <?xml version="1.0" encoding="UTF-8" standalone="yes"?><EnumerationResults ServiceEndpoint="http://127.0.0.1:10000/devstoreaccount1"><Prefix/><MaxResults>5000</MaxResults><Containers><Container><Name>assets</Name><Properties><Last-Modified>Mon, 20 Apr 2020 19:36:22 GMT</Last-Modified><Etag>"0x1F1C67717930060"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><PublicAccess>container</PublicAccess><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container><Container><Name>drivers</Name><Properties><Last-Modified>Tue, 20 Oct 2020 19:05:45 GMT</Last-Modified><Etag>"0x1C68A6980DDF7D0"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><PublicAccess>container</PublicAccess><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container><Container><Name>lxp</Name><Properties><Last-Modified>Mon, 20 Apr 2020 19:37:42 GMT</Last-Modified><Etag>"0x22F2546867CBD40"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><PublicAccess>container</PublicAccess><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container><Container><Name>sco</Name><Properties><Last-Modified>Mon, 20 Apr 2020 19:36:22 GMT</Last-Modified><Etag>"0x2232950E64283A0"</Etag><LeaseStatus>unlocked</LeaseStatus><LeaseState>available</LeaseState><HasImmutabilityPolicy>false</HasImmutabilityPolicy><HasLegalHold>false</HasLegalHold></Properties></Container></Containers><NextMarker/></EnumerationResults>
2021-06-29T14:08:48.467Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: Serializer: Start returning stream body.
2021-06-29T14:08:48.467Z e1f1cb37-d99b-4c1e-8896-5b9dc17cde27 info: EndMiddleware: End response. TotalTimeInMS=14 StatusCode=200 StatusMessage=OK Headers={"server":"Azurite-Blob/3.13.0","x-ms-client-request-id":"f5c41eb1-aad5-4634-bfda-0bacbbfce7ec","x-ms-request-id":"e1f1cb37-d99b-4c1e-8896-5b9dc17cde27","x-ms-version":"2020-08-04","content-type":"application/xml"}
2021-06-29T14:08:48.479Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1/%24logs?restype=container RequestHeaders:{"x-ms-version":"2019-07-07","user-agent":"Microsoft Azure Storage Explorer, 1.19.1, darwin azsdk-js-storageblob/12.4.1 (NODE-VERSION v12.16.3; Darwin 20.5.0)","x-ms-client-request-id":"6f39292a-8554-4268-9244-34d9d26ba89a","x-ms-date":"Tue, 29 Jun 2021 14:08:48 GMT","authorization":"SharedKey devstoreaccount1:UQdHrMmni9cJ1TbC4dd3eYWewNQY7zm6IWGCXs3IxMs=","cookie":"","accept":"*/*","host":"127.0.0.1:10000","connection":"keep-alive"} ClientIP=172.27.0.1 Protocol=http HTTPVersion=1.1
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Received a MiddlewareError, fill error information to HTTP response
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: ErrorName=StorageError ErrorMessage=The specifed resource name contains invalid characters.  ErrorHTTPStatusCode=400 ErrorHTTPStatusMessage=The specifed resource name contains invalid characters. ErrorHTTPHeaders={"x-ms-error-code":"InvalidResourceName","x-ms-request-id":"49ceeb42-1ff7-4461-8e95-3ebeb6816716"} ErrorHTTPBody="<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidResourceName</Code>\n  <Message>The specifed resource name contains invalid characters.\nRequestId:49ceeb42-1ff7-4461-8e95-3ebeb6816716\nTime:2021-06-29T14:08:48.480Z</Message>\n</Error>" ErrorStack="StorageError: The specifed resource name contains invalid characters.\n    at Function.getInvalidResourceName (/opt/azurite/dist/src/blob/errors/StorageErrorFactory.js:228:16)\n    at Object.validateContainerName (/opt/azurite/dist/src/blob/utils/utils.js:109:45)\n    at blobStorageContextMiddleware (/opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:68:17)\n    at /opt/azurite/dist/src/blob/middlewares/blobStorageContext.middleware.js:14:16\n    at Layer.handle [as handle_request] (/opt/azurite/node_modules/express/lib/router/layer.js:95:5)\n    at trim_prefix (/opt/azurite/node_modules/express/lib/router/index.js:317:13)\n    at /opt/azurite/node_modules/express/lib/router/index.js:284:7\n    at Function.process_params (/opt/azurite/node_modules/express/lib/router/index.js:335:12)\n    at next (/opt/azurite/node_modules/express/lib/router/index.js:275:10)\n    at expressInit (/opt/azurite/node_modules/express/lib/middleware/init.js:40:5)"
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set HTTP code: 400
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set HTTP status message: The specifed resource name contains invalid characters.
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set HTTP Header: x-ms-error-code=InvalidResourceName
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set HTTP Header: x-ms-request-id=49ceeb42-1ff7-4461-8e95-3ebeb6816716
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set content type: application/xml
2021-06-29T14:08:48.482Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 error: ErrorMiddleware: Set HTTP body: "<?xml version=\"1.0\" encoding=\"UTF-8\" standalone=\"yes\"?>\n<Error>\n  <Code>InvalidResourceName</Code>\n  <Message>The specifed resource name contains invalid characters.\nRequestId:49ceeb42-1ff7-4461-8e95-3ebeb6816716\nTime:2021-06-29T14:08:48.480Z</Message>\n</Error>"
2021-06-29T14:08:48.483Z 49ceeb42-1ff7-4461-8e95-3ebeb6816716 info: EndMiddleware: End response. TotalTimeInMS=4 StatusCode=400 StatusMessage=The specifed resource name contains invalid characters. Headers={"server":"Azurite-Blob/3.13.0","x-ms-error-code":"InvalidResourceName","x-ms-request-id":"49ceeb42-1ff7-4461-8e95-3ebeb6816716","content-type":"application/xml"}
2021-06-29T14:08:48.615Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobStorageContextMiddleware: RequestMethod=GET RequestURL=http://127.0.0.1/devstoreaccount1?restype=service&comp=properties RequestHeaders:{"x-ms-version":"2019-07-07","user-agent":"Microsoft Azure Storage Explorer, 1.19.1, darwin azsdk-js-storageblob/12.4.1 (NODE-VERSION v12.16.3; Darwin 20.5.0)","x-ms-client-request-id":"e6de15b1-7047-4169-b6e0-8d185f4fa4eb","x-ms-date":"Tue, 29 Jun 2021 14:08:48 GMT","authorization":"SharedKey devstoreaccount1:jrrk7Y80YDO7vCs/zhAtjcjoMKpRD1jFdtmuN7mfQpo=","cookie":"","accept":"*/*","host":"127.0.0.1:10000","connection":"keep-alive"} ClientIP=172.27.0.1 Protocol=http HTTPVersion=1.1
2021-06-29T14:08:48.615Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobStorageContextMiddleware: Account=devstoreaccount1 Container=undefined Blob=
2021-06-29T14:08:48.615Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb verbose: DispatchMiddleware: Dispatching request...
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: DispatchMiddleware: Operation=Service_GetProperties
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb verbose: AuthenticationMiddlewareFactory:createAuthenticationMiddleware() Validating authentications.
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: PublicAccessAuthenticator:validate() Start validation against public access.
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb debug: PublicAccessAuthenticator:validate() Getting account properties...
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb debug: PublicAccessAuthenticator:validate() Retrieved account name from context: devstoreaccount1, container: undefined, blob:
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: PublicAccessAuthenticator:validate() Skip public access authentication. Container name is undefined.
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobSharedKeyAuthenticator:validate() Start validation against account shared key authentication.
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobSharedKeyAuthenticator:validate() [STRING TO SIGN]:"GET\n\n\n\n\n\n\n\n\n\n\n\nx-ms-client-request-id:e6de15b1-7047-4169-b6e0-8d185f4fa4eb\nx-ms-date:Tue, 29 Jun 2021 14:08:48 GMT\nx-ms-version:2019-07-07\n/devstoreaccount1/devstoreaccount1\ncomp:properties\nrestype:service"
2021-06-29T14:08:48.616Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobSharedKeyAuthenticator:validate() Calculated authentication header based on key1: SharedKey devstoreaccount1:jrrk7Y80YDO7vCs/zhAtjcjoMKpRD1jFdtmuN7mfQpo=
2021-06-29T14:08:48.617Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: BlobSharedKeyAuthenticator:validate() Signature 1 matched.
2021-06-29T14:08:48.617Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb verbose: DeserializerMiddleware: Start deserializing...
2021-06-29T14:08:48.617Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: HandlerMiddleware: DeserializedParameters={"options":{"requestId":"e6de15b1-7047-4169-b6e0-8d185f4fa4eb"},"restype":"service","comp":"properties","version":"2019-07-07"}
2021-06-29T14:08:48.618Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb verbose: SerializerMiddleware: Start serializing...
2021-06-29T14:08:48.619Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb debug: Serializer: Raw response body string is <?xml version="1.0" encoding="UTF-8" standalone="yes"?><StorageServiceProperties><Logging><Version>1.0</Version><Delete>true</Delete><Read>true</Read><Write>true</Write><RetentionPolicy><Enabled>false</Enabled></RetentionPolicy></Logging><HourMetrics><Version>1.0</Version><Enabled>false</Enabled><RetentionPolicy><Enabled>false</Enabled></RetentionPolicy></HourMetrics><MinuteMetrics><Version>1.0</Version><Enabled>false</Enabled><RetentionPolicy><Enabled>false</Enabled></RetentionPolicy></MinuteMetrics><Cors/><DefaultServiceVersion>2020-08-04</DefaultServiceVersion><StaticWebsite><Enabled>false</Enabled></StaticWebsite></StorageServiceProperties>
2021-06-29T14:08:48.619Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: Serializer: Start returning stream body.
2021-06-29T14:08:48.619Z 6ca00fea-9814-44f9-8dc9-776d239b4ccb info: EndMiddleware: End response. TotalTimeInMS=4 StatusCode=200 StatusMessage=OK Headers={"server":"Azurite-Blob/3.13.0","x-ms-client-request-id":"e6de15b1-7047-4169-b6e0-8d185f4fa4eb","x-ms-request-id":"6ca00fea-9814-44f9-8dc9-776d239b4ccb","x-ms-version":"2020-08-04","content-type":"application/xml"}```

@blueww
Copy link
Member

blueww commented Jun 30, 2021

@moraleslevi, @isidentical

Would you please confirm, do you use Azurite 3.13.0 or 3.13.1?
If you use 3.13.0, would you please upgrade to 3.13.1 and try again?

@isidentical
For list container fail with "include=" in Uri, but include nothing, the is a known issue of SDK not aligned with rest API. (See details in #813)
To workaround it, would you please include metadata in list container?

@isidentical
Copy link

To workaround it, would you please include metadata in list container?

Same error.

@blueww
Copy link
Member

blueww commented Jun 30, 2021

@isidentical

Would you please confirm you are using 3.13.1.
And if so, please give the Azurite debug log for the failed request.

@isidentical
Copy link

The last release I can see is 3.13.0, https://github.com/Azure/Azurite/releases

@blueww
Copy link
Member

blueww commented Jun 30, 2021

@isidentical

The latest release is 3.13.1 , see https://github.com/Azure/Azurite/blob/master/README.md
image

We still not tag it (so latest tag is still 3.13.0), but 3.13.1 has already released in all channels (npm, docker, VS Code extension).
Please upgrade to 3.13.1 and try again.

@isidentical
Copy link

Yes, 3.13.1 seem to work with docker.

@moraleslevi
Copy link

I can also confirm that updating the docker image to v3.13.1 resolved the issue for me with Azure Storage Explorer.

@blueww
Copy link
Member

blueww commented Jul 1, 2021

@moraleslevi, @isidentical
Thanks for the confirmation!
And good to know your issue is resolved after upgrade to latest release.

@blueww
Copy link
Member

blueww commented Jul 1, 2021

@Carael

Would you please help to update if you still see the error after add container name in the Uri when put blob? (like the Sample code test do)

@blueww
Copy link
Member

blueww commented Jul 14, 2021

@Carael

I will close the issue as not hear from you for a long time.
Feel free to contact us again if need any further assistance on Azurite.

@blueww blueww closed this as completed Jul 14, 2021
@narcis-ro
Copy link

@blueww This still reproduces even today.
It seems the issue reproduces when I use localhost instead of 127.0.0.1 in the connection string. This happens when azurite is hosted in docker via test-containers (using a mapperPort:10000). 140% the container "download" is created beforehand.
Azure Storage Explorer works, because it uses 127.0.0.1, I saw it in the logs..

public BlobServiceClient GetBlobServiceClient() =>  new(
    $"DefaultEndpointsProtocol=http;AccountName=devstoreaccount1;AccountKey=Eby8vdM02xNOcqFlqUwJPLlmEtlCDXJ1OUzFT50uSRZ6IFsuFq2UVErCz4I6tq/K1SZFPTOtr/KBHBeksoGMGw==;BlobEndpoint=http://localhost:{GetAzuritePort()}/devstoreaccount1;");

image

Hope this helps someone else

@blueww
Copy link
Member

blueww commented May 24, 2024

@narcis-ro

From the debug log picture in your above comment, we can see the issue is caused by the request send from you client to Azurite is not correct, it missed container name in the Uri.
As the request is not correct, so Azurite can't handle it correctly.

This should be a client issue. It looks Client code/sdk has come issue that send request with wrong uri which missed container name.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
blob-storage question Further information is requested
Projects
None yet
Development

No branches or pull requests

5 participants