Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

azure-storage-file-datalake DataLakeServiceClient.get_paths() raises ResourceNotFoundError #20773

Closed
pdworzynski opened this issue Sep 21, 2021 · 4 comments
Assignees
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)

Comments

@pdworzynski
Copy link

  • Package Name: azure-storage-file-datalake
  • Package Version: 12.5.0
  • Operating System: Ubuntu Linux
  • Python Version: 3.8.8

Describe the bug
DataLakeServiceClient class method get_paths() returns ResourceNotFoundError: Operation returned an invalid status 'The specified container does not exist.'

To Reproduce
Steps to reproduce the behavior:

service_client = DataLakeServiceClient(account_url=f"https://{azure_storage_account_name}.blob.core.windows.net/", credential=sas_token)
file_system_client = service_client.create_file_system("test-file-system")
print(f"{file_system_client.exists()=}") # prints "file_system_client.exists()=True"
print(next(file_system_client.get_paths()))

Throws ResourceNotFoundError: Operation returned an invalid status 'The specified container does not exist.' error. The error also seems to occur for non-empty file systems.

Expected behavior
The function should return an empty iterator.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

@ghost ghost added needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. customer-reported Issues that are reported by GitHub users external to the Azure organization. question The issue doesn't require a change to the product in order to be resolved. Most issues start as that labels Sep 21, 2021
@xiangyan99 xiangyan99 added bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files) and removed question The issue doesn't require a change to the product in order to be resolved. Most issues start as that needs-triage Workflow: This is a new issue that needs to be triaged to the appropriate team. labels Sep 21, 2021
@ghost
Copy link

ghost commented Sep 21, 2021

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @xgithubtriage.

Issue Details
  • Package Name: azure-storage-file-datalake
  • Package Version: 12.5.0
  • Operating System: Ubuntu Linux
  • Python Version: 3.8.8

Describe the bug
DataLakeServiceClient class method get_paths() returns ResourceNotFoundError: Operation returned an invalid status 'The specified container does not exist.'

To Reproduce
Steps to reproduce the behavior:

service_client = DataLakeServiceClient(account_url=f"https://{azure_storage_account_name}.blob.core.windows.net/", credential=sas_token)
file_system_client = service_client.create_file_system("test-file-system")
print(f"{file_system_client.exists()=}") # prints "file_system_client.exists()=True"
print(next(file_system_client.get_paths()))

Throws ResourceNotFoundError: Operation returned an invalid status 'The specified container does not exist.' error. The error also seems to occur for non-empty file systems.

Expected behavior
The function should return an empty iterator.

Screenshots
If applicable, add screenshots to help explain your problem.

Additional context
Add any other context about the problem here.

Author: pdworzynski
Assignees: -
Labels:

bug, Storage, Service Attention, Client, customer-reported

Milestone: -

@ghost ghost added the needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team label Sep 21, 2021
@xiangyan99
Copy link
Member

Thanks for the feedback, we’ll investigate asap.

@xiafu-msft
Copy link
Contributor

Hi @pdworzynski

Thanks for reaching out. If you are using an HNS enabled account, can you please change the url suffix to dfs.core.windows.net and try again?

Let me know if it doesn't work.

@pdworzynski
Copy link
Author

Hi @xiafu-msft

This solved the problem.

Thank you!

azure-sdk pushed a commit to azure-sdk/azure-sdk-for-python that referenced this issue Sep 22, 2022
Merge Microsoft.app 2022 06 01 preview to main (Azure#20787)

* Adds base for updating Microsoft.App from version stable/2022-03-01 to version 2022-05-01

* Updates readme

* Updates API version in new specs and examples

* Add app diag to 2022-05-01 (Azure#18678)

* Add container apps diagnostics

* Fix validation errors

* Move Microsoft.App diag APIs to 2022-05-01

* Fix validation errors

* Fix validation errors

* Fix validation errors 3

* Change operationIds

Co-authored-by: Michimune Kohno <[email protected]>

* Fix definition for customhostnameesult (Azure#19014)

* fix

* update

* more fix

* Cherry pick for the parameter name fixes for the 2022-03-01 and making the same fixes in 2022-05-01 (Azure#19048)

* Correct Probe enums (Azure#18839)

* fix probe enums

* Fix revision mode enum

* Fix patch example

* Fix the casing of enums in the latest api version

* Change the parameter names to unblock Terraform release (Azure#19005)

* contianerapps: fixing swagger inconsistencies in 2022-01-01-preview

* containerapps: fixing inconsistencies in the 2022-03-01 swagger

* containerapps: updating the examples for 2022-01-01-preview

* containerapps: updating the examples to account for the updated parameter names

* containerapps: fixing linting

* fix probe enums

* Fix revision mode enum

* Fix the policheck error

* Revert "containerapps: fixing linting"

This reverts commit 3c4872f4a43fbe1a285830461f48d8c0a5ffeee8.

* Revert "containerapps: updating the examples for 2022-01-01-preview"

This reverts commit 9b81fbd65652d558c0296d72a63ca1a7c4850e3c.

* Revert "contianerapps: fixing swagger inconsistencies in 2022-01-01-preview"

This reverts commit 553fc551b3239598157238ed6246085748421b87.

* Undo changes to 2022-01-01-preview

* Fix one issue

* fix remaining inconsistency

* fix authConfig and sourceControl names

Co-authored-by: tombuildsstuff <[email protected]>
Co-authored-by: Mike Vu <[email protected]>

* Change param names

* Fix examples

* Fix CI errors

Co-authored-by: Mike Vu <[email protected]>
Co-authored-by: tombuildsstuff <[email protected]>

* Fix description in all API versions (Azure#19313)

* Fix description for ManagedEnvironment.Internal property (Azure#19307)

* Fix descriptions in all api versions

* Add traffic info to revisions api (Azure#19054)

* Add traffic info to a revision api

* Add deprecated prop

* Prettier fixes

* 1pdate based on new proposal

* Rename

* Change names

* minor fix

* Add httpReadBufferSize to dapr config in Microsoft.App-2022-05-01 (Azure#19360)

* Add httpReadBufferSize to dapr config

* remove nullable

* Adds base for updating Microsoft.App from version stable/2022-05-01 to version 2022-06-01-preview

* Updates readme

* Updates API version in new specs and examples

* Add new resources for 2022-06-01-preview (Azure#19494)

* update

* update

* update

* fix

* update

* update

* update

* update

* fix

* update

* update

* Enable Dapr AI Connection string for Microsoft.App connected environment (Azure#19974)

* Enable Dapr AI Connection string for Microsoft.App connected environment

* Fix dapr ai connection string

* remove read

* remove read permission of dapr ai connection string and instrument key for managed environment

* fixes for comments

* update (Azure#20013)

* Fixdescriptions on microsoft.app 2022 06 01 preview (Azure#20133)

* Fix casing for detector properties

* Fix descriptions

* Fix descriptions and detectorProperties in 2022-06-01-preview

* Added `secretsStoreComponent` in 2022-06-01 spec (Azure#20044)

* Added `secretsStoreComponent` in 2022-06-01 spec

Signed-off-by: ItalyPaleAle <[email protected]>

* Fixed typos and added example of name/key

Signed-off-by: ItalyPaleAle <[email protected]>

* Add custom domain configuration for managed environment (Azure#20149)

Co-authored-by: Mengjun Qin <[email protected]>

* Add additional Dapr options (2022-06-01-preview) (Azure#19886)

* Add additional Dapr options

* Update examples for 2022-06-01-preview

* force ci

* force ci

* force ci

* adding ip-restriction feat (Azure#19841)

* adding ip-restriction feat

* remove from stable, add to preview

* fix lintDiff error

* fix modelvalidation error

* fix formatting changes

* rename names, add note to all allow or all deny

* fix spell check error

* update name and examples

* rename `ipAddress` to `ipAddressRange`

* making small change to re-run build pipeline

* trigger GitHub actions

* resolve comments

Co-authored-by: Taher Darolywala <[email protected]>

* Remove Traffic Labels from revisions because they haven't been impelemented (Azure#20261)

* Revert "Add traffic info to revisions api (Azure#19054)"

This reverts commit 056cc51226bf2eef3d3a7e7bdca875a4a571026b.

* Remove traffic labels from revisions

* Add property for tcp apps (Azure#20193)

* Add property for tcp apps

* Add scale

* Fix

* Fix

* Add Init Containers to common definitions (Azure#19855)

* add init containers to common definitions

* move changes to 06-01 preview

* remove repeated container definition

* fix allof syntax

* fix anonymous container definition

* fix lintdiff errors

* fix container allof syntax

* add more init examples

* split baseContainer definition

* remove description

Co-authored-by: p-bouchon <[email protected]>

* Update container app provisioningstate (Azure#20411)

* update

* update

* add property: maxInactiveRevisions (Azure#20334)

* add property: maxInactiveRevisions

* update

* update

* Add premium sku related properties for managed environment (Azure#19770)

* Add premium sku related properties for managed environment

* Move sku to base property

* Add workloadProfileType

* Examples for managed environment

* Add container app example

* Fix workloadProfiles

* Add custom word

* Fix WorkloadProfileType references

* prettier fixes

* Fix workload profile

* Fix typo

* remove readonly property for workload profile

* Add object type

* nit: Fix description location

* Fix XmsIdentifierValidation

* Remove spaces from enum values

* Remove unwanted change

* Add aka ms url

* Update examples for workload profile type

* Fix rebase issue

* Fix json

* Update description and move location

* Remove provisioning state for node pool

* sku capacity explanation

* Remove capacity attribute

* Remove capacity attribute from required

* Mark sku property as required

* Add x-ms-identifiers

* Rename mimum and maximum

* Rename skutype to skuName

* Make sku property as optional

* Make workload profile type as string

* fix go SDK automation config

Co-authored-by: Chenjie Shi <[email protected]>

* Fix source control status code in 2022-06-01-preview. (Azure#20429)

* Remove 20220501 because we will not support it in server side (Azure#20427)

* remove

* update

* Added support for user defined routing (Azure#20464)

* Added support for user defined routing

* Added examples

* fixed JSON syntax

* syntax fix

* fixed vnetenvironment syntax

* fixed camel case

* case fix

* enum syntax

* fixed comma cases ManagedEnvironmentOutBoundType

* fixing casing

* Some more APIs (Azure#20448)

* new APIs to 06-01

* test fix

* more fixes

* add last active time for revision (Azure#20658)

* Add custom domain for connected env (Azure#20731)

* add custom domain

* fhcekout

* billingMeters - GET endpoint (Azure#20485)

* Swagger documentation for /locations/{location}/billingMeters - GET endpoint

* Add example, fix validation errors

* Fix ARM validation errors, code review comments

* Fix response validation

* add proxy resource properties

* Response formatting

* add default error response

* ARM Review feedback + small naming change for a property

* Change GB to GiB

* Fix 2022-06-01-preview branch merge conflicts (Azure#20773)

* Reset

* more fixes

* More fixes

* another conflict fix

* Fix old preview version

* fix 2022-03-01

* supress lint

Signed-off-by: ItalyPaleAle <[email protected]>
Co-authored-by: Ruslan Yakushev <[email protected]>
Co-authored-by: michimune <[email protected]>
Co-authored-by: Michimune Kohno <[email protected]>
Co-authored-by: Mike Vu <[email protected]>
Co-authored-by: tombuildsstuff <[email protected]>
Co-authored-by: Ahmed ElSayed <[email protected]>
Co-authored-by: LaylaLiu-gmail <[email protected]>
Co-authored-by: Alessandro (Ale) Segala <[email protected]>
Co-authored-by: Mengjun Qin <[email protected]>
Co-authored-by: Mengjun Qin <[email protected]>
Co-authored-by: Taher Daroly <[email protected]>
Co-authored-by: Taher Darolywala <[email protected]>
Co-authored-by: zhenqxuMSFT <[email protected]>
Co-authored-by: p-bouchon <[email protected]>
Co-authored-by: p-bouchon <[email protected]>
Co-authored-by: njucz <[email protected]>
Co-authored-by: JJ <[email protected]>
Co-authored-by: Chenjie Shi <[email protected]>
Co-authored-by: yalixiang <[email protected]>
Co-authored-by: Mahmoud Desokey <[email protected]>
Co-authored-by: Xingjian Wang <[email protected]>
Co-authored-by: trajkobal <[email protected]>
azure-sdk pushed a commit to azure-sdk/azure-sdk-for-python that referenced this issue Sep 28, 2022
Availableworkloadprofiles - GET endpoint (Azure#20650)

* Adds base for updating Microsoft.App from version stable/2022-03-01 to version 2022-05-01

* Updates readme

* Updates API version in new specs and examples

* Add app diag to 2022-05-01 (Azure#18678)

* Add container apps diagnostics

* Fix validation errors

* Move Microsoft.App diag APIs to 2022-05-01

* Fix validation errors

* Fix validation errors

* Fix validation errors 3

* Change operationIds

Co-authored-by: Michimune Kohno <[email protected]>

* Fix definition for customhostnameesult (Azure#19014)

* fix

* update

* more fix

* Cherry pick for the parameter name fixes for the 2022-03-01 and making the same fixes in 2022-05-01 (Azure#19048)

* Correct Probe enums (Azure#18839)

* fix probe enums

* Fix revision mode enum

* Fix patch example

* Fix the casing of enums in the latest api version

* Change the parameter names to unblock Terraform release (Azure#19005)

* contianerapps: fixing swagger inconsistencies in 2022-01-01-preview

* containerapps: fixing inconsistencies in the 2022-03-01 swagger

* containerapps: updating the examples for 2022-01-01-preview

* containerapps: updating the examples to account for the updated parameter names

* containerapps: fixing linting

* fix probe enums

* Fix revision mode enum

* Fix the policheck error

* Revert "containerapps: fixing linting"

This reverts commit 3c4872f4a43fbe1a285830461f48d8c0a5ffeee8.

* Revert "containerapps: updating the examples for 2022-01-01-preview"

This reverts commit 9b81fbd65652d558c0296d72a63ca1a7c4850e3c.

* Revert "contianerapps: fixing swagger inconsistencies in 2022-01-01-preview"

This reverts commit 553fc551b3239598157238ed6246085748421b87.

* Undo changes to 2022-01-01-preview

* Fix one issue

* fix remaining inconsistency

* fix authConfig and sourceControl names

Co-authored-by: tombuildsstuff <[email protected]>
Co-authored-by: Mike Vu <[email protected]>

* Change param names

* Fix examples

* Fix CI errors

Co-authored-by: Mike Vu <[email protected]>
Co-authored-by: tombuildsstuff <[email protected]>

* Fix description in all API versions (Azure#19313)

* Fix description for ManagedEnvironment.Internal property (Azure#19307)

* Fix descriptions in all api versions

* Add traffic info to revisions api (Azure#19054)

* Add traffic info to a revision api

* Add deprecated prop

* Prettier fixes

* 1pdate based on new proposal

* Rename

* Change names

* minor fix

* Add httpReadBufferSize to dapr config in Microsoft.App-2022-05-01 (Azure#19360)

* Add httpReadBufferSize to dapr config

* remove nullable

* Adds base for updating Microsoft.App from version stable/2022-05-01 to version 2022-06-01-preview

* Updates readme

* Updates API version in new specs and examples

* Add new resources for 2022-06-01-preview (Azure#19494)

* update

* update

* update

* fix

* update

* update

* update

* update

* fix

* update

* update

* Enable Dapr AI Connection string for Microsoft.App connected environment (Azure#19974)

* Enable Dapr AI Connection string for Microsoft.App connected environment

* Fix dapr ai connection string

* remove read

* remove read permission of dapr ai connection string and instrument key for managed environment

* fixes for comments

* update (Azure#20013)

* Fixdescriptions on microsoft.app 2022 06 01 preview (Azure#20133)

* Fix casing for detector properties

* Fix descriptions

* Fix descriptions and detectorProperties in 2022-06-01-preview

* Added `secretsStoreComponent` in 2022-06-01 spec (Azure#20044)

* Added `secretsStoreComponent` in 2022-06-01 spec

Signed-off-by: ItalyPaleAle <[email protected]>

* Fixed typos and added example of name/key

Signed-off-by: ItalyPaleAle <[email protected]>

* Add custom domain configuration for managed environment (Azure#20149)

Co-authored-by: Mengjun Qin <[email protected]>

* Add additional Dapr options (2022-06-01-preview) (Azure#19886)

* Add additional Dapr options

* Update examples for 2022-06-01-preview

* force ci

* force ci

* force ci

* adding ip-restriction feat (Azure#19841)

* adding ip-restriction feat

* remove from stable, add to preview

* fix lintDiff error

* fix modelvalidation error

* fix formatting changes

* rename names, add note to all allow or all deny

* fix spell check error

* update name and examples

* rename `ipAddress` to `ipAddressRange`

* making small change to re-run build pipeline

* trigger GitHub actions

* resolve comments

Co-authored-by: Taher Darolywala <[email protected]>

* Remove Traffic Labels from revisions because they haven't been impelemented (Azure#20261)

* Revert "Add traffic info to revisions api (Azure#19054)"

This reverts commit 056cc51226bf2eef3d3a7e7bdca875a4a571026b.

* Remove traffic labels from revisions

* Add property for tcp apps (Azure#20193)

* Add property for tcp apps

* Add scale

* Fix

* Fix

* Add Init Containers to common definitions (Azure#19855)

* add init containers to common definitions

* move changes to 06-01 preview

* remove repeated container definition

* fix allof syntax

* fix anonymous container definition

* fix lintdiff errors

* fix container allof syntax

* add more init examples

* split baseContainer definition

* remove description

Co-authored-by: p-bouchon <[email protected]>

* Update container app provisioningstate (Azure#20411)

* update

* update

* add property: maxInactiveRevisions (Azure#20334)

* add property: maxInactiveRevisions

* update

* update

* Add premium sku related properties for managed environment (Azure#19770)

* Add premium sku related properties for managed environment

* Move sku to base property

* Add workloadProfileType

* Examples for managed environment

* Add container app example

* Fix workloadProfiles

* Add custom word

* Fix WorkloadProfileType references

* prettier fixes

* Fix workload profile

* Fix typo

* remove readonly property for workload profile

* Add object type

* nit: Fix description location

* Fix XmsIdentifierValidation

* Remove spaces from enum values

* Remove unwanted change

* Add aka ms url

* Update examples for workload profile type

* Fix rebase issue

* Fix json

* Update description and move location

* Remove provisioning state for node pool

* sku capacity explanation

* Remove capacity attribute

* Remove capacity attribute from required

* Mark sku property as required

* Add x-ms-identifiers

* Rename mimum and maximum

* Rename skutype to skuName

* Make sku property as optional

* Make workload profile type as string

* fix go SDK automation config

Co-authored-by: Chenjie Shi <[email protected]>

* Fix source control status code in 2022-06-01-preview. (Azure#20429)

* Swagger documentation for /locations/{location}/billingMeters - GET endpoint

* Add example, fix validation errors

* Fix ARM validation errors, code review comments

* Fix response validation

* Remove 20220501 because we will not support it in server side (Azure#20427)

* remove

* update

* add proxy resource properties

* Response formatting

* Added support for user defined routing (Azure#20464)

* Added support for user defined routing

* Added examples

* fixed JSON syntax

* syntax fix

* fixed vnetenvironment syntax

* fixed camel case

* case fix

* enum syntax

* fixed comma cases ManagedEnvironmentOutBoundType

* fixing casing

* add default error response

* Add available workprofiles -GET endpoint

* Remove duplicate schema name WorkloadProfile

* Change memoryGiB to memoryGB as suggested in code review

* Add x-ms-pageable - nextLink support to not introduce breaking changes in the future

* rename to memory in GiB because our case is Gibibytes base 1024

* change friendlyName to displayName

* Some more APIs (Azure#20448)

* new APIs to 06-01

* test fix

* more fixes

* Add new property to the response object

* add last active time for revision (Azure#20658)

* Add custom domain for connected env (Azure#20731)

* add custom domain

* fhcekout

* billingMeters - GET endpoint (Azure#20485)

* Swagger documentation for /locations/{location}/billingMeters - GET endpoint

* Add example, fix validation errors

* Fix ARM validation errors, code review comments

* Fix response validation

* add proxy resource properties

* Response formatting

* add default error response

* ARM Review feedback + small naming change for a property

* Change GB to GiB

* Change the boolean value for default to enum

* change error response to v3/ErrorResponse

* Fix 2022-06-01-preview branch merge conflicts (Azure#20773)

* Reset

* more fixes

* More fixes

* another conflict fix

* Fix old preview version

* fix 2022-03-01

Signed-off-by: ItalyPaleAle <[email protected]>
Co-authored-by: Ruslan Yakushev <[email protected]>
Co-authored-by: michimune <[email protected]>
Co-authored-by: Michimune Kohno <[email protected]>
Co-authored-by: Zunli Hu <[email protected]>
Co-authored-by: Mike Vu <[email protected]>
Co-authored-by: tombuildsstuff <[email protected]>
Co-authored-by: Ahmed ElSayed <[email protected]>
Co-authored-by: LaylaLiu-gmail <[email protected]>
Co-authored-by: Alessandro (Ale) Segala <[email protected]>
Co-authored-by: Mengjun Qin <[email protected]>
Co-authored-by: Mengjun Qin <[email protected]>
Co-authored-by: Taher Daroly <[email protected]>
Co-authored-by: Taher Darolywala <[email protected]>
Co-authored-by: zhenqxuMSFT <[email protected]>
Co-authored-by: p-bouchon <[email protected]>
Co-authored-by: p-bouchon <[email protected]>
Co-authored-by: njucz <[email protected]>
Co-authored-by: JJ <[email protected]>
Co-authored-by: Chenjie Shi <[email protected]>
Co-authored-by: yalixiang <[email protected]>
Co-authored-by: Mahmoud Desokey <[email protected]>
Co-authored-by: Xingjian Wang <[email protected]>
@github-actions github-actions bot locked and limited conversation to collaborators Apr 11, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
bug This issue requires a change to an existing behavior in the product in order to be resolved. Client This issue points to a problem in the data-plane of the library. customer-reported Issues that are reported by GitHub users external to the Azure organization. needs-team-attention Workflow: This issue needs attention from Azure service team or SDK team Service Attention Workflow: This issue is responsible by Azure service team. Storage Storage Service (Queues, Blobs, Files)
Projects
None yet
Development

No branches or pull requests

3 participants