delete_dir bug

[pesmeriz]

          I just tried through `azure.identity`and `azure.storage.filedatalake` to check if there were any problems on the permissions or whatever, and I managed to delete the file.

from azure.identity import ClientSecretCredential
from azure.storage.filedatalake import DataLakeServiceClient
credentials = {
    "AZURE_STORAGE_CLIENT_ID": config("AZURE_STORAGE_CLIENT_ID"),
    "AZURE_STORAGE_CLIENT_SECRET": config("AZURE_STORAGE_CLIENT_SECRET"),
    "AZURE_STORAGE_TENANT_ID": config("AZURE_STORAGE_TENANT_ID"),
}
path = f'abfss://{config("AZURE_CONTAINER_NAME")}@{config("AZURE_ACCOUNT_NAME")}.dfs.core.windows.net/{path}'
dataset.write_delta(
    target=path,
    delta_write_options={"partition_by": "date_part"},
    storage_options=credentials
)
delete_credentials = ClientSecretCredential(
    client_id=config("AZURE_STORAGE_CLIENT_ID"),
    client_secret=config("AZURE_STORAGE_CLIENT_SECRET"),
    tenant_id=config("AZURE_STORAGE_TENANT_ID"),
)

service_client = DataLakeServiceClient(account_url = f"https://{config('AZURE_ACCOUNT_NAME')}.dfs.core.windows.net/", credential = delete_credentials)
file_system_client = service_client.get_file_system_client(config('AZURE_CONTAINER_NAME'))
directory_client = file_system_client.get_directory_client(path)
directory_client.delete_directory()

I'll stick to this by now but I'd really like to know how to delete the delta table (dir and contents) through deltalake.

Originally posted by @pesmeriz in #2703 (comment)

[rtyler]

This is a configuration issue or bug with the Delta/Spark library and tooling. The deltalake Python package doesn't have these APIs :)

[pesmeriz]

I'm confused. These libraries I am mentioning is the way I got around the original issue, which is in #2703 (comment)

This was just to check that I had access and therefore privileges to delete the table I still can't do so via deltalake.

[pesmeriz]

Can someone please help me on this?