Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for WASB to native Azure file system implementation #23535

Merged
merged 1 commit into from
Sep 23, 2024

Conversation

dain
Copy link
Member

@dain dain commented Sep 23, 2024

Description

Add support for WASB to native Azure file system implementation. WASB urls can be supported by simply modifying the Azure location parsing code as the underlying Azure storage APIs are set by the storage account type. The main difference to a ABFS url is the scheme is wasb and the blob endpoint is referenced instead of dfs.

Fixes #23511

Release notes

( ) This is not user-visible or is docs only, and no release notes are required.
( ) Release notes are required. Please propose a release note for me.
(x) Release notes are required, with the following suggested text:

# Section
* Add support for WASB to native Azure file system implementation. ({issue}`23511`)

@dain dain requested a review from electrum September 23, 2024 18:58
@cla-bot cla-bot bot added the cla-signed label Sep 23, 2024
Copy link
Member

@electrum electrum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We should support wasbs as well, to be fully compatible with existing Hadoop locations.

@@ -152,6 +152,12 @@
<scope>runtime</scope>
</dependency>

<dependency>
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is it needed?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

One of the tests calls slice.toUtf8String so it has to be declared

@dain dain merged commit f553882 into trinodb:master Sep 23, 2024
58 checks passed
@dain dain deleted the wasb branch September 23, 2024 21:41
@github-actions github-actions bot added this to the 459 milestone Sep 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Support wasb scheme alias in native Azure filesystem
3 participants