-
Notifications
You must be signed in to change notification settings - Fork 3.6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
GH-38703: [C++][FS][Azure] Implement DeleteFile() #39840
Changes from 4 commits
6947360
7df1f8d
a08d5a1
79e6206
8559881
6e082dd
4863b20
fdde91a
b812aaa
3330284
b89a07a
4b1c635
516be74
6c0fa18
110932b
953da8e
6512c12
d2e6f9b
58888b4
5af7159
b96af56
79e80b7
022bd57
58c4ccd
3cce85c
168356e
dd60704
39eb5ae
fc7ada2
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -376,6 +376,10 @@ culpa qui officia deserunt mollit anim id est laborum. | |
return s; | ||
} | ||
|
||
std::string RandomFileName(RNG &rng) { return RandomChars(10, rng); } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Could you use "blob" not "file" because Azure Blob Storage uses "blob"? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. I can, yes. Note that my use case will be to access ADLS, which is why I referred to this as a file. Also because the method is named DeleteFile(), not DeleteBlob(). |
||
|
||
std::string RandomFileNameExtension(RNG &rng) { return RandomChars(3, rng); } | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need to define separated functions? There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. No, it generates the filename only, sans extension. I suppose that it could be modified to generate XXXXXXX.ZZZ however, if that would be preferred. See commentary above. I could just place everything inline and delete these functions if that is the route that is recommended. I'm open to suggestion. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Combined. |
||
|
||
static int RandomIndex(int end, RNG& rng) { | ||
return std::uniform_int_distribution<int>(0, end - 1)(rng); | ||
} | ||
|
@@ -1382,6 +1386,32 @@ TEST_F(TestAzuriteFileSystem, DeleteDirContentsFailureNonexistent) { | |
this->TestDeleteDirContentsFailureNonexistent(); | ||
} | ||
|
||
TEST_F(AzuriteFileSystemTest, DeleteFileSuccessHaveFile) { | ||
void CreateFile(FileSystem* fs, const std::string& path, const std::string& data) { | ||
const auto file_name = PreexistingData::RandomFileName() + | ||
PreexistingData::RandomFileNameExtension(); | ||
ASSERT_OK(CreateFile(fs_.get(), file_name, "abc")); | ||
arrow::fs::AssertFileInfo(fs_.get(), file_name, FileType::File); | ||
ASSERT_OK(fs_->DeleteFile(file_name)); | ||
arrow::fs::AssertFileInfo(fs_.get(), file_name, FileType::NotFound); | ||
} | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Hmm. Is this a valid test? (Why do you define a function in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Not certain that I understand the question. The idea behind this test was to create a file to delete, then delete it. Thus the pseudo-code was:
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. BTW, the choice to create the RandomFileName() and RandomFileNameExtension() methods was done in part because I prefer abstractions (generally) and also because there was a similar abstraction for the container name, which is found at line 369, thus: There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Alright, now I understand why kou asked why I defined a function in There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Finally, due to the structure of how the RandomChars() function is written within the PreexistingData class, I'll have to leave the random-name functionality that I need within this class. However, I'll take kou's suggestion and combine the two methods, as well as changing the name to blob. There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The reason we create random containers is to not have test runs interfere with later runs when the tests crash and containers can't be deleted from the storage account. The blobs inside the containers don't have to be random. Every test case is creating a new random container that you can freely add named blobs to. |
||
|
||
TEST_F(AzuriteFileSystemTest, DeleteFileSuccessNonexistent) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
|
||
const auto file_name = PreexistingData::RandomFileName() + | ||
PreexistingData::RandomFileNameExtension(); | ||
ASSERT_OK(fs_->DeleteFile(file_name)); | ||
arrow::fs::AssertFileInfo(fs_.get(), file_name, FileType::NotFound); | ||
} | ||
|
||
TEST_F(AzuriteFileSystemTest, DeleteFileSuccessNotAFile) { | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. ditto. |
||
const auto container_name = RandomContainerName(); | ||
ASSERT_OK(fs_->CreateDir(container_name)); | ||
arrow::fs::AssertFileInfo(fs_.get(), container_name, FileType::Directory); | ||
ASSERT_RAISES(IOError, fs_->DeleteFile(container_name)); | ||
ASSERT_OK(fs_->DeleteDir(container_name)); | ||
arrow::fs::AssertFileInfo(fs_.get(), container_name, FileType::NotFound); | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Do we need them for There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. The point to this test was to attempt to delete a target that was not a file. So I created a directory (container), then attempted to delete it, which should fail, which I verify via: |
||
} | ||
|
||
av8or1 marked this conversation as resolved.
Show resolved
Hide resolved
|
||
TEST_F(TestAzuriteFileSystem, CopyFileSuccessDestinationNonexistent) { | ||
auto data = SetUpPreexistingData(); | ||
const auto destination_path = data.ContainerPath("copy-destionation"); | ||
|
@@ -1868,6 +1898,5 @@ TEST_F(TestAzuriteFileSystem, OpenInputFileClosed) { | |
ASSERT_RAISES(Invalid, stream->ReadAt(1, 1)); | ||
ASSERT_RAISES(Invalid, stream->Seek(2)); | ||
} | ||
|
||
} // namespace fs | ||
} // namespace arrow | ||
} // namespace fs | ||
} // namespace arrow |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Could you use
ValidateFileLocation()
instead?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure, done. Those three lines have been replaced with:
RETURN_NOT_OK(ValidateFileLocation(location));