Writing to Data Links #9750
Writing to Data Links #9750
Conversation
…added as details but as part of main message
This reverts commit a6c07590bcbcf14b6c4a7e159a312f5115198033.
This reverts commit 162cdd5678f258859a2a46443fbe539c015c6c98.
|
Converting to draft as some further work on copying files is needed |
…alinks # Conflicts: # CHANGELOG.md
| copy_to : File_Like -> Boolean -> Any ! File_Error | ||
| copy_to self (destination : File_Like) (replace_existing : Boolean = False) = Data_Link_Helpers.disallow_links_in_copy self destination <| | ||
| if self.is_directory then Error.throw (S3_Error.Error "Copying S3 folders is currently not implemented." self.uri) else | ||
| Context.Output.if_enabled disabled_message="Copying an S3_File is forbidden as the Output context is disabled." panic=False <| | ||
| case destination.file of | ||
| destination_writable = Writable_File.from destination | ||
| case destination_writable.file of |
There was a problem hiding this comment.
I changed Writable_File to File_Like to avoid eagerly triggering the conversion.
The problem was - converting a data link to Writable_File 'follows it' but if it did not exist, that will fail with File_Error.Not_Found and it would fail too early, not giving us a chance to abandon the operation because of disallow_links_in_copy.
Briefly I thought that this indicates that the Writable_File 'class' is too loaded and there should be a separation of concerns between expressing the capability of 'a file that can be written to' and resolving the data link.
But actually, almost all code does want this to happen in one step. Our users and library devs, will most of the time want to write a function taking Writable_File and be able to use this file to write their fancy data formats to it, not caring how the underlying structures resolve the data links to write to the data link targets - implementors of new formats should not care about data links, only about taking a Writable_File and writing to it.
Only implementors of new file systems need to be more careful around data links and for operations that may not want to follow the link (e.g. copy_to and move_to) they will need to avoid the eager conversion like here and use some other type (I decided to use File_Like as this is our 'most generic' type class that every File type should be implementing). I think that is fine as a bit more complexity in the filesystem logic sounds better than complicating the file format logic: we will likely end up with more formats than filesystems and external contributors/users will be more likely to need to write a new format than a new filesystem altogether. Moreover it's not hard anyway as one can just copy the 'best practices' from the S3_File implementation.
…alinks # Conflicts: # CHANGELOG.md
Pull Request Description
Important Notes
Checklist
Please ensure that the following checklist has been satisfied before submitting the PR:
Scala,
Java,
and
Rust
style guides. In case you are using a language not listed above, follow the Rust style guide.
./run ide build.