[release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. #59532

jozkee · 2021-09-23T17:13:03Z

Backport of #59338 to release/6.0

Customer Impact

As per @stephentoub in #59338 (comment):

in .NET 6. If you end up writing fewer bytes than you preallocated, in release/6.0 Windows will truncate the file to the number of bytes actually written whereas Linux will end up with a file padded with zeros at the end to meet the preallocationSize. In main (before this change was merged in main), both Windows and Linux will end up with said zeros at the end. This PR tries to bring it to a state where preallocationSize is just a (non-observable) hint purely for performance, such that the resulting file size isn't dependent on the preallocationSize, and the latter is just there to help the system optimize.

The downside is that some OSes/platforms (e.g: browser) don't have the hint behavior and therefore will ignore preallocationSize.

Testing

Tests are being updated accordingly.

Risk

Low

Note

This PR replaces [release/6.0] Align preallocationSize behavior #59078.
Automatic backport failed because Align preallocationSize behavior #58726 is not in release/6.0 and this PR reverts several changes made by that PR.
For an easier port (and less prone to human error) I've cherry-picked both PRs to avoid manual conflict resolution.

ghost · 2021-09-23T17:13:11Z

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

Backport of #59338 to release/6.0

Customer Impact

Author:	Jozkee
Assignees:	-
Labels:	`area-System.IO`
Milestone:	-

danmoseley · 2021-09-23T17:40:24Z

not ready yet, but marking so tactics aware. last day is essentially tomorrow.

jozkee · 2021-09-23T17:46:39Z

Will send a commit to fix the browser error, strange that we didn't got it in the PR for main.

src/libraries/Native/Unix/System.Native/pal_io.c(1015,9): error G2D4C7688: (NETCORE_ENGINEERING_TELEMETRY=Build) unused variable 'fileDescriptor' [-Werror,-Wunused-variable]

jozkee · 2021-09-23T17:49:58Z

@danmoseley it should be ready now.

danmoseley · 2021-09-23T18:06:00Z

@jozkee we can merge as soon as it's green. @Anipik

jeffhandley · 2021-09-23T18:48:11Z

not ready yet, but marking so tactics aware. last day is essentially tomorrow.

For RC2, right? Is the intent/expectation to backport this PR into release/6.0-rc2 after it's merged into release/6.0?

danmoseley · 2021-09-24T00:34:27Z

Ah yes I didn't realize this wasn't targeted at RC2

danmoseley · 2021-09-24T00:36:34Z

Rebased, hopefully it works. Still need a review

stephentoub · 2021-09-24T01:01:38Z

eng/Versions.props

-    <PreReleaseVersionLabel>rc</PreReleaseVersionLabel>
-    <PreReleaseVersionIteration>2</PreReleaseVersionIteration>
+    <PreReleaseVersionLabel>rtm</PreReleaseVersionLabel>
+    <PreReleaseVersionIteration></PreReleaseVersionIteration>


@danmoseley, these rebases aren't doing the right things...

I suspect it would be easier/cleaner to back out (with a force push) the rebase commit, and change this back to target release/6.0, merge this as it was into that branch, and then use the backport bot to port from release/6.0 to release/6.0-rc2.

I think we could cherry-pick Adam and Tom's commits to release/6.0-rc2 then force-push that to this branch; then the bot that syncs rc2 with release/6.0 will port automatically.

Oh wow. I should have noticed what this did. I wonder how many others...

Yes into 6.0 first is fine it just needs to be in RC2 tomorrow. I'm hesitant to attempt to fix anything again..

To be clear, I rebased to include only the relevant commits and is already targeting RC2. No need to manually port to 6.0 as that is made automatically AFAIK.

Co-authored-by: Stephen Toub <[email protected]>

* File preallocationSize: align Windows and Unix behavior. This aligns Windows and Unix behavior of preallocationSize for the intended use-case of specifing the size of a file that will be written. For this use-case, the expected FileAccess is Write, and the file should be a new one (FileMode.Create*) or a truncated file (FileMode.Truncate). Specifing a preallocationSize for other modes, or non-writable files throws ArgumentException. The opened file will have a length of zero, and is ready to be written to by the user. If the requested size cannot be allocated, an IOException is thrown. When the OS/filesystem does not support pre-allocating, preallocationSize is ignored. * fix pal_io preprocessor checks * pal_io more fixes * ctor_options_as.Windows.cs: fix compilation * Update tests * tests: use preallocationSize from all public APIs * pal_io: add back FreeBSD, fix OSX * tests: check allocated is zero when preallocation is not supported. * Only throw for not enough space errors * Fix compilation * Add some more tests * Fix ExtendedPathsAreSupported test * Apply suggestions from code review Co-authored-by: David Cantú <[email protected]> * Update System.Private.CoreLib Strings.resx * PR feedback * Remove GetPathToNonExistingFile * Fix compilation * Skip checking allocated size on mobile platforms. Co-authored-by: David Cantú <[email protected]>

jeffhandley

Looks like a good retargeting to me; thanks, @jozkee. Either @tmds or @stephentoub should take another look through the changes to verify too.

tmds

lgtm

danmoseley · 2021-09-24T14:35:07Z

Want me to merge now @jozkee ?

src/libraries/System.IO.FileSystem/tests/FileStream/ctor_options.cs

stephentoub · 2021-09-24T15:47:12Z

src/libraries/System.IO.FileSystem/tests/FileStream/ctor_options.cs

+            }
+        }
+
+        [OuterLoop("Might allocate 1 TB file if there is enough space on the disk")]


Remind me never to run this suite on outerloop ;-)

src/libraries/System.IO.FileSystem/tests/FileStream/ctor_options.cs

src/libraries/System.IO.FileSystem/src/Resources/Strings.resx

danmoseley · 2021-09-24T17:51:12Z

I will be unavailable this afternoon (recruiting) so please rely on someone else to merge when ready.

dotnet-issue-labeler bot added the area-System.IO label Sep 23, 2021

danmoseley added the Servicing-consider Issue for next servicing release review label Sep 23, 2021

jozkee added this to the 6.0.0 milestone Sep 23, 2021

jozkee requested a review from danmoseley September 23, 2021 17:50

jamshedd added Servicing-approved Approved for servicing release and removed Servicing-consider Issue for next servicing release review labels Sep 23, 2021

danmoseley changed the base branch from release/6.0 to release/6.0-rc2 September 24, 2021 00:35

stephentoub reviewed Sep 24, 2021

View reviewed changes

adamsitnik and others added 4 commits September 23, 2021 19:08

Align preallocationSize behavior (dotnet#58726)

ee72add

Co-authored-by: Stephen Toub <[email protected]>

Fix unused fileDescriptor

8adba3a

void fd when unused

f85919f

jozkee force-pushed the prealloc_6.0 branch from 83aa152 to f85919f Compare September 24, 2021 02:09

jozkee changed the title ~~[release/6.0] File preallocationSize: align Windows and Unix behavior.~~ [release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. Sep 24, 2021

jeffhandley approved these changes Sep 24, 2021

View reviewed changes

tmds approved these changes Sep 24, 2021

View reviewed changes

runfoapp bot mentioned this pull request Sep 24, 2021

emsdk failure: emcmake not found #59556

Closed

stephentoub reviewed Sep 24, 2021

View reviewed changes

src/libraries/System.IO.FileSystem/tests/FileStream/ctor_options.cs Outdated Show resolved Hide resolved

stephentoub reviewed Sep 24, 2021

View reviewed changes

src/libraries/System.IO.FileSystem/tests/FileStream/ctor_options.cs Outdated Show resolved Hide resolved

stephentoub reviewed Sep 24, 2021

View reviewed changes

src/libraries/System.IO.FileSystem/src/Resources/Strings.resx Outdated Show resolved Hide resolved

stephentoub approved these changes Sep 24, 2021

View reviewed changes

Address feedback

861959e

Anipik merged commit 157d591 into dotnet:release/6.0-rc2 Sep 24, 2021

jozkee deleted the prealloc_6.0 branch September 24, 2021 21:57

ghost locked as resolved and limited conversation to collaborators Nov 3, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. #59532

[release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. #59532

jozkee commented Sep 23, 2021 •

edited

Loading

ghost commented Sep 23, 2021

Customer Impact

danmoseley commented Sep 23, 2021

jozkee commented Sep 23, 2021

jozkee commented Sep 23, 2021

danmoseley commented Sep 23, 2021

jeffhandley commented Sep 23, 2021

danmoseley commented Sep 24, 2021

danmoseley commented Sep 24, 2021

stephentoub Sep 24, 2021 •

edited

Loading

jeffhandley Sep 24, 2021

jozkee Sep 24, 2021 •

edited

Loading

danmoseley Sep 24, 2021

jozkee Sep 24, 2021

jeffhandley left a comment

tmds left a comment

danmoseley commented Sep 24, 2021

stephentoub Sep 24, 2021

danmoseley commented Sep 24, 2021

[release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. #59532

[release/6.0-rc2] File preallocationSize: align Windows and Unix behavior. #59532

Conversation

jozkee commented Sep 23, 2021 • edited Loading

Customer Impact

Testing

Risk

Note

ghost commented Sep 23, 2021

Customer Impact

danmoseley commented Sep 23, 2021

jozkee commented Sep 23, 2021

jozkee commented Sep 23, 2021

danmoseley commented Sep 23, 2021

jeffhandley commented Sep 23, 2021

danmoseley commented Sep 24, 2021

danmoseley commented Sep 24, 2021

stephentoub Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

jeffhandley Sep 24, 2021

Choose a reason for hiding this comment

jozkee Sep 24, 2021 • edited Loading

Choose a reason for hiding this comment

danmoseley Sep 24, 2021

Choose a reason for hiding this comment

jozkee Sep 24, 2021

Choose a reason for hiding this comment

jeffhandley left a comment

Choose a reason for hiding this comment

tmds left a comment

Choose a reason for hiding this comment

danmoseley commented Sep 24, 2021

stephentoub Sep 24, 2021

Choose a reason for hiding this comment

danmoseley commented Sep 24, 2021

jozkee commented Sep 23, 2021 •

edited

Loading

stephentoub Sep 24, 2021 •

edited

Loading

jozkee Sep 24, 2021 •

edited

Loading