Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[release/7.0] Backport tar bug fixes and improvements #74449

Merged
merged 10 commits into from
Aug 24, 2022

Conversation

carlossanlop
Copy link
Member

@carlossanlop carlossanlop commented Aug 23, 2022

Customer Impact

The tar APIs are new to 7.0. These changes make sure we ship the new APIs with great performance and fixes for important bugs we found in the last weeks.

Testing

Added a large number of tests to cover all the bugs fixed in these commits.

Risk

System.Formats.Tar is a new namespace and the changes are confined to only that space, so the risk is low.

stephentoub and others added 9 commits August 23, 2022 12:02
* Avoid unnecessary byte[] allocations

* Remove unnecessary use of FileStreamOptions

* Clean up Dispose{Async} implementations

* Clean up unnecessary consts

Not a perf thing, just readability.

* Remove MemoryStream/Encoding.UTF8.GetBytes allocations, unnecessary async variants, and overhaul GenerateExtendedAttributesDataStream

* Avoid string allocations in ReadMagicAttribute

* Avoid allocation in WriteAsOctal

* Improve handling of octal

* Avoid allocation for version string

* Removing boxing and char string allocation in GenerateExtendedAttributeName

* Fix a couple unnecessary dictionary lookups

* Replace Enum.HasFlag usage

* Remove allocations from Write{Posix}Name

* Replace ArrayPool use with string.Create

* Replace more superfluous ArrayPool usage

* Remove ArrayPool use from System.IO.Compression.ZipFile

* Fix inverted condition

* Use generic math to parse octal

* Remove allocations from StringReader and string.Split

* Remove magic string allocation for Ustar when not V7

* Remove file name and directory name allocation in GenerateExtendedAttributeName
* Fix a few Tar issues post perf improvements

* Update src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs
…74376)

* Skip directory symlink recursion on TarFile archive creation

* Add symlink verification

* Address suggestions by danmoseley

Co-authored-by: carlossanlop <[email protected]>
…r case bugs (dotnet#74412)

* Remove unused _readFirstEntry. Remnant from before we created PaxGlobalExtendedAttributesEntry.

* Set the position of the freshly copied data stream to 0, so the first user access of the DataStream property gives them a stream ready to read from the beginning.

* Process a PAX actual entry's data block only after the extended attributes are analyzed, in case the size is found as an extended attribute and needs to be overriden.

* Add tests to verify the entries of the new external tar assets can be read. Verify their DataStream if available.

* Add copyData argument to recent alignment padding tests.

* Throw an exception sooner and with a clearer message when a data section is unexpected for the entry type.

* Allow trailing nulls and spaces in octal fields.
Co-authored-by: @am11 Adeel Mujahid <[email protected]>

* Throw a clearer exception if the unsupported sparse file entry type is encountered. These entries have additional data that indicates the locations of sparse bytes, which cannot be read with just the size field. So to avoid accidentally offseting the reader, we throw.

* Tests.

* Rename to TrimLeadingNullsAndSpaces

Co-authored-by: carlossanlop <[email protected]>
@ghost
Copy link

ghost commented Aug 23, 2022

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

Customer Impact

The tar APIs are new to 7.0. These changes make sure we ship the new APIs with great performance and fixes for important bugs we found in the last weeks.

Testing

Added a large number of tests to cover all the bugs fixed in these commits.

Risk

System.Formats.Tar is a new namespace and the changes are confined to only that space, so the risk is low.

Pending: #74412

Author: carlossanlop
Assignees: carlossanlop
Labels:

area-System.IO

Milestone: -

@carlossanlop
Copy link
Member Author

Build failure fixed. I missed a using in the last commit that removed the Compression code.

@danmoseley
Copy link
Member

OK approved. It's a bigger change than we'd normally like at this stage, but it's strictly limited to tar. Without this, we're at risk of being incompatible with real-world files, so I think it's completing the feature, and it brings considerably more real world tests.

@carlossanlop
Copy link
Member Author

Approved/signed-off.
CI failures are unrelated: Timeout and job cancelled.
Ready to merge. :shipit:

@carlossanlop carlossanlop merged commit 35e86f9 into dotnet:release/7.0 Aug 24, 2022
@carlossanlop carlossanlop deleted the TarBackport7 branch August 24, 2022 00:11
@ghost ghost locked as resolved and limited conversation to collaborators Sep 23, 2022
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

6 participants