Skip directory symlink recursion on TarFile archive creation #74376

carlossanlop · 2022-08-22T19:37:19Z

ghost · 2022-08-22T19:37:44Z

Tagging subscribers to this area: @dotnet/area-system-io
See info in area-owners.md if you want to be subscribed.

Issue Details

Author:	carlossanlop
Assignees:	carlossanlop
Labels:	`area-System.IO`
Milestone:	-

jozkee · 2022-08-22T20:14:28Z

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

+                Assert.Equal("subDirectory/", entry.Name);
+


Suggested change

Assert.Equal("subDirectory/", entry.Name);

Assert.Equal("subDirectory/", entry.Name);

Assert.Equal(TarEntyrType.SymbolicLink, entry.EntryType);

Working on this for all the 4 tests.

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectory.File.Tests.cs

danmoseley

With this change can we read anything new in the new corpus ?

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectoryAsync.File.Tests.cs

carlossanlop · 2022-08-22T22:33:40Z

With this change can we read anything new in the new corpus ?

I looked through the new assets, didn't find any that could repro this issue. This was originally reported by @tmds.

jozkee

Otherwise; LGTM.

jozkee · 2022-08-22T23:02:43Z

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs

                if (includeBaseDirectory)
                {
                    writer.WriteEntry(di.FullName, GetEntryNameForBaseDirectory(di.Name));
+                    skipBaseDirRecursion = (di.Attributes & FileAttributes.ReparsePoint) != 0;


you don't need skipBaseDirRecursion.

Suggested change

skipBaseDirRecursion = (di.Attributes & FileAttributes.ReparsePoint) != 0;

if ((di.Attributes & FileAttributes.ReparsePoint) != 0)

{

return;

}

Both suggestions can be fixed in my next PR.

jozkee · 2022-08-22T23:04:32Z

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs

                if (includeBaseDirectory)
                {
                    await writer.WriteEntryAsync(di.FullName, GetEntryNameForBaseDirectory(di.Name), cancellationToken).ConfigureAwait(false);
+                    skipBaseDirRecursion = (di.Attributes & FileAttributes.ReparsePoint) != 0;


carlossanlop · 2022-08-23T01:08:10Z

Test failures unrelated:
System.Net.Quic.Functional.Tests - #73688
System.Net.Http.Functional.Tests - #74162

…74376) * Skip directory symlink recursion on TarFile archive creation * Add symlink verification * Address suggestions by danmoseley Co-authored-by: carlossanlop <[email protected]>

lewing · 2022-08-23T17:15:49Z

/backport to release/7.0-rc1

github-actions · 2022-08-23T17:15:58Z

Started backporting to release/7.0-rc1: https://github.com/dotnet/runtime/actions/runs/2913450860

github-actions · 2022-08-23T17:17:56Z

@lewing backporting to release/7.0-rc1 failed, the patch most likely resulted in conflicts:

$ git am --3way --ignore-whitespace --keep-non-patch changes.patch

Patch format detection failed.
Error: The process '/usr/bin/git' failed with exit code 128

Please backport manually!

carlossanlop · 2022-08-23T17:56:45Z

@lewing I am including this backport in this draft: #74414

…74376) * Skip directory symlink recursion on TarFile archive creation * Add symlink verification * Address suggestions by danmoseley Co-authored-by: carlossanlop <[email protected]>

@am11

* Improve performance of Tar library (#74281) * Avoid unnecessary byte[] allocations * Remove unnecessary use of FileStreamOptions * Clean up Dispose{Async} implementations * Clean up unnecessary consts Not a perf thing, just readability. * Remove MemoryStream/Encoding.UTF8.GetBytes allocations, unnecessary async variants, and overhaul GenerateExtendedAttributesDataStream * Avoid string allocations in ReadMagicAttribute * Avoid allocation in WriteAsOctal * Improve handling of octal * Avoid allocation for version string * Removing boxing and char string allocation in GenerateExtendedAttributeName * Fix a couple unnecessary dictionary lookups * Replace Enum.HasFlag usage * Remove allocations from Write{Posix}Name * Replace ArrayPool use with string.Create * Replace more superfluous ArrayPool usage * Remove ArrayPool use from System.IO.Compression.ZipFile * Fix inverted condition * Use generic math to parse octal * Remove allocations from StringReader and string.Split * Remove magic string allocation for Ustar when not V7 * Remove file name and directory name allocation in GenerateExtendedAttributeName * fix tar strings (#74321) * Fix some corner cases in TarReader (#74329) * Fix a few Tar issues post perf improvements (#74338) * Fix a few Tar issues post perf improvements * Update src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarHeader.Write.cs * Skip directory symlink recursion on TarFile archive creation (#74376) * Skip directory symlink recursion on TarFile archive creation * Add symlink verification * Address suggestions by danmoseley Co-authored-by: carlossanlop <[email protected]> * SkipBlockAlignmentPadding must be executed for all entries (#74396) * Set modified timestamps on files being extracted from tar archives (#74400) * Add tests for exotic external tar asset archives, fix some more corner case bugs (#74412) * Remove unused _readFirstEntry. Remnant from before we created PaxGlobalExtendedAttributesEntry. * Set the position of the freshly copied data stream to 0, so the first user access of the DataStream property gives them a stream ready to read from the beginning. * Process a PAX actual entry's data block only after the extended attributes are analyzed, in case the size is found as an extended attribute and needs to be overriden. * Add tests to verify the entries of the new external tar assets can be read. Verify their DataStream if available. * Add copyData argument to recent alignment padding tests. * Throw an exception sooner and with a clearer message when a data section is unexpected for the entry type. * Allow trailing nulls and spaces in octal fields. Co-authored-by: @am11 Adeel Mujahid <[email protected]> * Throw a clearer exception if the unsupported sparse file entry type is encountered. These entries have additional data that indicates the locations of sparse bytes, which cannot be read with just the size field. So to avoid accidentally offseting the reader, we throw. * Tests. * Rename to TrimLeadingNullsAndSpaces Co-authored-by: carlossanlop <[email protected]> * Remove Compression changes, keep changes confined to Tar. * Fix build failure due to missing using in TarHelpers.Windows Co-authored-by: Stephen Toub <[email protected]> Co-authored-by: Dan Moseley <[email protected]> Co-authored-by: Adeel Mujahid <[email protected]> Co-authored-by: carlossanlop <[email protected]> Co-authored-by: David Cantú <[email protected]>

Skip directory symlink recursion on TarFile archive creation

629e67f

carlossanlop added the area-System.IO label Aug 22, 2022

carlossanlop requested review from tmds, adamsitnik, danmoseley and jozkee August 22, 2022 19:37

carlossanlop self-assigned this Aug 22, 2022

jozkee reviewed Aug 22, 2022

View reviewed changes

Add symlink verification

36e8791

carlossanlop requested a review from jozkee August 22, 2022 21:07

danmoseley reviewed Aug 22, 2022

View reviewed changes

src/libraries/System.Formats.Tar/src/System/Formats/Tar/TarFile.cs Outdated Show resolved Hide resolved

src/libraries/System.Formats.Tar/tests/TarFile/TarFile.CreateFromDirectoryAsync.File.Tests.cs Outdated Show resolved Hide resolved

Address suggestions by danmoseley

5da8dd9

jozkee approved these changes Aug 22, 2022

View reviewed changes

This was referenced Aug 23, 2022

[QUIC] Abort on cancellation throws QUIC_STATUS_INVALID_PARAMETER #73688

Closed

Assert failed: System.Net.Http.Http3Connection.SendAsync in coreclr Linux_musl x64 Debug #74162

Closed

carlossanlop merged commit 298c04e into dotnet:main Aug 23, 2022

carlossanlop deleted the symlink branch August 23, 2022 01:08

ghost locked as resolved and limited conversation to collaborators Sep 22, 2022

carlossanlop added area-System.Formats.Tar and removed area-System.IO labels Nov 22, 2022

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Skip directory symlink recursion on TarFile archive creation #74376

Skip directory symlink recursion on TarFile archive creation #74376

carlossanlop commented Aug 22, 2022

ghost commented Aug 22, 2022

jozkee Aug 22, 2022

carlossanlop Aug 22, 2022

danmoseley left a comment

carlossanlop commented Aug 22, 2022

jozkee left a comment

jozkee Aug 22, 2022

carlossanlop Aug 22, 2022

jozkee Aug 22, 2022

carlossanlop commented Aug 23, 2022

lewing commented Aug 23, 2022

github-actions bot commented Aug 23, 2022

github-actions bot commented Aug 23, 2022

carlossanlop commented Aug 23, 2022

	Assert.Equal("subDirectory/", entry.Name);
	Assert.Equal("subDirectory/", entry.Name);
	Assert.Equal(TarEntyrType.SymbolicLink, entry.EntryType);

Skip directory symlink recursion on TarFile archive creation #74376

Skip directory symlink recursion on TarFile archive creation #74376

Conversation

carlossanlop commented Aug 22, 2022

ghost commented Aug 22, 2022

jozkee Aug 22, 2022

Choose a reason for hiding this comment

carlossanlop Aug 22, 2022

Choose a reason for hiding this comment

danmoseley left a comment

Choose a reason for hiding this comment

carlossanlop commented Aug 22, 2022

jozkee left a comment

Choose a reason for hiding this comment

jozkee Aug 22, 2022

Choose a reason for hiding this comment

carlossanlop Aug 22, 2022

Choose a reason for hiding this comment

jozkee Aug 22, 2022

Choose a reason for hiding this comment

carlossanlop commented Aug 23, 2022

lewing commented Aug 23, 2022

github-actions bot commented Aug 23, 2022

github-actions bot commented Aug 23, 2022

carlossanlop commented Aug 23, 2022