dotnet-restore Too many open files on Mac

[zhili1208]

went through all stream in NuGet, Did not find any leak, set the concurrency request number to 64, but it still doesn't work. need to look into this issue deeply.

[moozzyk]

Repro steps:

1. Clone the https://github.com/aspnet/SignalR-SqlServer
2. Switch to pawelka/dotnet branch
3. Run ./build.sh

info: Exec
info:   program: dotnet
info:   commandline: restore
info:   workingdir: /Users/Administrator/source/SignalR-SqlServer
info : Running restore with 16 concurrent jobs.
info : Restoring packages for /Users/Administrator/source/SignalR-SqlServer/test/Microsoft.AspNetCore.SignalR.SqlServer.Tests/project.json...
info :   NotFound https://api.nuget.org/v3-flatcontainer/microsoft.extensions.platformabstractions.dnx/index.json 306ms
info :   NotFound https://api.nuget.org/v3-flatcontainer/microsoft.extensions.compilationabstractions/index.json 298ms
info :   NotFound https://api.nuget.org/v3-flatcontainer/microsoft.extensions.commandlineutils/index.json 299ms
info : Installing Microsoft.Extensions.DependencyInjection.Abstractions 1.0.0-rc2-15991.
info : Installing Microsoft.Extensions.Logging 1.0.0-rc2-16037.
info : Installing Microsoft.Extensions.CompilationAbstractions 1.0.0-rc2-16441.
info : Installing Microsoft.Extensions.CommandLineUtils 1.0.0-rc2-16043.
info : Installing Microsoft.Extensions.Logging.Abstractions 1.0.0-rc2-16037.
info : Installing Microsoft.Dnx.TestHost 1.0.0-rc2-16130.
info : Installing Microsoft.Dnx.Testing.Abstractions 1.0.0-rc2-16130.
info : Installing Microsoft.Extensions.PlatformAbstractions 1.0.0-rc2-16536.
info : Installing Microsoft.Extensions.PlatformAbstractions.Dnx 1.0.0-rc2-16441.
info : Installing xunit.runner.aspnet 2.0.0-aspnet-rc2-16097.
info : All packages are compatible with DNX,Version=v4.5.1.
info : All packages are compatible with DNX,Version=v4.5.1 (osx.10.11-x64).
info : Committing restore...
info : Restore completed in 13632ms.
info : Running restore with 16 concurrent jobs.
info : Restoring packages for /Users/Administrator/source/SignalR-SqlServer/src/Microsoft.AspNetCore.SignalR.SqlServer/project.json...
error: Too many open files
fail: Exit code 1 from dotnet

[muratg]

I hit this with basic HelloWeb sample on OS X 10.10.5

[julielerman]

@muratg the temp work around helps (for now) .. ulimit -n 512 (found here: https://github.com/dotnet/cli/issues/809)

[borgdylan]

Even on mono, I have to limit the number of threads per CPU or else the memory balloons up very quickly and uses up all 4GB of virtual memory my PC gives to each process. Does nuget keep files open for no reason?

[muratg]

@julielerman Yup, thanks! I've been using the workaround.

[seesharper]

Confirms that ulimit -n 512 works as a workaround

[yishaigalatzer]

@zhili1208 checked in a fix and verified it

[kichalla]

@zhili1208 I am seeing this issue again on Mac. I am using the dotnet CLI version 1.0.0.001496. (On a side note: How easy it is for me to find if a given dotnet CLI has the fix or not? I tried looking for a commit SHA embedded in the NuGet.Client and other NuGet.* assemblies but couldn't find one)

@muratg

[moozzyk]

@kichalla - I could not reopen this so I filed a new issue for Too Many Open files yesterday. See #2163

[muratg]

@zhili1208 Could you have a look? This was OK for a while, so not sure what the regression is. Perhaps CLI picked up an older build of nuget?