Change NuGet package native dlls deployment mode

[nulltoken]

Latest NuGet version (2.5) introduces a feature that should help the deployment of native dlls: Automatic import of msbuild targets and propos files.

This could help removing the nuget.package/Tools/*.ps1 scripts by adding a .targets file in the /build/<framework> package folder, which can contain tags that have items that are set to copy to output.

A sample is provided below:

<?xml version="1.0" encoding="utf-8"?>
<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003" InitialTargets="zlib_redist_init">


<ItemGroup >
    <None Include="$(MSBuildThisFileDirectory)..\..\build\native\bin\x64\v110\dynamic\cdecl\Release\zlib1.dll" >
      <DeploymentContent>true</DeploymentContent>
    </None>
</ItemGroup>

</Project>

Some additional low-level information on the topic: How to Build a Package

Note: We may also add some minimum NuGet Client version requirement, but it is unclear how a NuGet client < 2.5 will react to this directive and to the additional .targets file

[nulltoken]

@davidebbo Have you tried to convert SqlCE NuGet package to this new format (cf. NuGet issue #109)?

/cc @haacked @fearthecowboy

[davidebbo]

I don't own that package these days. Maybe @jeffhandley?

[fearthecowboy]

FYI, That tutorial is geared towards native (C/C++) packaging. For managed projects there's not a lot of benefit for moving to the CoApp tools (well, not yet) ... as we get the native scenarios completely nailed with the tool, I'll start adding more support for managed projects too.

[nulltoken]

From AndrewNurse, in Nuget Jabbr room: "Yeah, the NuGet 2.5 changes should definitely make that easier. You'd basically want to have a "libgit2sharp.targets" file in the "build/" folder (or if it is framework independent, just "build/"). We'll automatically add an <Import Project="path/to/libgit2sharp.targets" Condition="Exists('path/to/libgit2sharp.targets') /> node to the project file of every project that installs that package"

[nulltoken]

Currently, installing the NuGet package triggers the execution of the install.ps1 hook. This scripts add the following lines to the post-build events

if not exist "$(TargetDir)NativeBinaries" md "$(TargetDir)NativeBinaries"
if not exist "$(TargetDir)NativeBinaries\x86" md "$(TargetDir)NativeBinaries\x86"
xcopy /s /y "$(SolutionDir)packages\LibGit2Sharp.0.11.0.0\NativeBinaries\x86*.*" "$(TargetDir)NativeBinaries\x86"
if not exist "$(TargetDir)NativeBinaries\amd64" md "$(TargetDir)NativeBinaries\amd64"
xcopy /s /y "$(SolutionDir)packages\LibGit2Sharp.0.11.0.0\NativeBinaries\amd64*.*" "$(TargetDir)NativeBinaries\amd64"

Topics to cover:

• How to describe in the .targets file that a directory structure has to be built in the output directory and that the native library should be copied here
• Check that git2.dll not found in 0.11.0.0, Win 7 64, VS Express 2012 #432 is fixed with this new way of packaging/deploying

[nulltoken]

Hand made workaround:

• Create a new Console App
• Install LibGit2Sharp NuGet package
• Edit the .csproj file
• Remove the PostBuildEvent element
• Add the following element after the last <Import>

 <Import Project="..\packages\LibGit2Sharp.0.11.0.0\build\libgit2sharp.targets" Condition="Exists('..\packages\LibGit2Sharp.0.11.0.0\build\libgit2sharp.targets')" />

• Create a file named libgit2sharp.targets in packages/build
• Set its content to

<Project ToolsVersion="4.0" xmlns="http://schemas.microsoft.com/developer/msbuild/2003">
  <Target Name="AfterBuild">
    <CreateItem Include="$(MSBuildThisFileDirectory)..\NativeBinaries\**\*.*">
      <Output TaskParameter="Include" ItemName="NativeBinaries" />
    </CreateItem>
    <Copy SourceFiles="@(NativeBinaries)" DestinationFiles="@(NativeBinaries->'$(OutputPath)NativeBinaries\%(RecursiveDir)%(Filename)%(Extension)')" SkipUnchangedFiles="true" />
  </Target>
</Project>

• Build

[nulltoken]

Building the package generates the following message

WARNING: 2 issue(s) found with package 'LibGit2Sharp'.

Issue: Assembly outside lib folder.
Description: The assembly 'NativeBinaries\amd64\git2-12f831f.dll' is not inside the 'lib' folder and hence it won't be added as reference when the package is in stalled into a project.
Solution: Move it into the 'lib' folder if it should be referenced.

Issue: Assembly outside lib folder.
Description: The assembly 'NativeBinaries\x86\git2-12f831f.dll' is not inside the 'lib' folder and hence it won't be added  as reference when the package is installed into a project.
Solution: Move it into the 'lib' folder if it should be referenced.

Question:
Where should those native dlls go? Within build\native directory structure?

[nulltoken]

Ensure that this is compatible is VS Publish mechanism (cf. projectkudu/kudu#943)

[nulltoken]

For reference: Number of LibGit2Sharp downloads per client NuGet version

Seems like 2.7 is now mostly used, so it might be "safe" to merge this PR without bothering too many users.

@davidebbo Do you know if dealing with NativeBinaries this way would be kudu compliant?

[davidebbo]

It should. You can test it without Kudu by following these steps. If you see the native binaries in the output folder, then it will almost certainly work in Kudu as well.

[nulltoken]

@davidebbo I've rebased this PR and generated a fake v0.15.1 NuGet package based on it.

I've created a plain ASP.Net MVC web application and referenced this fake package from a local feed. Building the project copies the LibGit2Sharp.dll and the NativeBinaries hierarchy under bin, as expected. However, publishing only considers the managed assembly.

Could you please help me troubleshoot where I messed up, please? The fake package's available for download here.

Below the Publish output

Connecting to D:\temp\tttr...
Transformed Web.config using c:\users\emeric\documents\visual studio 2013\Projects\WebApplication1\WebApplication1\Web.Release.config into obj\Release\TransformWebConfig\transformed\Web.config.
Copying all files to temporary location below for package/publish:
obj\Release\AspnetCompileMerge\Source.
C:\Windows\Microsoft.NET\Framework\v4.0.30319\aspnet_compiler.exe -v / -p "c:\users\ntk\documents\visual studio 2013\Projects\WebApplication1\WebApplication1\obj\Release\AspnetCompileMerge\Source" -u "c:\users\emeric\documents\visual studio 2013\Projects\WebApplication1\WebApplication1\obj\Release\AspnetCompileMerge\TempBuildDir" 
Copying all files to temporary location below for package/publish:
obj\Release\Package\PackageTmp.
Deleting existing files...
Publishing folder /...
Publishing folder bin...
Publishing folder Content...
Publishing folder fonts...
Publishing folder Scripts...
Publishing folder Views...
Publishing folder Views/Account...
Publishing folder Views/Home...
Publishing folder Views/Shared...
Site was published successfully file:///D:/temp/tttr

[davidebbo]

I tried it, but the package doesn't have any Tools folder. Try installing Microsoft.SqlServer.Compact in the same test project to compare:

• it has install/uninstall scripts in tools
• the script adds the native binaries to a _bin_deployableAssemblies folder, which is a magic folder that makes these things work.

Maybe starting with these scripts and modifying them with the relevant list of binaries is the simplest approach.

[davidebbo]

Hmmm, though it seems the sucky part with that approach is that users would end up having to commit all those binaries in _bin_deployableAssemblies, which is really evil. :(

@jeffhand @howarddierking is there better guidance for writing packages that need native binaries? I searched for it and found this issue, which ironically I had filed myself 3 years ago! :)

[nulltoken]

I tried it, but the package doesn't have any Tools folder

@davidebbo Actually, that was the very purpose of this PR: to remove the powershell scripts 😉

@fearthecowboy Can you think of a NuGet tweak that would allow us to not rely on powershell scripts to deploy the native binaries while still being Publish compliant?

[fearthecowboy]

Creating a .targets file is the best way to handle this (without having a .ps1 file). Files that are not expected to be referenced (but rather, copied at build time) should be in the /build/<framework>/... folder in the package.

[davidebbo]

@fearthecowboy do you know if there is a sample package that does this correctly today? I thought Microsoft.SqlServer.Compact was the 'standard' for dealing with native binaries, but it's using a .ps1 file.

[fearthecowboy]

I don't know any example off the top of my head. I don't know that anyone sat down and cranked one out.

What are the requirements for being "Publish compliant?" Is there an msbuild script being run at that point?

Shouldn't it just boil down to adding some items to an already existing itemgroup?

[davidebbo]

At a high level, being 'publish compliant' simply mean that when you deploy (e.g. to file system or via WebDeploy), the native binaries end up being deployed in the correct location in bin. And this should be done without having to commit the binaries into the bin folder of the app.

[fearthecowboy]

Ah, and the 'deploy' is handled via a Target in the msbuild script I assume? (I've been hiding away from building webapps for the last few years :D )

So if a given package had the extra (native) items in the right item group, when the consuming project was 'deployed' the items would get copied to the appropriate location.

Sounds easy enough. Lemme see if I can find some time this week to play with a web project that deploys that way and build you a proof of concept package.

[davidebbo]

Yes, that's correct, it uses some magic target files. e.g. look under \Program Files (x86)\MSBuild\Microsoft\VisualStudio\v12.0\WebApplications. Thanks for helping out!

[amibar]

@davidebbo, @nulltoken, @fearthecowboy.
I investigated a little bit and come up with a solution to the Web Application publishing.
See PR #615

[nulltoken]

Superseded by #821.