Add build support for .NET core

[szekerest]

It should be an additional option for the C# builds

[szekerest]

@rouault .Net Standard is considered to be a subset of .Net core and any other framework implementations. By targeting .NET Standard, you can build libraries that you can share across all your .NET apps, no matter on which .NET implementation or OS they run. This is basically for class libraries (like for gdal_csharp.dll and ogr_csharp.dll). The console applications should target .NET Core as a minimum requirement.

For more info see: https://msdn.microsoft.com/en-us/magazine/mt842506.aspx

[evilpilaf]

Is there any progress on this?

[flensrocker]

This gets a little bit urgent.

Spatial support was added to EF Core 2.2, which is EOL - and Asp.Net Core 3.1 (the current LTS release) is without support for the classic .Net Framework.

So, going back to 2.1 (the last LTS) is not an option.

Being stuck to the 2.2 release is not good, since there was recently a security patch on the 2.1 release, which will not get ported to 2.2.

Porting this library to .Net Standard 2.0 should be the way to go.

[jsmith-pathway]

@szekerest @rouault Will this be in one of the upcoming releases?

[BrannonKing]

Batchfile code I'm using to generate the .NET Standard 2.0 files:

rem Build the old wrap and csharp DLLs
cd swig\csharp
nmake -f makefile.vc interface MSVC_VER=1920 WIN64=1
nmake -f makefile.vc ogr_dir gdal_dir osr_dir const_dir gdal_csharp MSVC_VER=1920 WIN64=1
nmake -f makefile.vc install MSVC_VER=1920 WIN64=1

rem Remove the .NET Framework files
del "%GDAL_HOME%\csharp\*_csharp.dll"

rem Now make the .netstandard wrappers
setlocal ENABLEDELAYEDEXPANSION
set PROJECT_FILE_DEF=^<Project Sdk="Microsoft.NET.Sdk"^>^<PropertyGroup^>^<TargetFramework^>netstandard2.0^</TargetFramework^>^<AssemblyVersion^>3.1.2.0^</AssemblyVersion^>^</PropertyGroup^>^</Project^>
echo !PROJECT_FILE_DEF! > const\gdalconst_csharp.csproj
echo !PROJECT_FILE_DEF! > osr\osr_csharp.csproj
echo !PROJECT_FILE_DEF! > ogr\ogr_csharp.csproj
echo !PROJECT_FILE_DEF! > gdal\gdal_csharp.csproj
endlocal

cd const
dotnet build -c Release -o "%GDAL_HOME%\csharp" -r win-x64
cd ..
cd osr
dotnet build -c Release -o "%GDAL_HOME%\csharp" -r win-x64
cd ..
cd ogr
dotnet add reference ..\osr\osr_csharp.csproj
dotnet build -c Release -o "%GDAL_HOME%\csharp" -r win-x64
cd ..
cd gdal
dotnet add reference ..\osr\osr_csharp.csproj ..\ogr\ogr_csharp.csproj
dotnet build -c Release -o "%GDAL_HOME%\csharp" -r win-x64
cd ..\..\..\..\..

[szekerest]

@BrannonKing Thank you for sharing this, it looks very cool. I'll be trying to incorporate it in the makefiles

[BrannonKing]

A few notes:

1. I don't think there is any need to keep building the old .NET Framework DLLs as the .NET Standard will work fine in its place (unless you're using a version of .NET older than 4.6, which would be rare).
2. It would be worth understanding the *.vc files here: https://github.com/MaxRev-Dev/gdal.netcore
3. Having the *_wrap.DLL and the *_csharp.DLL both get built in the same make step was problematic for me (requiring me to delete the latter later).
4. Is it possible to make SWIG generate a pure P/invoke wrapper, thus avoiding the *_wrap.dll files altogether?
5. You can replace the "win-x64" with other RID values for targeting Linux, etc.
6. I wasn't sure how to read in the current version (as seen near AssemblyVersion). I was confused as to why the code in gdal_version.h stayed commented out during build.
7. My full script also includes this line:
   xcopy /D /Y "%VCToolsRedistDir%x64\Microsoft.VC142.CRT\*.dll" "%GDAL_HOME%\bin"

[BrannonKing]

The recent changes you put in are close. It looks like there is an issue with copying (installing) the *_csharp.dll files in that the makefile expects them to be in the csharp folder. In actuality they are in csharp/ogr, csharp/ogs, csharp/gdal, etc.

[szekerest]

Good point about the install target. I've never used that. Applying a fix for the problem.

[BrannonKing]

My previous statement wasn't entirely correct. It looks like those csproj files by default install files to .\bin\x64\Release\netstandard2.0\win-x64

[szekerest]

@BrannonKing I don't see the problem. The build commands contain -o "." with causes that the result is generated into the same directory as the sources.

[BrannonKing]

Looking at this again today, the -o parameter does something to botch the build. When I compile with -o I get a 3KB gdal_csharp.dll file. When I compile without it I get the right size of file in path I mentioned above.

[jany-tenaj]

Is there any chance that this will be released any time soon?

[runette]

@rouault @szekerest - I was looking through any other issues that should be included in #5046.

Given that the new CMake build scripts are building on Net5.0 - should we close this issue or move any remaining concerns to #5046 ?

[runette]

Is there any chance that this will be released any time soon?

@jany-tenaj - can you be more explicit about what you are expecting to be released?

The main repo makefile.vc on Windows has been building on NET Standard 2.0 and Net Core for a couple of years now?
BTW - as per https://gdal.org/development/rfc/rfc84_cmake.html - the build process is moving to CMake and there is an open PR #5046 addressing the CMake build scripts. I would expect any enhancements to C# build to happen there and not here.

[jany-tenaj]

@runette we use the nuget packages for Gdal, Gdal.Plugins and Gdal.Native in .Net 4.7.2. I was trying to move our source code to .Net 6, but could not add the nuget packages. At the time I left my comment this here read like it could be what we needed.

[runette]

@jany-tenaj The NuGET packages are not managed as part of the GDAL repository - you need to talk to the maintainers directly and not through here. As far as GDAL is concerned, the change from .NET 4.7.2 was "released" a long time ago.

When the CMake build scripts are merged - you will be able to locally create NuGET packages based on .NET 5. Keep an eye on that PR I linked to.

[szekerest]

@jany-tenaj I'm planning to create a netcore nuget build for Gdal, Gdal.Plugins and Gdal.Native, but there are a couple of issues I prefer to address before those packages are released.

[jany-tenaj]

@szekerest that sounds good. Any idea how long we'll have to wait?

[MaxRev-Dev]

Hi everyone, I'm the maintainer of unofficial gdal.netcore packages.
Not so long ago, I've released a new version (v3.3.3) of bindings.

It all started as a pet project, and currently, I'm trying to build them on my own in my free time. I do not know if you would be interested in contributing, but asking is better than assuming.
These packages are targeting all common .net TMFs and have some test codebase even with Azure Functions. There are no x86 binaries and no MacOS targets. Also, there are some missing drivers like NetCDF (some linking errors) or SDK specific like Teigha and by now there are no complex CI workflows. Overall, everything can be improved over time.

A huge thanks to the maintainers of this repo, for the frequent updates and continuous support.
I know that support of the infrastructure takes a big amount of work. Bindings are out of the main geospatial scope to cover them also, so there can be a separate repository, aka packages build engine to handle this.
Tell me what you think or maybe we can create some discussion on this?

[jsmith-pathway]

Historically I've gone to @szekerest and his maintained site for the .net packages and wrappers. I think what he has done is what is most widely used. I may be wrong, but those wrappers are created via swig in this repository and get daily built for ease of use through that site.
I think @szekerest also updates the Gdal, Gdal.Plugins, and Gdal.Native nuget packages. But, I'm not sure where the code is to build those packages and what needs to happen for them to be updated for .net 5/6. It'd be nice if that was the primary place where things were updated.
I would like it to be in one location that I could track and possibly contribute to in the future.

[szekerest]

@jsmith-pathway The current versions of the packaging scripts for Gdal, Gdal.Plugins, and Gdal.Native can be found here https://github.com/gisinternals/buildsystem/tree/refactor/nuget
The code was originally contributed by Felix Obermaier and the SharpMap Team and it need a bit of modifications to be able to work with the new set of bineries and the gdal .net core build. @jany-tenaj In the next few days I'll modify the scripts to be able to get the nuget packages released with the latest GDAL version.

However I'm still uncertain about what is the most reasonable concept to release the GDAL environment including a number of unmanaged dlls in nuget packages. With these packages an initialization source code is also emitted to the target project to set up the necessary environment settings, but it might not be so convenient in most cases.
@MaxRev-Dev How your packages address such problems?

[MaxRev-Dev]

@szekerest yes, I'm familiar with the gisintenals build system. It wrote on VB, uses compile-time templates, and nuspec targets. For me, it's way too complicated to maintain but still, it's a good automation tool.
I'm using only 2 scripts to configure gdal startup - one for gdal plugins and one for proj. They handle many cases of sources lookup on various deployment environments (docker, azure, testhosts, vsdebug, etc.).

So about initialization - all core logic and C# bindings are shipped in the main MaxRev.Gdal.Core package. Please note, there is no hardcoded list of drivers in build pipeline (like in nuspec), only in build requirements shared between make/nmake platforms.
During the build (after compilation of sources) all binaries (.dll or .so) are gathered in the separated directory of the target runtime and the Proj database is also shipped with those binaries. In my opinion, after some tests and trials, it's the best way to deploy the binaries, as it does not require after deployment code generation.
Also, there was an idea to ship minimal/full versions of drivers included optimizing the size of the package. By now there were no complaints about it, only for adding some additional drivers.

[jany-tenaj]

@szekerest We're using the initialization C# file that is shipped with the nuget package. Based on the fact that there isn't much gdal documentation for C# I'm glad we didn't have to figure that out ourselves. ;) The vb version we simple delete. It would be nice if the vb version wouldn't be added to C# projects.

I think if you add those initialization files people have a chance to get started. Those who don't need them because they have their own can delete them. If the way things are done changes, we don't have to worry why the old code is no longer working, but get the new code that works.

[szekerest]

@MaxRev-Dev Thank you, it looks like your startup code in compiled into the dll instead of adding that in the source code. I'm not entirely sure which one is the better, but I'd somewhat prefer not to have a startup code at all, assuming the binaries can be deployed in relative locations to the executable. I'd also keep support for both the .net framework and .net core with both x86 and x64 versions in the package, but it is still not trivial to me how the AnyCPU setting is handled with a .net core build. Also referring to dll-s targeting the netstandard runtime doesn't seem to be supported with .net framework earlier than 4.6.1 so I'd probably need to provide a .net 4.x compatible version side by side.

[szekerest]

@jany-tenaj That is relatively easy to implement in a custom script for the project that using the older packages.config method. The projects using the newer PackageReference approach, are supported by the contentFiles item in nuspec which considers the language dependency.

[MaxRev-Dev]

startup code in compiled into the dll instead of adding that in the source code

@szekerest After bootstrapping, it just does not create a mess in the user's project folder. As it shipped as a library, all code should be internal and closed for changes. Additional configuration can be provided, for example, via a fluent API pattern.

how the AnyCPU setting is handled with a .net core build

By default, all projects are configured with AnyCPU. At startup, this configuration performs a lookup for target platform binaries.
We can use architecture specific folders to package binaries. But I prefer to ship binaries in separate packages, for example Gdal.Windows and Gdal.Linux. Linux runtime has a bunch of additional dynamic libraries and it weights 5 times as Windows binaries.

Anyway, you can combine target architectures in a package as you want. Ideally, the package folder structure looks like this:

\runtimes           # GDAL.Windows
    \win10-x64
    \win10-x86
    \net46          # net framework 4.6 binaries 
\runtimes           # GDAL.Linux 
    \linux-x64
    \linux-arm 

Useful sources:
Runtime identifiers: https://docs.microsoft.com/en-us/dotnet/core/rid-catalog
Support multiple .NET versions: https://docs.microsoft.com/en-us/nuget/create-packages/supporting-multiple-target-frameworks

[senritsu]

Having used the package by @MaxRev-Dev, I quite enjoyed that it just works ™ without any project folder pollution, dlls to take care of, or anything else really. It was way nicer to use than previous ways of interfacing with Gdal we used at our company, where we typically had some issues sooner or later. Just a random observation since the topic of ease of use and cleanliness came up.

[szekerest]

@MaxRev-Dev Thanks for the information and the links.

@szekerest After bootstrapping, it just does not create a mess in the user's project folder. As it shipped as a library, all code should be internal and closed for changes. Additional configuration can be provided, for example, via a fluent API pattern.

How the fluent api pattern is currently implemented in your packages? Which is the target interface on which the pattern is being executed? Are you using something exising like IHostBuilder?

By default, all projects are configured with AnyCPU. At startup, this configuration performs a lookup for target platform binaries. We can use architecture specific folders to package binaries. But I prefer to ship binaries in separate packages, for example Gdal.Windows and Gdal.Linux. Linux runtime has a bunch of additional dynamic libraries and it weights 5 times as Windows binaries.

This probably depends on how the compiler is invoked. I mostly use 'dotnet build' in the build scripts and that seems to rely on the Platform environment setting by default. For example an x64 Visual Studio Command Prompt builds x64 assemblies by default. However that can be controlled by adding the corresponding setting in the csproj file or in the build command line. There are many further issues I always encounter when building just a simple project and the documentation is very poor on that. For example if I specify the output folder to the current folder it doesn't compile the sources in the project directory:

mkdir sampe
cd sample
dotnet new classlib
dotnet build -o "."

The resulting assembly will not contain the code in the sample namespace. However specifying a directory like -o "..\build" would work.

[szekerest]

I think the current build is now working with either netcore and netframework 4.6.1 or above. The corresponding nuget packages GDAL, GDAL.Native and GDAL.Plugins have now been pushed. The new cmake build system (targeting GDAL 3.5) will replace this nmake based build configuration, but we strive to make the result compatible. Closing this issue now.

[mumumi]

Use Gdal.Native to convert data into a dxf file. The EntityHandle is 0 (approximately on line 1150). In dxf syntax, 0 means unknown. AutoCad fails to open. When using the same version of ogr2ogr.exe for conversion without any parameters, the EntityHandle value in properties is not correctly obtained either(seems like the effect of default value of unset-fid option). However, it is in line with the syntax and AutoCad can open normally. The main code is as follows:
using var oSrc = driver.CreateDataSource(outputPath, outputDataSourceOptions);
using var oLyr = oSrc.CopyLayer(iLyr, layerName, outputLayerOptions);
{"type":"FeatureCollection","features":[{"type":"Feature","id":0,"properties":{"SubClasses":"AcDbEntity : AcDbLwPolyline","Layer":"mylayer","EntityHandle":"12E0"},"geometry":{"type":"Polygon","coordinates":[[[1,1],[1,2],[2,2],[2,1],[1,1]]]}}]}