PublishSingleFile with Windows Service doesn't work with config-file

[ststeiger]

The PublishSingleFile option is rubbish.
I have a windows-service - simplified code posted below.
That windows service has an appsettings.json file, posted below.
In that config file, I can set the the query interval and the GlobalMemoryStatusEx memory limit.

The code of the service is basically

If GlobalMemoryStatusEx.Load > KillSettings.KillChromeLoadRatio 
then kill all chrome instances running on the machine
wait for KillSettings.MeasureInterval milliseconds before repeat

As you can see in the code, I listen for changes in the appsetings.json file, and update the two member variables m_maxLoadRatioBeforeKill and m_measureInterval (by the way this bug from Jan 2 2018 is still not fixed, and ms-bot closed and blocked the issue... if you/Microsoft want to destroy github-issues and github, just keep that up... are you seriously trying to reanimate microsoft-connect ? Aka Closed as won't fix ?).

Now this works fine (after
private async System.Threading.Tasks.Task IgnoreIrrelevantFileAttributeUpdate(), which is obviously an open issue with filewatcher since ancient times, but not my actual problem)

Now, if I set PublishSingleFile to true, either there is no appsettings.json in the publish directory, and appsettings.json is zipped together with the executable and copied to whereever shadowcopy copies that crap.

or I set

    <ItemGroup>
        <Content Update="appsettings.json">
            <CopyToPublishDirectory>PreserveNewest</CopyToPublishDirectory>
            <ExcludeFromSingleFile>true</ExcludeFromSingleFile>
        </Content>
    </ItemGroup>

as discussed here and here.

Now I have the appsettings.json file separately, but...
Now there is no appsettings.json file read (or shadow-copied), and the service just executes at an interval of 0 millisenconds...
Hilarious - if I hadn't to get it to work NOW...

How is that supposed to work if I update the config file in the install directory of the single-file windows service when it goes into production ? (Note: the install directory, not the shadow-copy directory)

Maybe instead of zipping and shadow-copying, you should actually do static linking, so you don't need to shadow-copy ?
You know, like in golang ?
Because if you don't, you'll never run out of issues like this.
(The next issue would certainly be native libraries, but as I can see now, it fails long before that )

Also, how can I do this correctly ?
Obviously,
<ExcludeFromSingleFile>true</ExcludeFromSingleFile>
is not the way to go.

So what can I do, when I want to publish as single file >without< loosing the ability to update appsettings.json on the fly ?

appsettings.json

{
  "Logging": {
    "LogLevel": {
      "Default": "Information",
      "Microsoft": "Warning",
      "Microsoft.Hosting.Lifetime": "Information"
    }
  },
  
  "KillSettings": {
    "KillChromeLoadRatio": 79,
    "MeasureInterval": 1000
  }
  
}

Worker.cs

using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Logging;


// http://www.dotnetspeak.com/net-core/creating-schedule-driven-windows-service-in-net-core-3-0/
// https://andrewlock.net/new-in-aspnetcore-3-structured-logging-for-startup-messages/
// https://andrewlock.net/suppressing-the-startup-and-shutdown-messages-in-asp-net-core/


// dotnet restore -r win-x86
// dotnet build -r win-x86
// dotnet publish -f netcoreapp3.1 -c Release -r win-x86
// dotnet publish -f netcoreapp3.1 -c Release -r win-x86 -o D:\inetpub\RamMonitor
// dotnet publish -f netcoreapp3.1 -c Release -r win-x86 -o D:\inetpub\RamMonitor
// dotnet publish -r linux-x64 -c Release /p:PublishSingleFile=true
// dotnet publish -r win-x86 -c Release /p:PublishSingleFile=true

namespace RamMonitor
{
    
    
    public class Worker 
        : Microsoft.Extensions.Hosting.BackgroundService
    {
        
        
        private readonly ILogger<Worker> m_logger;
        private IConfiguration m_configuration;
        private int m_maxLoadRatioBeforeKill;
        private int m_measureInterval;
        private bool m_ignoreChange;
        
        
        public Worker(ILogger<Worker> logger, IConfiguration conf)
        {
            this.m_logger = logger;
            this.m_configuration = conf;
            IConfigurationSection section = conf.GetSection("KillSettings");
            
            this.m_maxLoadRatioBeforeKill = section.TryGetValue<int>("KillChromeLoadRatio", 79);
            this.m_measureInterval = section.TryGetValue<int>("MeasureInterval", 5000);
            
            Microsoft.Extensions.Primitives.ChangeToken.OnChange(
                  delegate() { return this.m_configuration.GetReloadToken(); } 
                , delegate(IConfigurationSection state) { InvokeChanged(state); }
                , section
            );
            
        } // End Constructor 
        
        
        
        private async System.Threading.Tasks.Task IgnoreIrrelevantFileAttributeUpdate()
        {
            await System.Threading.Tasks.Task.Delay(1000);
            this.m_ignoreChange = false;
        }


        private void InvokeChanged(IConfigurationSection section)
        {
            if (this.m_ignoreChange)
                return;
            
            this.m_ignoreChange = true;
            
            this.m_maxLoadRatioBeforeKill = section.TryGetValue<int>("KillChromeLoadRatio", 79);
            this.m_measureInterval = section.TryGetValue<int>("MeasureInterval", 5000);
            
// #pragma warning disable 1998
            IgnoreIrrelevantFileAttributeUpdate();
// #pragma warning restore 1998
            
        }
        
        
        protected override async System.Threading.Tasks.Task ExecuteAsync(System.Threading.CancellationToken stoppingToken)
        {
            IConfigurationSection section = this.m_configuration.GetSection("KillSettings");
            CollectingTextWriter ctw = new CollectingTextWriter();
            
            while (!stoppingToken.IsCancellationRequested)
            {
#if false // for debugging appsettings.json update
                GlobalMemoryMetrics metrics = OsInfo.MemoryMetrics;
                metrics.WriteMemory(ctw);
                
                if (metrics.Load > this.m_maxLoadRatioBeforeKill)
                {
                    this.m_logger.LogInformation("Killing chrome processes at: {time}", System.DateTimeOffset.Now);
                    ProcessManager.KillProcessGroup("chrome");
                    this.m_logger.LogInformation("Killed chrome processes at: {time}", System.DateTimeOffset.Now);
                } // End if (metrics.Load > this.m_maxLoadRatioBeforeKill)
                
                this.m_logger.LogInformation("Worker running at: {time}", System.DateTimeOffset.Now);
                this.m_logger.LogInformation(ctw.StringValue);
#else
                System.Console.WriteLine(this.m_measureInterval);
                System.Console.WriteLine(this.m_maxLoadRatioBeforeKill);
#endif
                
                await System.Threading.Tasks.Task.Delay(this.m_measureInterval, stoppingToken);
                // so, show me the actual values...
                System.Console.WriteLine(this.m_measureInterval); 
                System.Console.WriteLine(this.m_maxLoadRatioBeforeKill); 
            } // Whend 
            
        } // End Task ExecuteAsync 
        
        
    } // End Class Worker  
    
    
}  // End Namespace RamMonitor 

Program.cs

using Microsoft.Extensions.Hosting;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.DependencyInjection;
using Microsoft.Extensions.Logging;
using Microsoft.Extensions.Logging.EventLog;


// Add package Microsoft.Extensions.Hosting.WindowsServices
// version 3.1.0


namespace RamMonitor
{
    
    
    public static class Program
    {
        
        
        public static void Main(string[] args)
        {
            CreateHostBuilder(args).Build().Run();
        } // End Sub Main 


        private static IHostBuilder CreateHostBuilder(string[] args)
        {
            IHostBuilder builder = new HostBuilder();
            
            // builder.UseContentRoot(System.IO.Directory.GetCurrentDirectory());
            // builder.UseContentRoot(System.IO.Path.GetDirectoryName(typeof(Program).Assembly.Location));
            builder.UseContentRoot(System.AppContext.BaseDirectory);
            
            if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows))
            {
                // https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/windows-service?view=aspnetcore-3.1&tabs=visual-studio#app-configuration
                // Requires Microsoft.Extensions.Hosting.WindowsServices
                builder.UseWindowsService();
            } 
            else if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Linux))
            {
                // https://devblogs.microsoft.com/dotnet/net-core-and-systemd/
                // Requires Microsoft.Extensions.Hosting.WindowsServices
                builder.UseSystemd(); // Add: Microsoft.Extensions.Hosting.Systemd
            }
            else if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices
                .OSPlatform.OSX))
            {
                throw new System.NotImplementedException("Service for this Platform is NOT implemented.");
            }
            else
            {
                throw new System.NotSupportedException("This Platform is NOT supported.");
            }

            builder.ConfigureHostConfiguration(config =>
            {
                config.AddEnvironmentVariables(prefix: "DOTNET_");
                if (args != null)
                {
                    config.AddCommandLine(args);
                }
            });

            builder.ConfigureAppConfiguration((hostingContext, config) =>
                {
                    IHostEnvironment env = hostingContext.HostingEnvironment;

                    config.AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                        .AddJsonFile($"appsettings.{env.EnvironmentName}.json", optional: true, reloadOnChange: true);

                    if (env.IsDevelopment() && !string.IsNullOrEmpty(env.ApplicationName))
                    {
                        System.Reflection.Assembly appAssembly =
                            System.Reflection.Assembly.Load(new System.Reflection.AssemblyName(env.ApplicationName));
                        if (appAssembly != null)
                        {
                            config.AddUserSecrets(appAssembly, optional: true);
                        }
                    }

                    config.AddEnvironmentVariables();

                    if (args != null)
                    {
                        config.AddCommandLine(args);
                    }
                })
                .ConfigureLogging((hostingContext, logging) =>
                {
                    bool isWindows = System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(
                            System.Runtime.InteropServices.OSPlatform.Windows
                    );
                    
                    // IMPORTANT: This needs to be added *before* configuration is loaded, this lets
                    // the defaults be overridden by the configuration.
                    if (isWindows)
                    {
                        // Default the EventLogLoggerProvider to warning or above
                        logging.AddFilter<EventLogLoggerProvider>(level => level >= LogLevel.Warning);
                    } // End if (isWindows) 
                    
                    logging.AddConfiguration(hostingContext.Configuration.GetSection("Logging"));
                    logging.AddConsole();
                    logging.AddDebug();
                    logging.AddEventSourceLogger();

                    if (isWindows)
                    {
                        // Add the EventLogLoggerProvider on windows machines
                        logging.AddEventLog();
                    } // End if (isWindows) 
                })
                .ConfigureServices((hostingContext, services) =>
                {
                    // AWSSDK.Extensions.NETCore.Setup
                    // AWSSDK.SQS
                    // Microsoft.VisualStudio.Azure.Containers.Tools.Targets

                    // AWS Configuration
                    // AWSOptions options = hostingContext.Configuration.GetAWSOptions();
                    // services.AddDefaultAWSOptions(options);
                    // services.AddAWSService<IAmazonSQS>();

                    // Worker Service
                    services.AddHostedService<Worker>();
                })
                .UseDefaultServiceProvider((context, options) =>
                {
                    bool isDevelopment = context.HostingEnvironment.IsDevelopment();
                    options.ValidateScopes = isDevelopment;
                    options.ValidateOnBuild = isDevelopment;
                });

            return builder;
        } // End Function CreateHostBuilder 
        
        
        private static IHostBuilder OldCreateHostBuilder(string[] args)
        {
            // return Host.CreateDefaultBuilder(args).ConfigureServices((hostContext, services) => { services.AddHostedService<Worker>(); });

            return Host.CreateDefaultBuilder(args)
                .ConfigureAppConfiguration(
                    delegate(HostBuilderContext hostingContext, IConfigurationBuilder config)
                    {
                        IConfigurationRoot builder = new ConfigurationBuilder()
                            .SetBasePath(System.IO.Directory.GetCurrentDirectory())
                            .AddJsonFile("appsettings.json", optional: true, reloadOnChange: true)
                            .AddJsonFile($"appsettings.{hostingContext.HostingEnvironment.EnvironmentName}.json",
                                optional: true, reloadOnChange: true)
                            .AddEnvironmentVariables()
                            .Build();
                    })
                .ConfigureServices(
                    delegate(HostBuilderContext hostingContext, IServiceCollection services)
                    {
                        // AWSSDK.Extensions.NETCore.Setup
                        // AWSSDK.SQS
                        // Microsoft.VisualStudio.Azure.Containers.Tools.Targets

                        // AWS Configuration
                        // AWSOptions options = hostingContext.Configuration.GetAWSOptions();
                        // services.AddDefaultAWSOptions(options);
                        // services.AddAWSService<IAmazonSQS>();

                        // Worker Service
                        services.AddHostedService<Worker>();
                    }
                );
        } // End Function OldCreateHostBuilder 
        
        
    } // End Class Program 
    
    
} // End Namespace RamMonitor 

[swaroop-sridhar]

@ststeiger In order for the application's settings to be editable by users as a separate file, and survive across app-builds, it is best kept separate from the app. So, excluding the appsettings.json file from the single-file using the ExcludeFromSingleFile is a recommended solution.

Once appsettings.json is a separate file, it needs to be loaded from the correct path. In the Program.cs posted above, when the app.dll tries to load the appsettings.json file, I think it needs to include the single-file app's path when locating the appsettings.json file.

That is, in a single-file app, the location of assemblies is abstract. The fact that assemblies are extracted to a temp-location (I believe this is what you mean by shadow-copy location) is a matter of implementation. In .net 5, there will be no extraction, or a disk-path associated with assemblies bundled into a single-file app.

So, apps need to be aware of the fact that appsettings.json cannot be loaded relative to app.dll location, but relative to the host location (obtainable through Process.GetCurrentProcess().MainModule.FileName).

If you use WebSDK, all this work (including exclusion of appsettings.json from the single-file app) is automatically handled by the SDK. But I understand that your app may need to do these steps on its own.

CC: @vitek-karas @samsp-msft

[ststeiger]

Ah, I see, this is again like in the 2.x series, where the current working directory was set to system32. That got me there, too.
So in that code

string dir = System.IO.Path.GetDirectoryName(typeof(Program).Assembly.Location);
// dir = System.IO.Directory.GetCurrentDirectory();
configHost.SetBasePath(dir);

I'll just have to use Process.GetCurrentProcess().MainModule.FileName instead of Assembly.Location and it will work, and had I used a web-application which uses websdk, it would have worked out of the box.

May I use this opportunity to say I find it strange that the template for worker/service doesn't by default add the Microsoft.Extensions.Hosting.WindowsServices assembly, and call
UseWindowsService if the os is windows.

Not that I mind adding it, but if I need to google it from 3rd party blogs, to get microsoft software to work on the microsoft OS, and in addition have to figure out that since the blog was written, the method's name has changed to "UseWindowsService", then it's badly documented. Why not just include this in the template ?
Anybody who doesn't want it can remove it, then - but that shouldn't be too many.
That way, it would work out of the box by default.
I shouldn't have to google anything for this.

if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows))
{
    // https://docs.microsoft.com/en-us/aspnet/core/host-and-deploy/windows-service?view=aspnetcore-3.1&tabs=visual-studio#app-configuration
    // Requires Microsoft.Extensions.Hosting.WindowsServices
    builder.UseWindowsService();
} 
else if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Linux))
{
    // https://devblogs.microsoft.com/dotnet/net-core-and-systemd/
    // Requires Microsoft.Extensions.Hosting.Systemd
    builder.UseSystemd();
}
// and whatever launchd needs on OSX

[swaroop-sridhar]

I'll just have to use Process.GetCurrentProcess().MainModule.FileName instead of Assembly.Location and it will work, and had I used a web-application which uses websdk, it would have worked out of the box.

Correct, thanks.

May I use this opportunity to say I find it strange that the template for worker/service doesn't by default add the Microsoft.Extensions.Hosting.WindowsServices assembly, and call
UseWindowsService if the os is windows.

Can you please file a CLI template/documentation issue about this in the SDK repo?

[swaroop-sridhar]

Related: #3704

[swaroop-sridhar]

I believe there's no further action required on this issue.

[vitek-karas]

@ststeiger May I ask which template you used to create your service? I want to make sure we find the right people to at least get them your feedback about the template lacking Windows-Service integration.

[swaroop-sridhar]

My understanding was that @ststeiger used worker template. Please confirm it. Thanks.

[ststeiger]

@vitek-karas: Worker-Template in JetBrains Rider, which should use the same template as Visual Studio, assuming the template is licensed to be reused.

Can you please file a CLI template/documentation issue about this in the SDK repo?

OK, filed issue 10724 in SDK repo.

@swaroop-sridhar:

I believe there's no further action required on this issue.

Correct :)
Though, FileSystemWatcher/ChangeToken should be configurable so Microsoft.Extensions.Primitives.ChangeToken.OnChange does not get called twice when I edit a config file in notepad. I googled a bit, and I now understand this is because change 1 is due to content changed, and change 2 is due to notepad updating the lastwritetime attribute of the file. ChangeToken should probably only consider config-file changes if the content changes, not if the file-attributes change, especially if notepad or other text-editors don't do both in 1 transaction.

Though theoretically, since at least theoretically, somebody could be interested in attributes, it should probably be configurable with a default of content-only - for sanity - or with a default to content&attributes for backwards-compatibility.

This should probably be a separate issue though.
Where is the right place to file an issue for Microsoft.Extensions.Primitives.ChangeToken.OnChange ?
https://github.com/dotnet/extensions (Primitives/ChangeToken.cs) or https://github.com/dotnet/corefx (System/IO/FileSystemWatcher.cs) ?

[ststeiger]

Note that System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName is "C:\Program Files\dotnet\dotnet.exe" when debugging (and when executing with dotnet).

So fully correct is:

string executablePath = System.Diagnostics.Process.GetCurrentProcess().MainModule.FileName;
string executable = System.IO.Path.GetFileNameWithoutExtension(executablePath);

if ("dotnet".Equals(executable, System.StringComparison.InvariantCultureIgnoreCase))
{
    builder.UseContentRoot(System.IO.Path.GetDirectoryName(typeof(Program).Assembly.Location));
}
else
{
    builder.UseContentRoot(System.IO.Path.GetDirectoryName(executablePath));   
}