Environment Variable configuration provider doesn't work if the prefix has 2 underscores

[Emzi0767]

Description

Environment variable loader doesn't load variables if the prefix is set to a string with one or more double underscores.

The problem lies in the incorrect order of operations here:

runtime/src/libraries/Microsoft.Extensions.Configuration.EnvironmentVariables/src/EnvironmentVariablesConfigurationProvider.cs

Lines 53 to 56 in 98eecc7

	IEnumerable<DictionaryEntry> filteredEnvVariables = envVariables
	.Cast<DictionaryEntry>()
	.SelectMany(AzureEnvToAppEnv)
	.Where(entry => ((string)entry.Key).StartsWith(_prefix, StringComparison.OrdinalIgnoreCase));

The keys are first normalized (that is __ is replaced with separator; default :), then they are filtered. The operations should happen the other way around.

To reproduce the issue:

1. Create a new ASP.NET Core application
2. In Startup.cs, in the constructor, add the following code:

var cfg = new ConfigurationBuilder().AddEnvironmentVariables("TEST__").Build();
Console.WriteLine(cfg["Test1:Test2"] ?? "<null>");

3. In project properties, under Debug, add the following environment variable: TEST__TEST1__TEST2 with a value of 42.
4. Run the project.

Expected result

Console prints 42.

Actual result

Console prints <null>.

Configuration

This affected my application running on ASP.NET Core 3.1.7, running on Windows 10 Pro 2004 x64, as well as Alpine Linux 3.12.0 x64 (running in Docker).

Browsing source of the master branch reveals the buggy code remains, which suggests the bug is present in .NET 5 as well.

Regression?

Not as far as I can tell.

Other information

Tried on Core 3.1, the problem appears to be present in .NET 5 as well.

Currently, the problem can be mitigated by replacing all instances of __ in the prefix with : or whatever other separator was configured, if one was configured. Modifying line 1 from repro to .AddEnvironmentVariables("TEST:") will cause the program to print 42 instead of <null>.

This caveat is not documented, and therefore I consider it a bug.

The issue might not be trivial to fix, as I've noticed that several other prefixes are tested as well.

In my code, I decided to use __ for consistency (so that Test1:Test2 becomes TEST__TEST1__TEST2 rather than TEST_TEST1__TEST2).

[Dotnet-GitSync-Bot]

I couldn't figure out the best area label to add to this issue. If you have write-permissions please help me learn by adding exactly one area label.

[ghost]

Tagging subscribers to this area: @maryamariyan
See info in area-owners.md if you want to be subscribed.

[eerhardt]

Moving to 6.0 as this isn't a regression in 5.0, and isn't critical to fix for 5.0.

[am11]

Unless there is a reason to confine it to strictly one or two underscores, it would probably make sense if the solution be bit more versatile like handling more than two underscores and mixed cases like prefix: _____EXPERIMENTAL__ key: FOO.