TypeLoadException from dynamically-generated assembly; does not occur from the same assembly loaded from disc

[masonwheeler]

Description

While testing the Boo compiler's System.Reflection.Metadata port, I ran across a very strange error. When creating two assemblies in-memory, A and B, where A is created first and B has a dependency on it, if B attempts to load an Attribute defined on A, it will throw a TypeLoadException. However, if both assemblies are saved to disc and loaded as project references of assembly C, all the types load as expected.

Both assemblies pass ILVerify, and the process works just fine when loaded from disc. This suggests that the problem is not in the assemblies, but rather the runtime. Either that or there's some obscure corner case with my code generation?

Reproduction Steps

1. Clone this commit from the in-progress codegen work.
2. Open src\Boo.Core.sln in Visual Studio. Restore NuGet packages and build. (Project "Test" will not build yet. Ignore it.)
3. Run the varargs_attribute_external test. Observe TypeLoadException being thrown when it tries to access the VarArgsAttribute class.
4. Grab this zipfile, containing the two assemblies built by this testcase. Plug them in as the broken dependencies on project "Test," and build it.
5. Run Test, which duplicates the basic functionality of the testcase here. See that everything runs as expected.

Expected behavior

In both cases, the same output should be generated.

Actual behavior

Step 5 works as expected, but step 3 crashes.

Regression?

This worked as expected under Framework when generating the assemblies with System.Reflection.Emit.

Configuration

.NET 5, Windows 10, x64

Other information

It would be a lot easier to debug this if TypeLoadException would provide some relevant information regarding what went wrong.

[ghost]

Tagging subscribers to this area: @vitek-karas, @agocke, @VSadov
See info in area-owners.md if you want to be subscribed.

Issue Details

---

Description

While testing the Boo compiler's System.Reflection.Metadata port, I ran across a very strange error. When creating two assemblies in-memory, A and B, where A is created first and B has a dependency on it, if B attempts to load an Attribute defined on A, it will throw a TypeLoadException. However, if both assemblies are saved to disc and loaded as project references of assembly C, all the types load as expected.

Both assemblies pass ILVerify, and the process works just fine when loaded from disc. This suggests that the problem is not in the assemblies, but rather the runtime. Either that or there's some obscure corner case with my code generation?

Reproduction Steps

1. Clone this commit from the in-progress codegen work.
2. Open src\Boo.Core.sln in Visual Studio. Restore NuGet packages and build. (Project "Test" will not build yet. Ignore it.)
3. Run the varargs_attribute_external test. Observe TypeLoadException being thrown when it tries to access the VarArgsAttribute class.
4. Grab this zipfile, containing the two assemblies built by this testcase. Plug them in as the broken dependencies on project "Test," and build it.
5. Run Test, which duplicates the basic functionality of the testcase here. See that everything runs as expected.

Expected behavior

In both cases, the same output should be generated.

Actual behavior

Step 5 works as expected, but step 3 crashes.

Regression?

This worked as expected under Framework when generating the assemblies with System.Reflection.Emit.

Known Workarounds

No response

Configuration

.NET 5, Windows 10, x64

Other information

It would be a lot easier to debug this if TypeLoadException would provide some relevant information regarding what went wrong.

Author:	masonwheeler
Assignees:	-
Labels:	area-AssemblyLoader-coreclr, untriaged
Milestone:	-

[webczat]

how do you load the assemblies? note metadata doesn't load.

[masonwheeler]

@webczat Code for loading generated assembly in memory. Also, code for saving assembly to disc, in case it's relevant. They're basically the same thing, except in one case the PE blob bytes get written to a file, and in the other they get passed to Assembly.Load.

[masonwheeler]

@jkotas @MichalStrehovsky Ping! This issue may or may not be more relevant to you guys than to the people msftbot pinged.

[webczat]

not fully sure how that loading works when it goes to dependency loading, but:
first, make sure you load them in the correct order because obviously, it cannot autoload a dependency.
Second, if it is the case... can you check/verify what is the assembly load context the new assembly lands into? the documentation for Load(byte[]) method claims to always create a new assembly, so that means they might not see each other by name and might have different ALC. You could try loading via AssemblyLoadContext.Default to make sure, but I'd first verify my claims.

[masonwheeler]

first, make sure you load them in the correct order because obviously, it cannot autoload a dependency.

Not a problem here. The first assembly gets built in-memory, then runs. It calls into the compiler to build the second one and sets itself as a dependency of the second one, which it's building.

Second, if it is the case... can you check/verify what is the assembly load context the new assembly lands into?

According to this page, "assemblies that are loaded from byte arrays are loaded without context unless their identity (after policy is applied) establishes that they are in the global assembly cache." (Which doesn't apply here.) So if both of them are "loaded without context," what are the implications?

[webczat]

i don't mean these contexts. although this doc suggests your use case should work so someone else could comment on that. AssemblyLoadContext is a completely new dotnet core concept.

[jkotas]

Observe TypeLoadException being thrown when it tries to access the VarArgsAttribute class.

I am observing "System.IO.FileNotFoundException: Could not load file or assembly 'testcase...`.

"assemblies that are loaded from byte arrays are loaded without context unless their identity (after policy is applied) establishes that they are in the global assembly cache." (Which doesn't apply here.) So if both of them are "loaded without context," what are the implications?

Correct, assemblies loaded from byte arrays always go into own context in .NET Core. The documentation says "Other assemblies cannot bind to assemblies that are loaded without context, unless you handle the AppDomain.AssemblyResolve event.". Are you handling the AssemblyResolve event to somewhere in your test harness?

[masonwheeler]

I am observing "System.IO.FileNotFoundException: Could not load file or assembly 'testcase...`.

Weird! What's the stack trace?

Are you handling the AssemblyResolve event to somewhere in your test harness?

I just tried adding the following to the run code:

ResolveEventHandler resolver =
	(_, args) => { 
		return args.Name == asm.FullName ? asm : null; 
	};
AppDomain.CurrentDomain.AssemblyResolve += resolver;
try
{
//actual run code here
} 
finally
{
	AppDomain.CurrentDomain.AssemblyResolve -= resolver;
}

Placing a breakpoint in the body of resolver, it never triggers. Is there something I need to do differently, or...?

[jkotas]

Weird! What's the stack trace?

[masonwheeler]

@jkotas Very strange. Here's what I'm getting:

D:\GitHub\boo\tests/testcases\integration/attributes\varargs-attribute-external.boo: BCE0011: Boo.Lang.Compiler.CompilerError: An error occurred during the execution of the step 'Boo.Lang.Compiler.Steps.RunAssembly': 'Exception has been thrown by the target of an invocation.'.
 ---> System.Reflection.TargetInvocationException: Exception has been thrown by the target of an invocation.
 ---> System.TypeLoadException: Could not load type 'VarArgsAttribute' from assembly 'testcase, Version=0.0.0.0, Culture=neutral, PublicKeyToken=null'.
   at System.ModuleHandle.ResolveType(QCallModule module, Int32 typeToken, IntPtr* typeInstArgs, Int32 typeInstCount, IntPtr* methodInstArgs, Int32 methodInstCount, ObjectHandleOnStack type)
   at System.ModuleHandle.ResolveTypeHandleInternal(RuntimeModule module, Int32 typeToken, RuntimeTypeHandle[] typeInstantiationContext, RuntimeTypeHandle[] methodInstantiationContext)
   at System.Reflection.RuntimeModule.ResolveType(Int32 metadataToken, Type[] genericTypeArguments, Type[] genericMethodArguments)
   at System.Reflection.CustomAttribute.FilterCustomAttributeRecord(MetadataToken caCtorToken, MetadataImport& scope, RuntimeModule decoratedModule, MetadataToken decoratedToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1& derivedAttributes, RuntimeType& attributeType, IRuntimeMethodInfo& ctorWithParameters, Boolean& isVarArg)
   at System.Reflection.CustomAttribute.AddCustomAttributes(ListBuilder`1& attributes, RuntimeModule decoratedModule, Int32 decoratedMetadataToken, RuntimeType attributeFilterType, Boolean mustBeInheritable, ListBuilder`1 derivedAttributes)
   at System.Reflection.CustomAttribute.GetCustomAttributes(RuntimeType type, RuntimeType caType, Boolean inherit)
   at System.Attribute.GetCustomAttributes(MemberInfo element, Type type, Boolean inherit)
   at System.Attribute.GetCustomAttribute(MemberInfo element, Type attributeType, Boolean inherit)
   at System.Attribute.GetCustomAttribute(MemberInfo element, Type attributeType)
   at Varargs_attribute_externalModule.Main(String[] argv)
   --- End of inner exception stack trace ---
   at System.RuntimeMethodHandle.InvokeMethod(Object target, Object[] arguments, Signature sig, Boolean constructor, Boolean wrapExceptions)
   at System.Reflection.RuntimeMethodInfo.Invoke(Object obj, BindingFlags invokeAttr, Binder binder, Object[] parameters, CultureInfo culture)
   at System.Reflection.MethodBase.Invoke(Object obj, Object[] parameters)
   at Boo.Lang.Compiler.Steps.RunAssembly.Run() in D:\GitHub\boo\src\Boo.Lang.Compiler\Steps\RunAssembly.cs:line 53
   at Boo.Lang.Compiler.CompilerPipeline.RunStep(CompilerContext context, ICompilerStep step) in D:\GitHub\boo\src\Boo.Lang.Compiler\CompilerPipeline.cs:line 323
   --- End of inner exception stack trace ---

Very similar to yours, but breaking for a different reason.

It looks like you have the generated assembly loaded, from the line mentioning Varargs_attribute_externalModule. Can you run under the debugger, break at the exception, and see if testcase is listed in VS's Modules window?

[jkotas]

I do see testcase listed in VS module window.

I am running the repro on .NET Core 5.0.12. What is the exact runtime version that the test is running on your machine?

[masonwheeler]

@jkotas OK, I just found the discrepancy. Apparently I had an old version of testcase sitting around in the output folder, and the loader was loading that. After deleting it, I see the same error you do. 🤦‍♂️

My assembly resolver is still never being called, though. You can verify this in src\Boo.Lang.Compiler\Steps\RunAssembly.cs. Modify Run() as described above. Any idea what's going wrong there?

[jkotas]

The version number in the error message looks odd: 0.0.65535.65535. I think it may be the problem.

[masonwheeler]

That's actually correct. It shows up in the version metadata if you decompile it with ILDASM and look at the MANIFEST. This appears to be the default; figuring out how to get SRM to produce a better version number is on the to-do list.

[webczat]

well that is the thing I can tell you.. version number is always x.x.x.x so it fills missing pieces. 📦

[masonwheeler]

So why does it fill in the first two with 0 and the last two with -1 by default? And how do I plug in my own numbers? It kinda looks like they should go in PEHeaderBuilder, but I haven't been able to find values that actually change the output of this. (Admittedly I haven't tried very much; I've been focused on getting the test cases working first.)

[webczat]

aaaah. no no it's not pe builder, it's assembly manifest. add a correct entry to the assembly table, the AddAssembly method has a version parameter.

[masonwheeler]

OK, after changing that to use new Version(1, 0, 0, 0), (src\Boo.Lang.Compiler\Steps\EmitAssemblyCore.cs, line 5139,) the behavior is the same. The error message shows the updated version number, but nothing else has changed. So @jkotas, I think we can rule out the version number as the cause of this problem.

[jkotas]

Are you handling the AssemblyResolve event to somewhere in your test harness?

I just tried adding the following to the run code:

Can you try again? The AssemblyResolve event is getting hit for me.

[masonwheeler]

Bleh.

Thanks for the help. It was running afterall; not sure why the breakpoint wasn't triggering. Must be Visual Studio being flaky. I've been able to resolve this locally.