Consider refactoring monomorphizing into a separate step following type_check?

[mitchmindtree]

At the moment, the process of monomorphizing generics is intertwined with the type-checking process.

I'm curious whether this is out of necessity, i.e. do we need to monomorphize in order to successfully type check? Or is it more the case that we can do it at the same time so we might as well?

In the case that it's the latter, I wonder if it might be worth splitting it into a separate step that immediately follows type checking?

The type checking process is quite complex, and as a newcomer to sway-core I've found grokking how the namespace is updated to be particularly tricky (could be just me!).

Building a mental model of how the namespace is updated requires following two axes:

1. through the creation of new scopes and populating the scope's contents and
2. subtle inner mutation within methods like namespace.find_method_for_type due to the inner resolve_type_with_self call that may monomorphize internally.

1. is quite intuitive to follow, but 2. seems a little shoe-horned in and is making #1213 particularly finicky (though I think that PR's still worth tackling before more global mutation sites start cropping up).

I'd imagine having distinct steps might make debugging issues related to type-checking and monomorphizing a little easier too? This is purely speculation though as I'm still new to sway-core.

I'm also unsure just how much work would be involved in such a refactor. I'd imagine at the very least the Namespace::resolve_* methods, and the semantics around TypeIds would need some rethinking.

---

Ahh I just noticed #862 which seems related.

[emilyaherbert]

Also related to:

• Current "inline-everything" approach without context limits debugger support #1527
• Refactor type-checking to not inline everything #1557

[sezna]

It is related to needing to monomorphize in order to successfully type check. Ideally the inference engine is constantly improving its inference during the compilation process, but the original canonical function declarations do not get mutated.

Do you still want to pursue this now that #1213 has gone in? It would be a pretty big type system change to no longer require "passing" type checks and just use unresolved generics until we resolve them afterwards

[tritao]

Just came across this issue after having a bit of a challenge trying to grok how the entire system is working as well, so it's not just you.

Separating the steps seems like it could simplify things, and might even be a necessary step to get #2636 to work with full optimality, because right now when we do monomorphization we still don't know the full inferenced signature, just a partial signature with explicit generic types.

[emilyaherbert]

Note that this include trait constraint solving.

[emilyaherbert]

A potential 80%-baked solution: use de bruijn indices to track generic types (and self types) as a separate entity in the AST in order to separate monomoprhization.

This is what Rust does. See the references here:

• https://rustc-dev-guide.rust-lang.org/ty.html#question-why-not-substitute-inside-the-adtdef
• https://rustc-dev-guide.rust-lang.org/generics.html
• https://doc.rust-lang.org/nightly/nightly-rustc/rustc_middle/ty/subst/type.SubstsRef.html
• https://rustc-dev-guide.rust-lang.org/variance.html?highlight=self%20type#variance-and-object-types

80%-baked solution for implementing "the self type" with de bruijn indices: #3762

• the remaining 20% will be to update the design (probably something akin to the design in Perform monomorphization and constraint solving after type checking #3744 as Remove TypeInfo::SelfType in favor of tracking the self type as a type param #3762 is a precursor to Perform monomorphization and constraint solving after type checking #3744)

80%-baked solution for implementing generic types + self types with de bruijn indices: #3744

• the remaining 20% will be to complete the monomorphization solver and AST pipeline

This solution will subsume writing a monomorphization cache (#2636).

[anton-trunov]

It seems that if we push the monomorphization phase after typechecking, we'll run into issues with a lot of Sway intrinsics, because those are often "polymorphic".

We can forbid some bounded polymorphic intrinsics like __eq<T>(lhs: T, rhs: T) if their arguments come from generics, but for things like __is_reference_type<T>() -> bool it looks more complicated. Or, for instance, for __check_str_type<T>() -> u64, which throws a compile time error if its type argument T is not a string type (which we cannot do for generic functions, so the write_str stdlib function won't compile anymore:

sway/sway-lib-std/src/hash.sw

Line 39 in 159a240

pub fn write_str<S>(ref mut self, s: S) {

)