Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[refactor]: add resolvers; refactor evaluator #1385

Open
wants to merge 8 commits into
base: master
Choose a base branch
from

Conversation

scolsen
Copy link
Contributor

@scolsen scolsen commented Feb 17, 2022

This PR does two things:

  1. Add a new Resolver abstraction. Resolvers dictate the order in which lookups should be performed. They're a new layer between the Environment lookup functions and provide a means of performing lookups at a slightly higher level--each Resolver is a combination of lookup techniques and Resolvers themselves can be flexibly chained together. For example, localShadowResolver <> globalDynamicResolver will return a resolver that first attempts to find shadowed symbols in the local environment and then attempts to find globally defined dynamic symbols. The comments in Resolver.hs explain this in a bit more detail.
  2. The Resolvers allow us to remove all the inline lookup code in the main eval loops in favor of Resolvers. In addition, I've moved all the local evaluator functions out of the main eval loop and into the top level. Instead of relying on implicitly closured state, one has to pass additional state directly--but this should make the primary loop much more modular and should make the individual evaluators easier to modify.

I'm hoping (2.) makes it a bit easier to start debugging potential performance hotspots in the evaluation loop (see #1215)

Adds a new abstraction, the *Resolver* which controls the lookup order
when resolving symbols. Env, provides mechanisms for storing and
retrieving bindings, Context bundles environments together (and provides
some retrieval utilities) and Resolver determines the order in which
multiple lookups should be executed.

The Resolver abstractions allows us to remove much of the gnarly lookup
code that was previously directly defined in the evaluator. The
abstraction also provides a Semigroup instance for the flexible
combination of resolvers, making it significantly easier to try out
different orders in the evaluator.

This commit changes no behavior -- it just uses Resolvers to emulate the
previous lookup order in the evaluator. Future commits will leverage the
abstraction to simplify lookups and refine behaviors where needed.
This refactor moves all the form-specific evaluation functions that were
previously defined as local functions in the main eval loop into
top-level functions in Eval.hs. This should make it easier to tweak and
swap out these functions in the main eval loop.

As a result of the move, the functions need to take additional
parameters to carry state across the evaluator loops, namely, we need to
pass:

- a resolution mode
- a resolver
- a root xobj
- a context

While this may seem tedious, it gives us flexibility. Evaluators that
don't need the global eval loop's resolver can refrain from using it,
and it's easier to tell what's being passed since we're not referencing
values captured much further up in a function's body.

Additionally, I've moved out local fold function definitions
"successiveEval" and tried to give them more expressive names. I also
used shorter definitions (mostly dropping explicit case statements)
where possible.

I'm hoping this change will also make it easier to try and pinpoint
opportunities to improve evaluator performance on large inputs.

This commit contains no behavioral differences.
This commit finalizes the migration of local definitions in the main
eval loop into top level functions. In addition:

- I removed privacy checking. We do this in Expand.hs and it currently
  covers our test cases.
- I consolidated some static form checks. Note that the evaluator is
  particularly sensitive about return values--I was stuck for a while
  because I inadvertently returned the head of a list from a static
  check instead of the list being evaluated itself -- this totally
  breaks the evaluator. Because of this, I set the return value of this
  function to Error||Unit.
- I've also combined the array evaluation functions.
- Finally, I've removed the Fn check from isResolvableObj. After
  checkStatic consolidation, this introduces a problematic case, and
  removing it doesn't seem to break anything, so it seems extraneous.
Remove unused functions and data definitions; add some comments.
@scolsen scolsen requested a review from a team February 17, 2022 21:20
@scolsen
Copy link
Contributor Author

scolsen commented Feb 17, 2022

Oh, also, Resolver's Show instance makes it significantly easier to debug lookup problems. Each resolver has a given name, and if you Show any resolver it will display the order in which the Resolvers combined to build it (with the semigroup) are run:

"GlobalDynamic -> DynamicFull -> LocalFull -> GlobalFull -> CurrentModule -> Type -> UseModule ..." 

etc. so, doing trace (show resolver) helps quite a bit when there's lookup issues.

Copy link
Member

@hellerve hellerve left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I only gave this a casual glance for now, but it looks pretty awesome! Great work!

isResolvableStaticObj (Instantiate _) = True
isResolvableStaticObj (Deftemplate _) = True
isResolvableStaticObj (External _) = True
isResolvableStaticObj (Match _) = True
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What does this change do?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's some context in commit msg 5cc5752 -- basically there used to be multiple approaches to checking static calls, two different functions in eval and one direct use of isResolvableObj. I merged the two different definitions of checkStatic by making them rely on isResolvableObj--however, this caused failures in macros when Fns were encountered since they were checked by isResolvable. Removing that check doesn't seem to break anything, so I dropped it.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Also, the addition of Match is similar -- merged to bring parity with the two old separate definitions.

src/Resolver.hs Outdated Show resolved Hide resolved
src/Eval.hs Outdated Show resolved Hide resolved
scolsen and others added 2 commits February 17, 2022 18:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants