Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add the concept of "backend intrinsics" to Cranelift #9651

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

alexcrichton
Copy link
Member

This commit is an initial stab at implementing interpreter-to-host communication in Pulley. The basic problem is that Pulley needs the ability to call back into Wasmtime to implement tasks such as memory.grow, imported functions, etc. For native platforms this is a simple call_indirect operation in Cranelift but the story for Pulley must be different because it's effectively switching from interpreted code to native code.

The solution I've ended up settling on looks pretty similar to native platforms but with a few important tweaks:

  • A new call_indirect_host opcode is added to Pulley.
    • Function signatures that can be called from Pulley bytecode are statically enumerated at build-time.
    • This enables the implementation of call_indirect_host to take an immediate of which signature is being used and cast the function pointer to the right type.
  • A new "backend intrinsic" concept is added to Cranelift.
    • This is a new variant of ExternalName.
    • The intention is that this has backend-specific meaning.
    • For Pulley, this means that the Nth function signature is being called.
  • Code generation for Pulley in wasmtime-cranelift now has Pulley-specific handling of the wasm-to-host transition where all previous call_indirect instructions are replaced with a call to a "backend intrinsic" which gets lowered to a call_indirect_host.

Note that most of this still isn't hooked up everywhere in Wasmtime. That means that the testing here is pretty light at this time. It'll require a fair bit more work to get everything fully integrated from Wasmtime in Pulley. This is expected to be one of the significant remaining chunks of work and should help unblock future testing (or make those diffs smaller ideally).

This commit is an initial stab at implementing interpreter-to-host
communication in Pulley. The basic problem is that Pulley needs the
ability to call back into Wasmtime to implement tasks such as
`memory.grow`, imported functions, etc. For native platforms this is a
simple `call_indirect` operation in Cranelift but the story for Pulley
must be different because it's effectively switching from interpreted
code to native code.

The solution I've ended up settling on looks pretty similar to native
platforms but with a few important tweaks:

* A new `call_indirect_host` opcode is added to Pulley.
  * Function signatures that can be called from Pulley bytecode are
    statically enumerated at build-time.
  * This enables the implementation of `call_indirect_host` to take an
    immediate of which signature is being used and cast the function
    pointer to the right type.
* A new "backend intrinsic" concept is added to Cranelift.
  * This is a new variant of `ExternalName`.
  * The intention is that this has backend-specific meaning.
  * For Pulley, this means that the Nth function signature is being called.
* Code generation for Pulley in `wasmtime-cranelift` now has
  Pulley-specific handling of the wasm-to-host transition where all
  previous `call_indirect` instructions are replaced with a call to a
  "backend intrinsic" which gets lowered to a `call_indirect_host`.

Note that most of this still isn't hooked up everywhere in Wasmtime.
That means that the testing here is pretty light at this time. It'll
require a fair bit more work to get everything fully integrated from
Wasmtime in Pulley. This is expected to be one of the significant
remaining chunks of work and should help unblock future testing (or make
those diffs smaller ideally).
@alexcrichton alexcrichton requested review from a team as code owners November 21, 2024 21:54
@alexcrichton alexcrichton requested review from abrown and pchickey and removed request for a team November 21, 2024 21:54
@alexcrichton
Copy link
Member Author

cc @cfallin as you're also no doubt interested in this as well

Copy link
Member

@cfallin cfallin left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM on the Cranelift bits -- happy to dive deeper into the Pulley and Wasmtime bits too if you'd like but I'm not as deep into that context at the moment.

cranelift/codegen/src/ir/extname.rs Outdated Show resolved Hide resolved
cranelift/codegen/src/isa/pulley_shared/abi.rs Outdated Show resolved Hide resolved
@github-actions github-actions bot added cranelift Issues related to the Cranelift code generator cranelift:module pulley Issues related to the Pulley interpreter labels Nov 21, 2024
Copy link

Subscribe to Label Action

cc @fitzgen

This issue or pull request has been labeled: "cranelift", "cranelift:module", "pulley"

Thus the following users have been cc'd because of the following labels:

  • fitzgen: pulley

To subscribe or unsubscribe from this label, edit the .github/subscribe-to-label.json configuration file.

Learn more.

@bjorn3
Copy link
Contributor

bjorn3 commented Nov 22, 2024

Instead of adding a whole new instruction, would it be possible to use a UserExternalName with a reserved namespace like u32::MAX? Or the Pulley interpreter could have a set of UserExternalName for which to call the specified host function without reserving any namespace.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cranelift:module cranelift Issues related to the Cranelift code generator pulley Issues related to the Pulley interpreter
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants