Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

core: introduce extensions #9800

Merged
merged 65 commits into from
Apr 28, 2021
Merged

Conversation

AaronO
Copy link
Contributor

@AaronO AaronO commented Mar 15, 2021

This PR implements ideas discussed in #9738, specifically the concept of JsRuntimeModules which makes modular runtime features plug-n-play.

Goals

Reduce complexity in runtime, make modules self-contained and streamline snapshotting, etc...

This should make many parts of deno simpler and reusable by third-parties and reduce the implicit complexity in bootstrapping JsRuntimes (when snapshotting or not ...). op_crates should now be easily usable without runtime

Subtasks

  • Core JsRuntimeModule foundations/traits (core/runtime_modules.rs)
  • Implement JsRuntime init via modules
  • Refactor all op_crates to be ::init() -> JsRuntimeModule
    (essentially move code from runtime/ops/*.rs to op_crates/*/lib.rs)
  • Introduce include_js_files!() and declare_ops!() macros
  • Refactor module consumers (places where we init JsRuntimes & ops to use these new modules)
  • Implement OpMiddleware
    • OpMetrics (replacing op_metrics)
    • Maybe implement OpTracing middleware (--optrace flag)
    • Maybe turn the UnstableChecker into an OpMiddleware
  • Make deno_runtime use JsRuntimeModules ?
  • Explore declarative plugins ? (a subset of the JsRuntimeModule interface)
  • Review new abstractions (naming, fit, etc...)
  • Maybe add a web_modules() helper function that inits all Web-API modules (url, fetch, ... essentially op_crates today)
  • Maybe add a --no-snapshot flag to allow running without snapshots (not sure if useful at all, even when debugging, besides optimize(core): move internal js src off heap #9713)

Potential Follow-ups

  • Restore extension builder pattern (more user friendly)
  • Restore declare_ops!() macros
  • Restore MultiModule abstraction, thus reverting Extension to a Trait
  • Use extensions in runtime
  • Use extensions in cli
  • Allow passing extra extensions to Worker and WebWorker, making them more reusable and composable
  • UnstableChecker OpMiddleware for fully unstable ops
  • Tracing OpMiddleware for debugging
  • Plugins (might only be a subset of the full trait) ...
  • Allow running Worker and MainWorker without snapshots, good for debugging and in general snapshots should be optional and viewed as a graceful enhancement (perf boost)

// ```
// fn web_modules(args: WebModuleArgs) -> MultiModule {
// MultiModule::new(vec![deno_url::init(), deno_fetch::init(...), ...])
// }
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Interesting!

@ry
Copy link
Member

ry commented Mar 16, 2021

Nice! I fully support this effort.

If you can get the existing op crates working and the tests pass, I'm rather confident that this will generalize to future needs. I think it will be challenging tho.

runtime/build.rs Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
core/runtime_modules.rs Outdated Show resolved Hide resolved
op_crates/fetch/lib.rs Outdated Show resolved Hide resolved
runtime/build.rs Outdated Show resolved Hide resolved
AaronO added 4 commits April 21, 2021 21:14
Don't check bytes sent/received since those are irrelevant and not functional since serde_v8 op-layer rewrite
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

After more thought I really do think we shouldn't go with the name "extensions". A couple reasons:

  • term "op crates" has been used for almost a year, it will be very confusing to transition to "extensions"
  • "extension" is not very specific
  • these are not "extensions" per se - they do not augment the capabilities of the runtime in any way, but rather allow to group ops inside a single crate

core/extensions.rs Outdated Show resolved Hide resolved
core/extensions.rs Outdated Show resolved Hide resolved
core/runtime.rs Show resolved Hide resolved
core/extensions.rs Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
core/extensions.rs Show resolved Hide resolved
core/runtime.rs Show resolved Hide resolved
runtime/web_worker.rs Show resolved Hide resolved
core/extensions.rs Outdated Show resolved Hide resolved
core/runtime.rs Outdated Show resolved Hide resolved
Copy link
Member

@bartlomieju bartlomieju left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@bartlomieju bartlomieju merged commit 0260b48 into denoland:main Apr 28, 2021
@AaronO AaronO deleted the core/js-runtime-modules branch April 28, 2021 16:44
AaronO added a commit to AaronO/deno that referenced this pull request Apr 28, 2021
This implementation is both cleaner and more flexible than the original prototype in denoland#9800
@AaronO AaronO mentioned this pull request May 1, 2021
3 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

7 participants