Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

src: stop copying code cache #47144

Merged
merged 5 commits into from
May 7, 2023
Merged

Conversation

kvakil
Copy link
Contributor

@kvakil kvakil commented Mar 17, 2023

The code cache is quite large - around 1.3 MiB. Change the code to use non-owning buffers to avoid copying it. For starting up an otherwise empty main isolate, this saves around 1.3 MiB of unique set size memory (9.9 MiB -> 8.6 MiB) and 1.1ms elapsed time (22.9 ms -> 21.8 ms).

Copying the code cache is unnecessary since:

  1. for the builtin snapshot, the code cache data has static lifetime.
  2. for non-builtin snapshots, we create copies of the code cache data in SnapshotDeserializer::ReadVector. These copies are owned by the Environment (through IsolateData -> SnapshotData), so they won't be deallocated.
  3. a worker thread can copy a parent's isolate's code cache, but in that case we still know that the parent isolate will outlive the worker isolate.

(Admittedly point (2) feels a little fragile from a lifetime perspective, and I would be happy to restrict this optimization to the builtin snapshot.)

$ perf stat -r 100 -e ... ./node -e 0

 Performance counter stats for './node -e 0' (100 runs):

             21.78 msec task-clock
              2760      page-faults
         113161604      instructions
          18437648      branches
            423230      branch-misses
            853093      cache-references
             41474      cache-misses

         0.0225473 +- 0.0000504 seconds time elapsed  ( +-  0.22% )

$ perf stat -r 100 -e ... ./node-main -e 0

 Performance counter stats for './node-main -e 0' (100 runs):

             22.91 msec task-clock
              3102      page-faults
         114890673      instructions
          18751329      branches
            428909      branch-misses
            895721      cache-references
             45202      cache-misses

         0.0233760 +- 0.0000741 seconds time elapsed  ( +-  0.32% )

@nodejs-github-bot
Copy link
Collaborator

Review requested:

  • @nodejs/startup

@nodejs-github-bot nodejs-github-bot added c++ Issues and PRs that require attention from people who are familiar with C++. needs-ci PRs that need a full CI run. labels Mar 17, 2023
@kvakil kvakil added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 18, 2023
@nodejs-github-bot
Copy link
Collaborator

@kvakil kvakil added the snapshot Issues and PRs related to the startup snapshot label Mar 18, 2023
Copy link
Member

@joyeecheung joyeecheung left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % nits

src/node_builtins.cc Outdated Show resolved Hide resolved
src/node_builtins.cc Outdated Show resolved Hide resolved
@joyeecheung
Copy link
Member

For 2) I think we can just document in node.h that the snapshot data must outlive the isolate/Environment. That's already the case internally.

@kvakil
Copy link
Contributor Author

kvakil commented Mar 19, 2023

For 2) I think we can just document in node.h that the snapshot data must outlive the isolate/Environment. That's already the case internally.

I added a line about this to the documentation of EmbedderSnapshotData. Other comments should also be addressed. PTAL :)

@kvakil kvakil added the request-ci Add this label to start a Jenkins CI on a PR. label Mar 19, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Mar 19, 2023
@nodejs-github-bot
Copy link
Collaborator

src/node.h Outdated
Comment on lines 507 to 508
// The snapshot *must* be kept alive during the execution of the environment /
// isolate that it is derived from.
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
// The snapshot *must* be kept alive during the execution of the environment /
// isolate that it is derived from.
// The snapshot *must* be kept alive during the execution of the Isolate
// that derives from it.

or

Suggested change
// The snapshot *must* be kept alive during the execution of the environment /
// isolate that it is derived from.
// The snapshot *must* be kept alive during the execution of the Isolate
// that was created using it.

(this header refers to Isolate/Environment as capitalized names to distinguish them from the regular English-language words; Environment instances do not outlive their Isolate instances; and the Isolate derives from the snapshot in this scenario, not the other way around, since this is about the case in which the snapshot is being consumed, not when it is being built)

The code cache is quite large - around 1.3 MiB. Change the code to use
non-owning buffers to avoid copying it. For starting up an otherwise
empty main isolate, this saves around 1.3 MiB of unique set size memory
(9.9 MiB -> 8.6 MiB) and 1.1ms elapsed time (22.9 ms -> 21.8 ms).

Copying the code cache is unnecessary since:

1. for the builtin snapshot, the code cache data has static lifetime.
2. for non-builtin snapshots, we create copies of the code cache data in
   `SnapshotDeserializer::ReadVector`. These copies are owned by the
   `Environment` (through `IsolateData` -> `SnapshotData`), so they
   won't be deallocated.
3. a worker thread can copy a parent's isolate's code cache, but in that
   case we still know that the parent isolate will outlive the worker
   isolate.

(Admittedly point (2) feels a little fragile from a lifetime
perspective, and I would be happy to restrict this optimization to the
builtin snapshot.)

```console
$ perf stat -r 100 -e ... ./node -e 0

 Performance counter stats for './node -e 0' (100 runs):

             21.78 msec task-clock
              2760      page-faults
         113161604      instructions
          18437648      branches
            423230      branch-misses
            853093      cache-references
             41474      cache-misses

         0.0225473 +- 0.0000504 seconds time elapsed  ( +-  0.22% )

$ perf stat -r 100 -e ... ./node-main -e 0

 Performance counter stats for './node-main -e 0' (100 runs):

             22.91 msec task-clock
              3102      page-faults
         114890673      instructions
          18751329      branches
            428909      branch-misses
            895721      cache-references
             45202      cache-misses

         0.0233760 +- 0.0000741 seconds time elapsed  ( +-  0.32% )
```
@kvakil kvakil added the request-ci Add this label to start a Jenkins CI on a PR. label Apr 10, 2023
@github-actions github-actions bot removed the request-ci Add this label to start a Jenkins CI on a PR. label Apr 10, 2023
@nodejs-github-bot
Copy link
Collaborator

@kvakil
Copy link
Contributor Author

kvakil commented Apr 10, 2023

I reworded the comment to @addaleax's suggestion. Test failures look like obvious flakes. PTAL.

@nodejs-github-bot
Copy link
Collaborator

@kvakil kvakil added the author ready PRs that have at least one approval, no pending requests for changes, and a CI started. label May 5, 2023
@nodejs-github-bot
Copy link
Collaborator

@debadree25 debadree25 added commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. commit-queue Add this label to land a pull request using GitHub Actions. labels May 6, 2023
@nodejs-github-bot nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label May 7, 2023
@nodejs-github-bot nodejs-github-bot merged commit 1e6e5a9 into nodejs:main May 7, 2023
@nodejs-github-bot
Copy link
Collaborator

Landed in 1e6e5a9

targos pushed a commit that referenced this pull request May 12, 2023
The code cache is quite large - around 1.3 MiB. Change the code to use
non-owning buffers to avoid copying it. For starting up an otherwise
empty main isolate, this saves around 1.3 MiB of unique set size memory
(9.9 MiB -> 8.6 MiB) and 1.1ms elapsed time (22.9 ms -> 21.8 ms).

Copying the code cache is unnecessary since:

1. for the builtin snapshot, the code cache data has static lifetime.
2. for non-builtin snapshots, we create copies of the code cache data in
   `SnapshotDeserializer::ReadVector`. These copies are owned by the
   `Environment` (through `IsolateData` -> `SnapshotData`), so they
   won't be deallocated.
3. a worker thread can copy a parent's isolate's code cache, but in that
   case we still know that the parent isolate will outlive the worker
   isolate.

(Admittedly point (2) feels a little fragile from a lifetime
perspective, and I would be happy to restrict this optimization to the
builtin snapshot.)

```console
$ perf stat -r 100 -e ... ./node -e 0

 Performance counter stats for './node -e 0' (100 runs):

             21.78 msec task-clock
              2760      page-faults
         113161604      instructions
          18437648      branches
            423230      branch-misses
            853093      cache-references
             41474      cache-misses

         0.0225473 +- 0.0000504 seconds time elapsed  ( +-  0.22% )

$ perf stat -r 100 -e ... ./node-main -e 0

 Performance counter stats for './node-main -e 0' (100 runs):

             22.91 msec task-clock
              3102      page-faults
         114890673      instructions
          18751329      branches
            428909      branch-misses
            895721      cache-references
             45202      cache-misses

         0.0233760 +- 0.0000741 seconds time elapsed  ( +-  0.32% )
```

PR-URL: #47144
Reviewed-By: Joyee Cheung <[email protected]>
Reviewed-By: Yagiz Nizipli <[email protected]>
Reviewed-By: Chengzhong Wu <[email protected]>
@danielleadams danielleadams added the backport-requested-v18.x PRs awaiting manual backport to the v18.x-staging branch. label Jun 29, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
author ready PRs that have at least one approval, no pending requests for changes, and a CI started. backport-requested-v18.x PRs awaiting manual backport to the v18.x-staging branch. c++ Issues and PRs that require attention from people who are familiar with C++. commit-queue-squash Add this label to instruct the Commit Queue to squash all the PR commits into the first one. needs-ci PRs that need a full CI run. snapshot Issues and PRs related to the startup snapshot
Projects
None yet
Development

Successfully merging this pull request may close these issues.

8 participants