src: large page support for macOS #28977

devnexen · 2019-08-05T13:33:00Z

Proposal to bring the support for this platform.
We assume the pse36 cpu flag is present for 2MB large page support present
in recent years in mac line (not to be backported to 10.x anyway).
Recommended better for mac production servers rather than casual mac books.

bnoordhuis

Thanks, LGTM % some style nits.

bnoordhuis · 2019-08-06T11:56:08Z

configure.py

    if options.shared or options.enable_static:
      raise Exception(
        'Large pages are supported only while creating node executable.')
    if target_arch!="x64":
      raise Exception(
        'Large pages are supported only x64 platform.')
+    if flavor == 'mac':
+      print("macOS server with 32GB or more is recommended")


Style: single quotes

Substance: info('macOS server...') might be better although plain print() isn't exactly wrong either, it's used in plenty of places.

Putting @bnoordhuis's comment above as a suggestion for ease of adoption:

Suggested change

print("macOS server with 32GB or more is recommended")

info('macOS server with 32GB or more is recommended')

bnoordhuis · 2019-08-06T11:57:27Z

src/large_pages/node_large_page.cc

+
+  while (true) {
+    if (vm_region_recurse_64(mach_task_self(), &addr, &size, &depth,
+          (vm_region_info_64_t)&map, &count) != KERN_SUCCESS) {


reinterpret_cast please, no C style casts (and preferably line up the arguments.)

bnoordhuis · 2019-08-06T11:59:13Z

src/large_pages/node_large_page.cc

+  tmem = mmap(start, size,
+      PROT_READ | PROT_WRITE | PROT_EXEC,
+      MAP_PRIVATE | MAP_ANONYMOUS,
+      VM_FLAGS_SUPERPAGE_SIZE_2MB, 0);


Can you line up the arguments with the ones on the first line?

Proposal to bring the support for this platform. We assume the pse36 cpu flag is present for 2MB large page support present in recent years in mac line (not to be backported to 10.x anyway). Recommended better for mac production servers rather than casual mac books.

nodejs-github-bot · 2019-08-06T20:14:44Z

CI: https://ci.nodejs.org/job/node-test-pull-request/24868/

Trott · 2019-08-08T02:40:23Z

One remaining style nit?

Trott · 2019-08-19T02:08:04Z

Significant chunk of C++ code added, I think, since previous reviews, so re-requested reviews.

nodejs-github-bot · 2019-08-20T04:01:24Z

CI: https://ci.nodejs.org/job/node-test-pull-request/25114/

bnoordhuis · 2019-08-20T10:51:03Z

src/large_pages/node_large_page.cc

@@ -382,9 +382,25 @@ MoveTextRegionToLargePages(const text_region& r) {
    munmap(nmem, size);
    return -1;
  }
+  memcpy(tmem, nmem, size);
+  ret = mprotect(start, size, PROT_READ | PROT_WRITE | PROT_EXEC);


Can you explain why this is necessary? It's already mapped rwx, isn't it?

Should the mmap() call include MAP_FIXED in its flags like it does on Linux and FreeBSD? If not, can you update the comment atop of this function?

Aside: since there are a lot of munmap(nmem, size) calls now, it'd be nice to DRY it:

nmem = mmap(...); OnScopeLeave munmap_on_return([nmem, size] () { if (-1 == munmap(nmem, size)) PrintSystemError(errno); });

I guess you lose the ability to return an error from MoveTextRegionToLargePages() on munmap() failure but that doesn't seem too bad, it's not really a fatal error and probably hypothetical to boot. You could simply abort on failure.

To answer your first question it is rx but will add a comment (note that s start address not the new address). Ok for the rest (including the info change in configure.py).

Just to round out this discussion: using MAP_FIXED is not an option on macos? start is properly aligned, isn't it?

start is indeed aligned like other platforms, MAP_FIXED failed on this platform though with 24MB close to the start address reservation attempt.

Yeah, I'd have to go digging for the reason again but MAP_FIXED has long had problems with mmap for large page sizes on macos. Pretty sure the macos api docs recommend not to use it but if I recall correctly they don't really explain why.

src/large_pages/node_large_page.cc

bnoordhuis · 2019-08-20T12:18:58Z

src/large_pages/node_large_page.cc

  tmem = mmap(start, size,
              PROT_READ | PROT_WRITE | PROT_EXEC,
              MAP_PRIVATE | MAP_ANONYMOUS,
              VM_FLAGS_SUPERPAGE_SIZE_2MB, 0);
  if (tmem == MAP_FAILED) {
    PrintSystemError(errno);
-    munmap(nmem, size);


There are a few more munmap(nmem, size) calls in linux/freebsd code paths. Can you remove those as well?

bnoordhuis · 2019-08-20T12:52:48Z

src/large_pages/node_large_page.cc

@@ -382,9 +382,25 @@ MoveTextRegionToLargePages(const text_region& r) {
    munmap(nmem, size);
    return -1;
  }
+  memcpy(tmem, nmem, size);
+  ret = mprotect(start, size, PROT_READ | PROT_WRITE | PROT_EXEC);


Just to round out this discussion: using MAP_FIXED is not an option on macos? start is properly aligned, isn't it?

bnoordhuis · 2019-08-20T12:54:10Z

src/large_pages/node_large_page.cc

  return IsSuperPagesEnabled();
+#elif defined(__APPLE__)
+  // pse-36 flag is present in recent mac x64 products.


How recent is recent? Node.js supports macOS 10.11 and up.

Then it is fine El Capitan is recent enough :)

nodejs-github-bot · 2019-08-20T17:29:40Z

CI: https://ci.nodejs.org/job/node-test-pull-request/25130/

Proposal to bring the support for this platform. We assume the pse36 cpu flag is present for 2MB large page support present in recent years in mac line (not to be backported to 10.x anyway). Recommended better for mac production servers rather than casual mac books. PR-URL: #28977 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]>

Trott · 2019-08-20T18:44:34Z

Landed in 32df017.

Added dont-land-on-v10.x and dont-land-on-v8.x based on the commit message.

Proposal to bring the support for this platform. We assume the pse36 cpu flag is present for 2MB large page support present in recent years in mac line (not to be backported to 10.x anyway). Recommended better for mac production servers rather than casual mac books. PR-URL: #28977 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]>

Proposal to bring the support for this platform. We assume the pse36 cpu flag is present for 2MB large page support present in recent years in mac line (not to be backported to 10.x anyway). Recommended better for mac production servers rather than casual mac books. PR-URL: nodejs#28977 Reviewed-By: Ben Noordhuis <[email protected]> Reviewed-By: James M Snell <[email protected]>

nodejs-github-bot added build Issues and PRs related to build files or the CI. c++ Issues and PRs that require attention from people who are familiar with C++. labels Aug 5, 2019

bnoordhuis approved these changes Aug 6, 2019

View reviewed changes

jasnell approved these changes Aug 7, 2019

View reviewed changes

Fixing address copy handling.

59a2232

Trott requested review from jasnell and bnoordhuis August 19, 2019 02:07

bnoordhuis reviewed Aug 20, 2019

View reviewed changes

Tweaks from feedback

0f7190f

bnoordhuis approved these changes Aug 20, 2019

View reviewed changes

jasnell approved these changes Aug 20, 2019

View reviewed changes

Trott added author ready PRs that have at least one approval, no pending requests for changes, and a CI started. dont-land-on-v10.x labels Aug 20, 2019

Trott closed this Aug 20, 2019

BridgeAR mentioned this pull request Sep 3, 2019

v12.10.0 proposal #29429

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

src: large page support for macOS #28977

src: large page support for macOS #28977

devnexen commented Aug 5, 2019

bnoordhuis left a comment

bnoordhuis Aug 6, 2019

Trott Aug 19, 2019

bnoordhuis Aug 6, 2019

bnoordhuis Aug 6, 2019

nodejs-github-bot commented Aug 6, 2019

Trott commented Aug 8, 2019

Trott commented Aug 19, 2019

nodejs-github-bot commented Aug 20, 2019

bnoordhuis Aug 20, 2019

devnexen Aug 20, 2019 •

edited

Loading

bnoordhuis Aug 20, 2019

devnexen Aug 20, 2019

jasnell Aug 20, 2019

bnoordhuis Aug 20, 2019

bnoordhuis Aug 20, 2019

bnoordhuis Aug 20, 2019

devnexen Aug 20, 2019

nodejs-github-bot commented Aug 20, 2019

Trott commented Aug 20, 2019

	print("macOS server with 32GB or more is recommended")
	info('macOS server with 32GB or more is recommended')

src: large page support for macOS #28977

src: large page support for macOS #28977

Conversation

devnexen commented Aug 5, 2019

bnoordhuis left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nodejs-github-bot commented Aug 6, 2019

Trott commented Aug 8, 2019

Trott commented Aug 19, 2019

nodejs-github-bot commented Aug 20, 2019

Choose a reason for hiding this comment

devnexen Aug 20, 2019 • edited Loading

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

nodejs-github-bot commented Aug 20, 2019

Trott commented Aug 20, 2019

devnexen Aug 20, 2019 •

edited

Loading