deps,lib,src: add experimental web storage #52435

cjihrig · 2024-04-09T15:45:23Z

Reopening of #50169, which cannot be reopened. Opening as a draft because the original PR had a block, so the TSC would need to make a decision (#50169 (comment)).

This commit introduces an experimental implementation of the Web Storage API using SQLite as the backing data store.

nodejs-github-bot · 2024-04-09T15:45:28Z

Review requested:

@nodejs/gyp
@nodejs/security-wg
@nodejs/web-standards

anonrig · 2024-04-09T18:00:17Z

@nodejs/tsc can we add this to tomorrows meeting?

GeoffreyBooth · 2024-04-09T18:24:55Z

Looking at #50169 (review), I think all of those concerns are addressed by making SQLite an internal dependency that is not exposed in any way to the user; is that the case in this PR?

If so, then I think the block has been addressed, and there’s nothing to override.

ronag · 2024-04-09T19:02:52Z

Due to the history of this PR I propose we just quickly bring it up and approve it at the TSC to make sure there is no further frustration.

github-actions · 2024-04-10T02:15:51Z

The notable-change PRs with changes that should be highlighted in changelogs. label has been added by @anonrig.

Please suggest a text for the release notes if you'd like to include a more detailed summary, then proceed to update the PR description with the text or a link to the notable change suggested text comment. Otherwise, the commit will be placed in the Other Notable Changes section.

GeoffreyBooth · 2024-04-10T02:25:57Z

I’m all for adding localStorage support, but this PR isn’t exactly ready to land as is. The previous PR had no approvals, just the one block; and there was at least one note that hasn’t been addressed as far as I’m aware, of where to save the data (or how the user should specify where to save the data).

cjihrig · 2024-04-10T02:49:04Z

I think all of those concerns are addressed by making SQLite an internal dependency that is not exposed in any way to the user; is that the case in this PR?

This is currently the same implementation as #50169.

If so, then I think the block has been addressed, and there’s nothing to override.

The specific review containing the objection is not really relevant in my opinion. However, not trying to "keep up with the Joneses" (referring to Deno, txiki.js, and potentially Bun) was mentioned, as was a sentiment opposing browser APIs.

but this PR isn’t exactly ready to land as is

This is indeed not ready to land yet. I'm not going to invest any more time on it though until I know the TSC supports landing it in principle. I do believe the PR is far enough along for that decision to be made fairly easily.

mcollina

I’m in favor of landing this.

lib/internal/webstorage.js

jasnell · 2024-04-10T15:39:52Z

The @nodejs/tsc discussed this today and there is consensus on having web storage land. Obviously there are technical details to work through but there are no objections to this moving forward. There is also positive consensus on adding sqllite as an internal dependency in support of the implementation. Let's get this done :-)

jasnell · 2024-04-10T16:20:53Z

src/node_webstorage.cc

+    return;
+  }
+  CHECK(domexception_ctor_val->IsFunction());
+  Local<Function> domexception_ctor = domexception_ctor_val.As<Function>();


Non-blocking Nit: The code necessary to create a DOMException should likely be moved out into a separate utility method.

src/node_webstorage.h

src/node_webstorage.cc

cjihrig · 2024-04-10T18:39:54Z

The @nodejs/tsc discussed this today and there is consensus on having web storage land.

I appreciate it. I'll start addressing the feedback from this PR and the old one soon.

As a side note, I noticed on the TSC call that a few people had some technical questions, such as "why SQLite, specifically instead of writing to disk." I tried to answer this and other questions in #50169 (comment).

doc/api/globals.md

benjamingr

In general I am +1 on eventually landing this and like other experimental/new stuff I am in favor of landing this sooner rather than later and then iterating on it.

There is a lot of work to be done to make this stable but I think iterating usually is. abetter strategy than big changes and a lot of criticism.

My understanding is the TSC discussed this and approved the idea in principle so this is me +1ing (I couldn't attend due to a prod incident)

benjamingr · 2024-04-11T08:02:48Z

Note to self: create tickets after this lands with all the issues (e.g. range queries and others) so they can be addressed or discussed before this is stable.

mcollina · 2024-04-12T13:03:32Z

@cjihrig did you consider using:

leveldb
lmdb
berkeley db

Would any of the above provide slightly better characteristics (performance, binary size, source code size, manutenibility)?

Note that this does not matter much, and we can change the storage mechanism later (before it becomes stable).

cjihrig · 2024-04-12T13:12:08Z

did you consider using

Please see #50169 (comment) for my answer to why I chose SQLite.

benjamingr · 2024-04-12T14:49:54Z

I'd like to publicly agree with Colin on SQLite here.

I've also done a few projects with various options for embedded databases and have found SQLite to be an excellent choice due to its ease of integration, ultra-permissive license (public domain), omnipresence across hardware which means good support etc.

localStorage + sharing readers across workers seems right up its alley. A file with a global lock would work but would lead to a lot less extensibility and would make us solve problems in the future SQLite would avoid.

(If anything, we should eventully consider exposing SQLite the same way Python does)

mcollina · 2024-04-12T21:41:46Z

There might be a slight issue when adding SQLite as a dependency. It uses a non-approved license that is not OSI.

We would need to seek legal+board approval to land this. @nodejs/tsc can you weight in?

As an alternative, we could use libsql that is MIT license (but is a fork of sqlite).

nodejs-github-bot · 2024-06-14T18:40:08Z

Landed in 340cb68...15b3902

This commit introduces an experimental implementation of the Web Storage API using SQLite as the backing data store. PR-URL: #52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This is equivalent to the following upstream change: https://sqlite.org/src/info/6c103aee6f146869 Original commit message: Change constant expressions to pre-computed constants, because apparently MSVC on ARM requires that. [forum:/forumpost/4feb1685cced0a8e|Forum thread 4feb1685cced0a8e]. FossilOrigin-Name: 6c103aee6f146869a3e0c48694592f2e4c6b57ecdb4450f46e762c38b4e686f1 PR-URL: #52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This commit introduces an experimental implementation of the Web Storage API using SQLite as the backing data store. PR-URL: #52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This is equivalent to the following upstream change: https://sqlite.org/src/info/6c103aee6f146869 Original commit message: Change constant expressions to pre-computed constants, because apparently MSVC on ARM requires that. [forum:/forumpost/4feb1685cced0a8e|Forum thread 4feb1685cced0a8e]. FossilOrigin-Name: 6c103aee6f146869a3e0c48694592f2e4c6b57ecdb4450f46e762c38b4e686f1 PR-URL: #52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This commit introduces an experimental implementation of the Web Storage API using SQLite as the backing data store. PR-URL: nodejs#52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This is equivalent to the following upstream change: https://sqlite.org/src/info/6c103aee6f146869 Original commit message: Change constant expressions to pre-computed constants, because apparently MSVC on ARM requires that. [forum:/forumpost/4feb1685cced0a8e|Forum thread 4feb1685cced0a8e]. FossilOrigin-Name: 6c103aee6f146869a3e0c48694592f2e4c6b57ecdb4450f46e762c38b4e686f1 PR-URL: nodejs#52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This commit introduces an experimental implementation of the Web Storage API using SQLite as the backing data store. PR-URL: nodejs#52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

This is equivalent to the following upstream change: https://sqlite.org/src/info/6c103aee6f146869 Original commit message: Change constant expressions to pre-computed constants, because apparently MSVC on ARM requires that. [forum:/forumpost/4feb1685cced0a8e|Forum thread 4feb1685cced0a8e]. FossilOrigin-Name: 6c103aee6f146869a3e0c48694592f2e4c6b57ecdb4450f46e762c38b4e686f1 PR-URL: nodejs#52435 Reviewed-By: Yagiz Nizipli <[email protected]> Reviewed-By: Matteo Collina <[email protected]> Reviewed-By: Benjamin Gruenbaum <[email protected]> Reviewed-By: James M Snell <[email protected]> Reviewed-By: Ethan Arrowood <[email protected]>

Notable changes: deps,lib,src: * (SEMVER-MINOR) add experimental web storage (Colin Ihrig) #52435 doc: * move `node --run` stability to rc (Yagiz Nizipli) #53433 * mark WebSocket as stable (Matthew Aitken) #53352 * mark --heap-prof and related flags stable (Joyee Cheung) #53343 * mark --cpu-prof and related flags stable (Joyee Cheung) #53343 * doc-only deprecate OpenSSL engine-based APIs (Richard Lau) #53329 inspector: * fix disable async hooks on Debugger.setAsyncCallStackDepth (Joyee Cheung) #53473 lib: * (SEMVER-MINOR) add diagnostics_channel events to module loading (RafaelGSS) #44340 util: * (SEMVER-MINOR) support `--no-` for argument with boolean type for parseArgs (Zhenwei Jin) #53107 PR-URL: #53583

nodejs/node#52435

* chore: bump Node.js to v22.9.0 * build: drop base64 dep in GN build nodejs/node#52856 * build,tools: make addons tests work with GN nodejs/node#50737 * fs: add fast api for InternalModuleStat nodejs/node#51344 * src: move package_json_reader cache to c++ nodejs/node#50322 * crypto: disable PKCS#1 padding for privateDecrypt nodejs-private/node-private#525 * src: move more crypto code to ncrypto nodejs/node#54320 * crypto: ensure valid point on elliptic curve in SubtleCrypto.importKey nodejs/node#50234 * src: shift more crypto impl details to ncrypto nodejs/node#54028 * src: switch crypto APIs to use Maybe<void> nodejs/node#54775 * crypto: remove DEFAULT_ENCODING nodejs/node#47182 * deps: update libuv to 1.47.0 nodejs/node#50650 * build: fix conflict gyp configs nodejs/node#53605 * lib,src: drop --experimental-network-imports nodejs/node#53822 * esm: align sync and async load implementations nodejs/node#49152 * esm: remove unnecessary toNamespacedPath calls nodejs/node#53656 * module: detect ESM syntax by trying to recompile as SourceTextModule nodejs/node#52413 * test: adapt debugger tests to V8 11.4 nodejs/node#49639 * lib: update usage of always on Atomics API nodejs/node#49639 * test: adapt test-fs-write to V8 internal changes nodejs/node#49639 * test: adapt to new V8 trusted memory spaces nodejs/node#50115 * deps: update libuv to 1.47.0 nodejs/node#50650 * src: use non-deprecated v8::Uint8Array::kMaxLength nodejs/node#50115 * src: update default V8 platform to override functions with location nodejs/node#51362 * src: add missing TryCatch nodejs/node#51362 * lib,test: handle new Iterator global nodejs/node#51362 * src: use non-deprecated version of CreateSyntheticModule nodejs/node#50115 * src: remove calls to recently deprecated V8 APIs nodejs/node#52996 * src: use new V8 API to define stream accessor nodejs/node#53084 * src: do not use deprecated V8 API nodejs/node#53084 * src: do not use soon-to-be-deprecated V8 API nodejs/node#53174 * src: migrate to new V8 interceptors API nodejs/node#52745 * src: use supported API to get stalled TLA messages nodejs/node#51362 * module: print location of unsettled top-level await in entry points nodejs/node#51999 * test: make snapshot comparison more flexible nodejs/node#54375 * test: do not set concurrency on parallelized runs nodejs/node#52177 * src: move FromNamespacedPath to path.cc nodejs/node#53540 * test: adapt to new V8 trusted memory spaces nodejs/node#50115 * build: add option to enable clang-cl on Windows nodejs/node#52870 * chore: fixup patch indices * chore: add/remove changed files * esm: drop support for import assertions nodejs/node#54890 * build: compile with C++20 support nodejs/node#52838 * deps: update nghttp2 to 1.62.1 nodejs/node#52966 * src: parse inspector profiles with simdjson nodejs/node#51783 * build: add GN build files nodejs/node#47637 * deps,lib,src: add experimental web storage nodejs/node#52435 * build: add missing BoringSSL dep * src: rewrite task runner in c++ nodejs/node#52609 * fixup! build: add GN build files * src: stop using deprecated fields of v8::FastApiCallbackOptions nodejs/node#54077 * fix: shadow variable * build: add back incorrectly removed SetAccessor patch * fixup! fixup! build: add GN build files * crypto: fix integer comparison in crypto for BoringSSL * src,lib: reducing C++ calls of esm legacy main resolve nodejs/node#48325 * src: move more crypto_dh.cc code to ncrypto nodejs/node#54459 * chore: fixup GN files for previous commit * src: move more crypto code to ncrypto nodejs/node#54320 * Fixup Perfetto ifdef guards * fix: missing electron_natives dep * fix: node_use_node_platform = false * fix: include src/node_snapshot_stub.cc in libnode * 5507047: [import-attributes] Remove support for import assertions https://chromium-review.googlesource.com/c/v8/v8/+/5507047 * fix: restore v8-sandbox.h in filenames.json * fix: re-add original-fs generation logic * fix: ngtcp2 openssl dep * test: try removing NAPI_VERSION undef * chore(deps): bump @types/node * src: move more crypto_dh.cc code to ncrypto nodejs/node#54459 * esm: remove unnecessary toNamespacedPath calls nodejs/node#53656 * buffer: fix out of range for toString nodejs/node#54553 * lib: rewrite AsyncLocalStorage without async_hooks nodejs/node#48528 * module: print amount of load time of a cjs module nodejs/node#52213 * test: skip reproducible snapshot test on 32-bit nodejs/node#53592 * fixup! src: move more crypto_dh.cc code to ncrypto * test: adjust emittedUntil return type * chore: remove redundant wpt streams patch * fixup! chore(deps): bump @types/node * fix: gn executable name on Windows * fix: build on Windows * fix: rename conflicting win32 symbols in //third_party/sqlite On Windows otherwise we get: lld-link: error: duplicate symbol: sqlite3_win32_write_debug >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:47987 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_sleep >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48042 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_is_nt >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48113 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_unicode >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48470 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_unicode_to_utf8 >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48486 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_mbcs_to_utf8 >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48502 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_mbcs_to_utf8_v2 >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48518 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_mbcs >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48534 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj lld-link: error: duplicate symbol: sqlite3_win32_utf8_to_mbcs_v2 >>> defined at .\..\..\third_party\electron_node\deps\sqlite\sqlite3.c:48550 >>> obj/third_party/electron_node/deps/sqlite/sqlite/sqlite3.obj >>> defined at obj/third_party/sqlite\chromium_sqlite3/sqlite3_shim.obj * docs: remove unnecessary ts-expect-error after types bump * src: move package resolver to c++ nodejs/node#50322 * build: set ASAN detect_container_overflow=0 nodejs/node#55584 * chore: fixup rebase * test: disable failing ASAN test * win: almost fix race detecting ESRCH in uv_kill libuv/libuv#4341

nodejs-github-bot added lib / src Issues and PRs related to general changes in the lib or src directory. needs-ci PRs that need a full CI run. labels Apr 9, 2024

cjihrig mentioned this pull request Apr 9, 2024

deps,doc,lib,src,test: add experimental web storage #50169

Closed

anonrig added the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Apr 9, 2024

anonrig approved these changes Apr 10, 2024

View reviewed changes

anonrig added the notable-change PRs with changes that should be highlighted in changelogs. label Apr 10, 2024

mcollina reviewed Apr 10, 2024

View reviewed changes

lib/internal/webstorage.js Outdated Show resolved Hide resolved

ShogunPanda mentioned this pull request Apr 10, 2024

Node.js Technical Steering Committee (TSC) Meeting 2024-04-10 nodejs/TSC#1526

Closed

jasnell reviewed Apr 10, 2024

View reviewed changes

src/node_webstorage.h Outdated Show resolved Hide resolved

jasnell reviewed Apr 10, 2024

View reviewed changes

src/node_webstorage.cc Outdated Show resolved Hide resolved

jasnell reviewed Apr 10, 2024

View reviewed changes

src/node_webstorage.cc Outdated Show resolved Hide resolved

jasnell reviewed Apr 10, 2024

View reviewed changes

src/node_webstorage.cc Outdated Show resolved Hide resolved

GeoffreyBooth reviewed Apr 10, 2024

View reviewed changes

doc/api/globals.md Outdated Show resolved Hide resolved

doc/api/globals.md Outdated Show resolved Hide resolved

doc/api/globals.md Outdated Show resolved Hide resolved

jasnell removed the tsc-agenda Issues and PRs to discuss during the meetings of the TSC. label Apr 10, 2024

benjamingr reviewed Apr 11, 2024

View reviewed changes

cjihrig added the commit-queue-rebase Add this label to allow the Commit Queue to land a PR in several commits. label Jun 14, 2024

nodejs-github-bot removed the commit-queue Add this label to land a pull request using GitHub Actions. label Jun 14, 2024

nodejs-github-bot closed this Jun 14, 2024

cjihrig deleted the webstorage-bak branch June 14, 2024 18:40

targos mentioned this pull request Jun 25, 2024

v22.4.0 release proposal #53583

Merged

Karliz24 mentioned this pull request Jul 9, 2024

## 2024-07-02, Version 22.4.0 (Current), @targos Karliz24/Marley-#16

Open

marco-ippolito added the baking-for-lts PRs that need to wait before landing in a LTS release. label Jul 19, 2024

targos added dont-land-on-v20.x PRs that should not land on the v20.x-staging branch and should not be released in v20.x. and removed baking-for-lts PRs that need to wait before landing in a LTS release. labels Sep 21, 2024

codebytere added a commit to electron/electron that referenced this pull request Oct 16, 2024

deps,lib,src: add experimental web storage

f1147db

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 21, 2024

deps,lib,src: add experimental web storage

1a886ff

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 21, 2024

deps,lib,src: add experimental web storage

c77bc2c

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 22, 2024

deps,lib,src: add experimental web storage

b24e44b

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 28, 2024

deps,lib,src: add experimental web storage

8b58a47

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 29, 2024

deps,lib,src: add experimental web storage

59f4d62

nodejs/node#52435

codebytere added a commit to electron/electron that referenced this pull request Oct 31, 2024

deps,lib,src: add experimental web storage

b5772c1

nodejs/node#52435

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

deps,lib,src: add experimental web storage #52435

deps,lib,src: add experimental web storage #52435

cjihrig commented Apr 9, 2024

nodejs-github-bot commented Apr 9, 2024

anonrig commented Apr 9, 2024

GeoffreyBooth commented Apr 9, 2024

ronag commented Apr 9, 2024

github-actions bot commented Apr 10, 2024

GeoffreyBooth commented Apr 10, 2024

cjihrig commented Apr 10, 2024

mcollina left a comment

jasnell commented Apr 10, 2024

jasnell Apr 10, 2024

cjihrig commented Apr 10, 2024

benjamingr left a comment

benjamingr commented Apr 11, 2024

mcollina commented Apr 12, 2024

cjihrig commented Apr 12, 2024

benjamingr commented Apr 12, 2024

mcollina commented Apr 12, 2024

nodejs-github-bot commented Jun 14, 2024

deps,lib,src: add experimental web storage #52435

deps,lib,src: add experimental web storage #52435

Conversation

cjihrig commented Apr 9, 2024

nodejs-github-bot commented Apr 9, 2024

anonrig commented Apr 9, 2024

GeoffreyBooth commented Apr 9, 2024

ronag commented Apr 9, 2024

github-actions bot commented Apr 10, 2024

GeoffreyBooth commented Apr 10, 2024

cjihrig commented Apr 10, 2024

mcollina left a comment

Choose a reason for hiding this comment

jasnell commented Apr 10, 2024

jasnell Apr 10, 2024

Choose a reason for hiding this comment

cjihrig commented Apr 10, 2024

benjamingr left a comment

Choose a reason for hiding this comment

benjamingr commented Apr 11, 2024

mcollina commented Apr 12, 2024

cjihrig commented Apr 12, 2024

benjamingr commented Apr 12, 2024

mcollina commented Apr 12, 2024

nodejs-github-bot commented Jun 14, 2024