Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Persistent Playground: Explore OPFS support #544

Closed
wants to merge 1 commit into from
Closed

Conversation

adamziel
Copy link
Collaborator

This commit explores a custom OPFS filesystem backend for Playground to avoid losing changes after a page refresh.

Status: Playground works in read-only mode. Writing and reading entire files and directories works, but there's something off with seeking. For example, inserting a post yields General error: 10 disk I/O error.

I could not reproduce it with a simple fseek() call from PHP, unfortunately.

Perhaps this Emscripten PR contains some clues: https://github.com/emscripten-core/emscripten/pull/16307/files#diff-6558aa76b3879121168686cc9b281ffcf4828c11d7f5aaa325231d744c276d1dR72

Solves #19

cc @dmsnell

This commit explores a custom OPFS filesystem backend for Playground to
avoid losing changes after a page refresh.

Status: Playground works in read-only mode. Writing and reading entire
files and directories works, but there's something off with seeking. For
example, inserting a post yields `General error: 10 disk I/O error.`

I could not reproduce it with a simple fseek() call from PHP,
unfortunately.
const parentDir = root.getDirectory(parentPath, {});
const realpath = PATH.join2(parentPath, node.name);
const filename = realpath.split('/').pop();
root.getFile(realpath, { create: true }).moveTo(
Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There's no touch() method, but I still don't like this hack

@adamziel
Copy link
Collaborator Author

console.log output looks like this:


php_8_0.js?t=1686587334137:366 .ht.sqlite-journal llseek {position: 4620, offset: 4620, whence: 0}
php_8_0.js?t=1686587334137:340 .ht.sqlite-journal write {offset: 20648152, length: 4096, position: 4620, canOwn: undefined}
php_8_0.js?t=1686587334137:366 .ht.sqlite-journal llseek {position: 8716, offset: 8716, whence: 0}
php_8_0.js?t=1686587334137:340 .ht.sqlite-journal write {offset: 7049180, length: 4, position: 8716, canOwn: undefined}
php_8_0.js?t=1686587334137:366 .ht.sqlite llseek {position: 200704, offset: 192512, whence: 0}
php_8_0.js?t=1686587334137:321 .ht.sqlite read {offset: 20643896, length: 4096, position: 192512}
php_8_0.js?t=1686587334137:366 .ht.sqlite-journal llseek {position: 8720, offset: 8720, whence: 0}
php_8_0.js?t=1686587334137:340 .ht.sqlite-journal write {offset: 7049176, length: 4, position: 8720, canOwn: undefined}
php_8_0.js?t=1686587334137:366 .ht.sqlite-journal llseek {position: 8724, offset: 8724, whence: 0}
php_8_0.js?t=1686587334137:340 .ht.sqlite-journal write {offset: 20643896, length: 4096, position: 8724, canOwn: undefined}
php_8_0.js?t=1686587334137:366 .ht.sqlite-journal llseek {position: 12820, offset: 12820, whence: 0}
php_8_0.js?t=1686587334137:340 .ht.sqlite-journal write {offset: 7049180, length: 4, position: 12820, canOwn: undefined}

@adamziel
Copy link
Collaborator Author

This code works, how weird! Must be something specific to $wpdb interaction with SQLite

// Create SQLite database, create a table, insert, update, select some data
$sqlite = new SQLite3('test.sqlite');
// start a transaction
$sqlite->exec('BEGIN');
$sqlite->exec('CREATE TABLE test (id INTEGER PRIMARY KEY, name TEXT)');
$sqlite->exec('INSERT INTO test (name) VALUES ("test")');
$sqlite->exec('UPDATE test SET name = "test2" WHERE id = 1');
// commit
$sqlite->exec('COMMIT');
var_dump($sqlite->querySingle('SELECT name FROM test WHERE id = 1'));

@adamziel
Copy link
Collaborator Author

adamziel commented Jun 12, 2023

Minimal reproduction:

<?php
$sqlite = new SQLite3('/wordpress/wp-content/database/.ht.sqlite');
$sqlite->exec("INSERT INTO wp_options (option_name, option_value) VALUES ('a', 'b');");
var_dump($sqlite->lastErrorCode());

It's the SQLITE_IOERR error constant:

SQLITE_PRIVATE const char *sqlite3ErrStr(int rc){
  static const char* const aMsg[] = {
    /* SQLITE_OK          */ "not an error",
    /* SQLITE_ERROR       */ "SQL logic error",
    /* SQLITE_INTERNAL    */ 0,
    /* SQLITE_PERM        */ "access permission denied",
    /* SQLITE_ABORT       */ "query aborted",
    /* SQLITE_BUSY        */ "database is locked",
    /* SQLITE_LOCKED      */ "database table is locked",
    /* SQLITE_NOMEM       */ "out of memory",
    /* SQLITE_READONLY    */ "attempt to write a readonly database",
    /* SQLITE_INTERRUPT   */ "interrupted",
    /* SQLITE_IOERR       */ "disk I/O error",

It can be returned only by these two SQLite functions:

static int findInodeInfo(
  unixFile *pFile,               /* Unix file with file desc used in the key */
  unixInodeInfo **ppInode        /* Return the unixInodeInfo object here */
);
static int proxyGetHostID(unsigned char *pHostID, int *pError)

The first one is particularly suspicious since SQLITE_IOERR happens on rc = osFstat(fd, &statbuf); failure

adamziel added a commit that referenced this pull request Jun 13, 2023
## Description

#544 explores a full Emscripten OPFS filesystem backend, but 
there is one last issue I may not be able to figure out before my
Sabbatical (June 26th - Sep 26th).

This PR attempts another approach I should be able to ship. Namely,
it synchronizes MEMFS changes to OPFS and restores them after a 
page refresh.

The main idea is:

1. Keep track of all modified files
2. Only sync files on that list

OPFS is only supported in Chrome-based browsers at the moment like Edge,
Android browser. Safari and Firefox users won't be able to benefit from
this feature yet

## Performance

* Full WordPress OPFS->MEMFS: ~340 ms
* Full WordPress MEMFS->OPFS: ~506 ms
* Typical sync MEMFS->OPFS: 2.5 ms

## Other explored approaches

This approach failed:

1. Compare last modified time
2. Copy MEMFS files to OPFS if they were updated more recently

`mtime` doesn't bubble up through directories and comparing all files is
too slow.
@adamziel
Copy link
Collaborator Author

adamziel commented Jun 14, 2023

Closing – this PR uses the old webkit Filesystem spec and is a dead-end. A follow-up should use the same cross-browser API as #547 and wrap all the newly async PHP functions (like stat()) in Asyncify.

@adamziel adamziel closed this Jun 14, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant