diff --git a/AccessHandle.md b/AccessHandle.md deleted file mode 100644 index 01163f3..0000000 --- a/AccessHandle.md +++ /dev/null @@ -1,304 +0,0 @@ -# AccessHandle Proposal - -## Authors: - -* Emanuel Krivoy (fivedots@chromium.org) -* Richard Stotz (rstz@chromium.org) - -## Participate - -* [Issue tracker](https://github.com/WICG/file-system-access/issues) - -## Table of Contents - - - - -- [Introduction](#introduction) -- [Goals & Use Cases](#goals--use-cases) -- [Non-goals](#non-goals) -- [What makes the new surface fast?](#what-makes-the-new-surface-fast) -- [Proposed API](#proposed-api) - - [New data access surface](#new-data-access-surface) - - [Locking semantics](#locking-semantics) -- [Open Questions](#open-questions) - - [Naming](#naming) - - [Assurances on non-awaited consistency](#assurances-on-non-awaited-consistency) -- [Appendix](#appendix) - - [AccessHandle IDL](#accesshandle-idl) -- [References & acknowledgements](#references--acknowledgements) - - - -## Introduction - -We propose augmenting the Origin Private File System (OPFS) with a new surface -that brings very performant access to data. This new surface differs from -existing ones by offering in-place and exclusive write access to a file’s -content. This change, along with the ability to consistently read unflushed -modifications and the availability of a synchronous variant on dedicated -workers, significantly improves performance and unblocks new use cases for the -File System Access API. - -More concretely, we would add a *createAccessHandle()* method to the -*FileSystemFileHandle* object. It would return an *AccessHandle* that contains -a [duplex stream](https://streams.spec.whatwg.org/#other-specs-duplex) and -auxiliary methods. The readable/writable pair in the duplex stream communicates -with the same backing file, allowing the user to read unflushed contents. -Another new method, *createSyncAccessHandle()*, would only be exposed on Worker -threads. This method would offer a more buffer-based surface with synchronous -reading and writing. The creation of AccessHandle also creates a lock that -prevents write access to the file across (and within the same) execution -contexts. - -This proposal is part of our effort to integrate [Storage Foundation -API](https://github.com/WICG/storage-foundation-api-explainer) into File System -Access API. For more context the origins of this proposal, and alternatives -considered, please check out: [Merging Storage Foundation API and the Origin -Private File -System](https://docs.google.com/document/d/121OZpRk7bKSF7qU3kQLqAEUVSNxqREnE98malHYwWec), -[Recommendation for Augmented -OPFS](https://docs.google.com/document/d/1g7ZCqZ5NdiU7oqyCpsc2iZ7rRAY1ZXO-9VoG4LfP7fM). - -Although this proposal is the successor "in spirit" to the Storage Foundation -API, the two APIs operate on entirely different sets of files. There exists no -way of accessing a file stored through Storage Foundation API using the Origin -Private File System, and vice versa. - -## Goals & Use Cases - -Our goal is to give developers flexibility by providing generic, simple, and -performant primitives upon which they can build higher-level storage -components. The new surface is particularly well suited for Wasm-based -libraries and applications that want to use custom storage algorithms to -fine-tune execution speed and memory usage. - -A few examples of what could be done with *AccessHandles*: - -* Distribute a performant Wasm port of SQLite. This gives developers the - ability to use a persistent and fast SQL engine without having to rely on - the deprecated WebSQL API. -* Allow a music production website to operate on large amounts of media, by - relying on the new surface's performance and direct buffered access to - offload sound segments to disk instead of holding them in memory. -* Provide a fast and persistent [Emscripten](https://emscripten.org/) - filesystem to act as generic and easily accessible storage for Wasm. - -## Non-goals - -This proposal is focused only on additions to the [Origin Private File -System](https://wicg.github.io/file-system-access/#sandboxed-filesystem), and -doesn't currently consider changes to the rest of File System Access API or how -files in the host machine are accessed. - -This proposal does not consider accessing files stored using the Storage -Foundation API through OPFS or vice versa. - -## What makes the new surface fast? - -There are a few design choices that primarily contribute to the performance of -AccessHandles: - -* Write operations are not guaranteed to be immediately persistent, rather - persistency is achieved through calls to *flush()*. At the same time, data - can be consistently read before flushing. This allows applications to only - schedule time consuming flushes when they are required for long-term data - storage, and not as a precondition to operate on recently written data. -* The exclusive write lock held by the AccessHandle saves implementations - from having to provide a central data access point across execution - contexts. In multi-process browsers, such as Chrome, this helps avoid costly - inter-process communication (IPCs) between renderer and browser processes. -* Data copies are avoided when reading or writing. In the async surface this - is achieved through SharedArrayBuffers and BYOB readers. In the sync - surface, we rely on user-allocated buffers to hold the data. - -For more information on what affects the performance of similar storage APIs, -see [Design considerations for the Storage Foundation -API](https://docs.google.com/document/d/1cOdnvuNIWWyJHz1uu8K_9DEgntMtedxfCzShI7d01cs) - -## Proposed API - -### New data access surface - -```javascript -// In all contexts -// For details on the `mode` parameter see "Exposing AccessHandles on all -// filesystems" below -const handle = await file.createAccessHandle({ mode: "in-place" }); -await handle.writable.getWriter().write(buffer); -const reader = handle.readable.getReader({ mode: "byob" }); -// Assumes seekable streams, and SharedArrayBuffer support are available -await reader.read(buffer, { at: 1 }); - -// Only in a worker context -const handle = await file.createSyncAccessHandle(); -const writtenBytes = handle.write(buffer); -const readBytes = handle.read(buffer, { at: 1 }); -``` - -As mentioned above, a new *createAccessHandle()* method would be added to -*FileSystemFileHandle*. Another method, *createSyncAccessHandle()*, would be -only exposed on Worker threads. An IDL description of the new interface can be -found in the [Appendix](#appendix). - -The reason for offering a Worker-only synchronous interface, is that consuming -asynchronous APIs from Wasm has severe performance implications (more details -[here](https://docs.google.com/document/d/1lsQhTsfcVIeOW80dr467Auud_VCeAUv2ZOkC63oSyKo)). -Since this overhead is most impactful on methods that are called often, we've -only made *read()* and *write()* synchronous. This allows us to keep a simpler -mental model (where the sync and async handle are identical, except reading and -writing) and reduce the number of new sync methods, while avoiding the most -important perfomance penalties. - -This proposal assumes that [seekable -streams](https://github.com/whatwg/streams/issues/1128) will be available. If -this doesn’t happen, we can emulate the seeking behavior by extending the -default reader and writer with a *seek()* method. - -### Locking semantics - -```javascript -const handle1 = await file.createAccessHandle({ mode: "in-place" }); -try { - const handle2 = await file.createAccessHandle({ mode: "in-place" }); -} catch (e) { - // This catch will always be executed, since there is an open access handle -} -await handle1.close(); -// Now a new access handle may be created -``` - -*createAccessHandle()* would take an exclusive write lock on the file that -prevents the creation of any other access handles or *WritableFileStreams*. -Similarly *createWritable()* would take a shared write lock that blocks the -creation of access handles, but not of other writable streams. This prevents -the file from being modified from multiple contexts, while still being -backwards compatible with the current OPFS spec and supporting multiple -*WritableFileStreams* at once. - -Creating a [File](https://www.w3.org/TR/FileAPI/#dfn-file) through *getFile()* -would be possible when a lock is in place. The returned File behaves as it -currently does in OPFS i.e., it is invalidated if file contents are changed -after it was created. It is worth noting that these Files could be used to -observe changes done through the new API, even if a lock is still being held. - -## Open Questions - -### Naming - -The exact name of the new methods hasn’t been defined. The current placeholder -for data access is *createAccessHandle()* and *createSyncAccessHandle()*. -*createUnflushedStreams()* and *createDuplexStream()* have been suggested. - -### Exposing AccessHandles on all filesystems - -This proposal only currently considers additions to OPFS, but it would probably -be worthwhile to expand the new functionality to arbitrary file handles. While -the exact behavior of *AccessHandles* outside of OPFS would need to be defined -in detail, it's almost certain that the one described in this proposal should -not be the default. To avoid setting it as such, we propose adding an optional -*mode* string parameter to *createAccessHandle()* and -*createSyncAccessHandle()*. Some possible values *mode* could take are: - -* 'shared': The current behavior seen in File System Access API in general, - there is no locking and modifications are atomic (meaning that they would - only actually change the file when the *AccessHandle* is closed). This mode - would be a safe choice as a default value. -* 'exclusive': An exclusive write lock is taken on the file, but modifications - are still atomic. This is a useful mode for developers that want to - coordinate various writing threads but still want "all or nothing" writes. -* 'in-place': The behavior described in this proposal, allowing developers to - use high performance access to files at the cost of not having atomic writes. - It's possible that this mode would only be allowed in OPFS. - -Both the naming and semantics of the *mode* parameter have to be more concretely -defined. - -### Assurances on non-awaited consistency - -It would be possible to clearly specify the behavior of an immediate async read -operation after a non-awaited write operation, by serializing file operations -(as is currently done in Storage Foundation API). We should decide if this is -convenient, both from a specification and performance point of view. - -## Trying It Out - -A prototype of the synchronous surface (i.e., *createSyncAccessHandles()* and -the *FileSystemSyncAccessHandle* object) is available in Chrome. If you're -using version 95 or higher, you can enable it by launching Chrome with the -`--enable-blink-features=FileSystemAccessAccessHandle` flag or enabling -"Experimental Web Platform features" in "chrome://flags". If you're using -version 94, launch Chrome with the -`--enable-features=FileSystemAccessAccessHandle` flag. - -Sync access handles are available in an Origin Trial, starting with Chrome 95. -Sign up -[here](https://developer.chrome.com/origintrials/#/view_trial/3378825620434714625) -to participate. - -We have also developed an Emscripten file system based on access handles. -Instructions on how to use it can be found -[here](https://github.com/rstz/emscripten-pthreadfs/blob/main/pthreadfs/README.md). - -## Appendix - -### AccessHandle IDL - -```webidl -interface FileSystemFileHandle : FileSystemHandle { - Promise getFile(); - Promise createWritable(optional FileSystemCreateWritableOptions options = {}); - - Promise createAccessHandle(optional FileSystemFileHandleCreateAccessHandleOptions options = {}); - [Exposed=DedicatedWorker] - Promise createSyncAccessHandle(optional FileSystemFileHandleCreateAccessHandleOptions options = {}); -}; - -dictionary FileSystemFileHandleCreateAccessHandleOptions { - AccessHandleMode mode; -}; - -// For more details and possible modes, see "Exposing AccessHandles on all -// filesystems" above -enum AccessHandleMode { "in-place" }; - -interface FileSystemAccessHandle { - // Assumes seekable streams are available. The - // Seekable extended attribute is ad-hoc notation for this proposal. - [Seekable] readonly attribute WritableStream writable; - [Seekable] readonly attribute ReadableStream readable; - - // Resizes the file to be size bytes long. If size is larger than the current - // size the file is padded with null bytes, otherwise it is truncated. - Promise truncate([EnforceRange] unsigned long long size); - // Returns the current size of the file. - Promise getSize(); - // Persists the changes that have been written to disk - Promise flush(); - // Flushes and closes the streams, then releases the lock on the file - Promise close(); -}; - -[Exposed=DedicatedWorker] -interface FileSystemSyncAccessHandle { - unsigned long long read([AllowShared] BufferSource buffer, - FilesystemReadWriteOptions options); - unsigned long long write([AllowShared] BufferSource buffer, - FilesystemReadWriteOptions options); - - Promise truncate([EnforceRange] unsigned long long size); - Promise getSize(); - Promise flush(); - Promise close(); -}; - -dictionary FilesystemReadWriteOptions { - [EnforceRange] unsigned long long at; -}; -``` - -## References & acknowledgements - -Many thanks for valuable feedback and advice from: - -Domenic Denicola, Marijn Kruisselbrink, Victor Costan diff --git a/index.bs b/index.bs index 85dc86b..aa4a06e 100644 --- a/index.bs +++ b/index.bs @@ -87,8 +87,48 @@ never be allowed using this API, rather than leaving it entirely up to underlyin systems. A file entry additionally consists of -binary data (a [=byte sequence=]) and a -modification timestamp (a number representing the number of milliseconds since the Unix Epoch). +binary data (a [=byte sequence=]), a +modification timestamp (a number representing the number of milliseconds since the Unix Epoch), +a lock (a string that may exclusively be "`open`", "`taken-exclusive`" or "`taken-shared`") +and a shared lock count (a number representing the number shared locks that are taken at a given point in time). + +
+To take a [=file entry/lock=] with a |value| of "`exclusive`" or "`shared`" on a given [=file entry=] |file|, +run the following steps: + +1. Let |lock| be the |file|'s [=file entry/lock=]. +1. Let |count| be the |file|'s [=file entry/shared lock count=]. +1. If |value| is "`exclusive`": + 1. If |lock| is "`open`": + 1. Set lock to "`taken-exclusive`". + 1. Return true. +1. If |value| is "`shared`": + 1. If |lock| is "`open`": + 1. Set |lock| to "`taken-shared`". + 1. Set |count| to 1. + 1. Return true. + 1. Otherwise, if |lock| is "`taken-shared`": + 1. Increase |count| by one. + 1. Return true. +1. Return false. + +
+ +
+To release a [=file entry/lock=] on a given [=file entry=] |file|, +run the following steps: + +1. Let |lock| be the |file|'s associated [=file entry/lock=]. +1. Let |count| be the |file|'s [=file entry/shared lock count=]. +1. If |lock| is "`taken-shared`": + 1. Decrease |count| by one. + 1. If |count| is 0, set |lock| to "`open`". +1. Otherwise, set |lock| to "`open`". + +
+ +Note: Locks help prevent concurrent modifications to a file. A {{FileSystemWritableFileStream}} +requires a shared lock, while a {{FileSystemSyncAccessHandle}} requires an exclusive one. A directory entry additionally consists of a [=/set=] of children, which are themselves [=/entries=]. Each member is either a [=/file=] or a [=directory=]. @@ -210,8 +250,8 @@ The isSameEntry(|other|) method, when inv 1. Let |p| be [=a new promise=] in |realm|. 1. Run the following steps [=in parallel=]: 1. If [=this=]'s [=FileSystemHandle/entry=] is [=the same as=] |other|'s [=FileSystemHandle/entry=], - [=/resolve=] |p| with `true`. - 1. Else [=/resolve=] |p| with `false`. + [=/resolve=] |p| with true. + 1. Otherwise [=/resolve=] |p| with false. 1. Return |p|. @@ -227,6 +267,8 @@ dictionary FileSystemCreateWritableOptions { interface FileSystemFileHandle : FileSystemHandle { Promise getFile(); Promise createWritable(optional FileSystemCreateWritableOptions options = {}); + [Exposed=DedicatedWorker] + Promise createSyncAccessHandle(); }; @@ -282,9 +324,13 @@ The getFile() method, when invoked, m This is typically implemented by writing data to a temporary file, and only replacing the file represented by |fileHandle| with the temporary file when the writable filestream is closed. - If {{FileSystemCreateWritableOptions/keepExistingData}} is `false` or not specified, + If {{FileSystemCreateWritableOptions/keepExistingData}} is false or not specified, the temporary file starts out empty, otherwise the existing file is first copied to this temporary file. + + Creating a {{FileSystemWritableFileStream}} [=file entry/lock/take|takes a shared lock=] on the + [=FileSystemHandle/entry=] associated with |fileHandle|. This prevents the creation of + {{FileSystemSyncAccessHandle|FileSystemSyncAccessHandles}} for the entry, until the stream is closed. Issue(67): There has been some discussion around and desire for a "inPlace" mode for createWritable @@ -305,15 +351,60 @@ The createWritable(|options|) method, 1. If |access| is not "{{PermissionState/granted}}", reject |result| with a {{NotAllowedError}} and abort. 1. Let |entry| be [=this=]'s [=FileSystemHandle/entry=]. + 1. Let |lockResult| be the result of [=file entry/lock/take|taking a lock=] with "`shared`" on |entry|. + 1. If |lockResult| is false, [=reject=] |result| with a {{NoModificationAllowedError}} and abort. 1. Let |stream| be the result of [=create a new FileSystemWritableFileStream|creating a new FileSystemWritableFileStream=] for |entry| in [=this=]'s [=relevant realm=]. - 1. If |options|.{{FileSystemCreateWritableOptions/keepExistingData}} is `true`: + 1. If |options|.{{FileSystemCreateWritableOptions/keepExistingData}} is true: 1. Set |stream|.[=[[buffer]]=] to a copy of |entry|'s [=file entry/binary data=]. 1. [=/Resolve=] |result| with |stream|. 1. Return |result|. +### The {{FileSystemFileHandle/createSyncAccessHandle()}} method ### {#api-filesystemfilehandle-createsyncaccesshandle} + +
+ : |handle| = await |fileHandle| . {{FileSystemFileHandle/createSyncAccessHandle()|createSyncAccessHandle}}() + :: Returns a {{FileSystemSyncAccessHandle}} that can be used to read from/write to the file. + Changes made through |handle| might be immediately reflected in the file represented by |fileHandle|. + To ensure the changes are reflected in this file, the handle can be flushed or closed. + + Creating a {{FileSystemSyncAccessHandle}} [=file entry/lock/take|takes an exclusive lock=] on the + [=FileSystemHandle/entry=] associated with |fileHandle|. This prevents the creation of + further {{FileSystemSyncAccessHandle}}s or {{FileSystemWritableFileStream}}s + for the entry, until the access handle is closed. + + The returned {{FileSystemSyncAccessHandle}} offers synchronous {{FileSystemSyncAccessHandle/read()}} and + {{FileSystemSyncAccessHandle/write()}} methods. This allows for higher performance for critical methods on + contexts where asynchronous operations come with high overhead, e.g., WebAssembly. + + For the time being, this method will only succeed when the |fileHandle| belongs to the + [=origin private file system=]. +
+ +
+The createSyncAccessHandle() method, when invoked, must run these steps: + +1. Let |result| be [=a new promise=]. +1. Run the following steps [=in parallel=]: + 1. Let |access| be the result of running [=this=]'s [=FileSystemHandle/entry=]'s + [=entry/request access=] given "`readwrite`". + If that throws an exception, [=reject=] |result| with that exception and abort. + 1. If |access| is not "{{PermissionState/granted}}", + reject |result| with a {{NotAllowedError}} and abort. + 1. Let |entry| be [=this=]'s [=FileSystemHandle/entry=]. + 1. If |entry| does not represent an [=/entry=] in an [=origin private file system=], + reject |result| with an {{InvalidStateError}} and abort. + 1. Let |lockResult| be the result of [=file entry/lock/take|taking a lock=] with "`exclusive`" on |entry|. + 1. If |lockResult| is false, [=reject=] |result| with a {{NoModificationAllowedError}} and abort. + 1. Let |handle| be the result of [=create a new FileSystemSyncAccessHandle|creating a new FileSystemSyncAccessHandle=] + for |entry| in [=this=]'s [=relevant realm=]. + 1. [=/Resolve=] |result| with |handle|. +1. Return |result|. + +
+ ## The {{FileSystemDirectoryHandle}} interface ## {#api-filesystemdirectoryhandle} @@ -444,7 +535,7 @@ must run these steps: 1. If |name| is not a [=valid file name=], [=/reject=] |result| with a {{TypeError}} and abort. 1. Let |entry| be [=this=]'s [=FileSystemHandle/entry=]. - 1. If |options|.{{FileSystemGetFileOptions/create}} is `true`: + 1. If |options|.{{FileSystemGetFileOptions/create}} is true: 1. Let |access| be the result of running [=this=]'s [=FileSystemHandle/entry=]'s [=entry/request access=] given "`readwrite`". If that throws an exception, [=reject=] |result| with that exception and abort. @@ -459,7 +550,7 @@ must run these steps: 1. If |child| is a [=directory entry=]: 1. [=/Reject=] |result| with a {{TypeMismatchError}} and abort. 1. [=/Resolve=] |result| with a new {{FileSystemFileHandle}} whose [=FileSystemHandle/entry=] is |child| and abort. - 1. If |options|.{{FileSystemGetFileOptions/create}} is `false`: + 1. If |options|.{{FileSystemGetFileOptions/create}} is false: 1. [=/Reject=] |result| with a {{NotFoundError}} and abort. 1. Let |child| be a new [=file entry=] whose [=query access=] and [=request access=] algorithms are those of |entry|. @@ -506,7 +597,7 @@ invoked, must run these steps: 1. If |name| is not a [=valid file name=], [=/reject=] |result| with a {{TypeError}} and abort. 1. Let |entry| be [=this=]'s [=FileSystemHandle/entry=]. - 1. If |options|.{{FileSystemGetDirectoryOptions/create}} is `true`: + 1. If |options|.{{FileSystemGetDirectoryOptions/create}} is true: 1. Let |access| be the result of running [=this=]'s [=FileSystemHandle/entry=]'s [=entry/request access=] given "`readwrite`". If that throws an exception, [=reject=] |result| with that exception and abort. @@ -521,7 +612,7 @@ invoked, must run these steps: 1. If |child| is a [=file entry=]: 1. [=/Reject=] |result| with a {{TypeMismatchError}} and abort. 1. [=/Resolve=] |result| with a new {{FileSystemDirectoryHandle}} whose [=FileSystemHandle/entry=] is |child| and abort. - 1. If |options|.{{FileSystemGetFileOptions/create}} is `false`: + 1. If |options|.{{FileSystemGetFileOptions/create}} is false: 1. [=/Reject=] |result| with a {{NotFoundError}} and abort. 1. Let |child| be a new [=directory entry=] whose [=query access=] and [=request access=] algorithms are those of |entry|. @@ -574,13 +665,13 @@ these steps: 1. [=set/For each=] |child| of |entry|'s [=directory entry/children=]: 1. If |child|'s [=entry/name=] equals |name|: 1. If |child| is a [=directory entry=]: - 1. If |child|'s [=directory entry/children=] is not [=set/is empty|empty=] and |options|.{{FileSystemRemoveOptions/recursive}} is `false`: + 1. If |child|'s [=directory entry/children=] is not [=set/is empty|empty=] and |options|.{{FileSystemRemoveOptions/recursive}} is false: 1. [=/Reject=] |result| with an {{InvalidModificationError}} and abort. 1. [=set/Remove=] |child| from |entry|'s [=directory entry/children=]. 1. If removing |child| in the underlying file system throws an exception, [=/reject=] |result| with that exception and abort. - Note: If {{FileSystemRemoveOptions/recursive}} is `true`, the removal can fail + Note: If {{FileSystemRemoveOptions/recursive}} is true, the removal can fail non-atomically. Some files or directories might have been removed while other files or directories still exist. @@ -724,18 +815,23 @@ in a [=/Realm=] |realm|, perform the following steps: reject |closeResult| with a {{NotAllowedError}} and abort. 1. Perform [=implementation-defined=] malware scans and safe browsing checks. If these checks fail, [=/reject=] |closeResult| with an {{AbortError}} and abort. - 1. Set |stream|.[=[[file]]=]'s [=file entry/binary data=] to |stream|.[=[[buffer]]=]. + 1. Set |stream|.[=FileSystemWritableFileStream/[[file]]=]'s [=file entry/binary data=] to |stream|.[=[[buffer]]=]. If that throws an exception, [=/reject=] |closeResult| with that exception and abort. Note: It is expected that this atomically updates the contents of the file on disk being written to. + + 1. [=file entry/lock/release|Release the lock=] on |stream|.[=FileSystemWritableFileStream/[[file]]=]. 1. [=/Resolve=] |closeResult| with `undefined`. 1. Return |closeResult|. +1. Let |abortAlgorithm| be the following step: + 1. [=file entry/lock/release|Release the lock=] on |stream|.[=FileSystemWritableFileStream/[[file]]=]. 1. Let |highWaterMark| be 1. 1. Let |sizeAlgorithm| be an algorithm that returns `1`. 1. [=WritableStream/Set up=] |stream| with <a for="WritableStream/set up"><var ignore>writeAlgorithm</var></a> set to |writeAlgorithm|, <a for="WritableStream/set up"><var ignore>closeAlgorithm</var></a> set to |closeAlgorithm|, <a for="WritableStream/set up"><var + ignore>abortAlgorithm</var></a> set to |abortAlgorithm|, <a for="WritableStream/set up"><var ignore>highWaterMark</var></a> set to |highWaterMark|, and <a for="WritableStream/set up"><var ignore>sizeAlgorithm</var></a> set to |sizeAlgorithm|. 1. Return |stream|. @@ -768,15 +864,15 @@ runs these steps: 1. Let |oldSize| be |stream|.[=[[buffer]]=]'s [=byte sequence/length=]. 1. If |data| is a {{BufferSource}}, let |dataBytes| be [=get a copy of the buffer source|a copy of=] |data|. - 1. Else if |data| is a {{Blob}}: + 1. Otherwise, if |data| is a {{Blob}}: 1. Let |dataBytes| be the result of performing the <a spec=FileAPI>read operation</a> on |data|. If this throws an exception, [=/reject=] |p| with that exception and abort. - 1. Else: + 1. Otherwise: 1. [=Assert=]: |data| is a {{USVString}}. 1. Let |dataBytes| be the result of [=UTF-8 encoding=] |data|. 1. If |writePosition| is larger than |oldSize|, - append |writePosition| - |oldSize| `0x00` (NUL) bytes to the end of |stream|.[=[[buffer]]=]. + append |writePosition| - |oldSize| 0x00 (NUL) bytes to the end of |stream|.[=[[buffer]]=]. Note: Implementations are expected to behave as if the skipped over file contents are indeed filled with NUL bytes. That doesn't mean these bytes have to actually be @@ -798,12 +894,12 @@ runs these steps: to runs out of disk space. 1. Set |stream|.[=[[seekOffset]]=] to |writePosition| + |data|.[=byte sequence/length=]. 1. [=/Resolve=] |p|. - 1. Else if |command| is {{WriteCommandType/"seek"}}: + 1. Otherwise, if |command| is {{WriteCommandType/"seek"}}: 1. If |chunk|.{{WriteParams/position}} is `undefined`, [=/reject=] |p| with a {{TypeError}} and abort. 1. Set |stream|.[=[[seekOffset]]=] to |chunk|.{{WriteParams/position}}. 1. [=/Resolve=] |p|. - 1. Else if |command| is {{WriteCommandType/"truncate"}}: + 1. Otherwise, if |command| is {{WriteCommandType/"truncate"}}: 1. If |chunk|.{{WriteParams/size}} is `undefined`, [=/reject=] |p| with a {{TypeError}} and abort. 1. Let |newSize| be |chunk|.{{WriteParams/size}}. @@ -818,7 +914,7 @@ runs these steps: Note: [=Storage quota=] only applies to files stored in the [=origin private file system=]. However this operation could still fail for other files, for example if the disk being written to runs out of disk space. - 1. Else if |newSize| is smaller than |oldSize|: + 1. Otherwise, if |newSize| is smaller than |oldSize|: 1. Set |stream|.[=[[buffer]]=] to a [=byte sequence=] containing the first |newSize| bytes in |stream|.[=[[buffer]]=]. 1. If |stream|.[=[[seekOffset]]=] is bigger than |newSize|, @@ -863,8 +959,8 @@ runs these steps: :: Resizes the file associated with |stream| to be |size| bytes long. If |size| is larger than the current file size this pads the file with null bytes, otherwise it truncates the file. - The file cursor is updated when {{truncate}} is called. If the offset is smaller than offset, - it remains unchanged. If the offset is larger than |size|, the offset is set to |size| to + The file cursor is updated when {{truncate}} is called. If the cursor is smaller than |size|, + it remains unchanged. If the cursor is larger than |size|, it is set to |size| to ensure that subsequent writes do not error. No changes are written to the actual file until on disk until the stream has been closed. @@ -910,8 +1006,8 @@ steps: :: Resizes the file associated with |stream| to be |size| bytes long. If |size| is larger than the current file size this pads the file with null bytes, otherwise it truncates the file. - The file cursor is updated when {{truncate}} is called. If the offset is smaller than offset, - it remains unchanged. If the offset is larger than |size|, the offset is set to |size| to + The file cursor is updated when {{truncate}} is called. If the cursor is smaller than |size|, + it remains unchanged. If the cursor is larger than |size|, it is set to |size| to ensure that subsequent writes do not error. No changes are written to the actual file until on disk until the stream has been closed. @@ -931,6 +1027,227 @@ steps: </div> +## The {{FileSystemSyncAccessHandle}} interface ## {#api-filesystemsyncaccesshandle} + +<xmp class=idl> + +dictionary FileSystemReadWriteOptions { + [EnforceRange] required unsigned long long at = 0; +}; + +[Exposed=DedicatedWorker, SecureContext] +interface FileSystemSyncAccessHandle { + unsigned long long read([AllowShared] BufferSource buffer, + optional FileSystemReadWriteOptions options = {}); + unsigned long long write([AllowShared] BufferSource buffer, + optional FileSystemReadWriteOptions options = {}); + + Promise<undefined> truncate([EnforceRange] unsigned long long newSize); + Promise<unsigned long long> getSize(); + Promise<undefined> flush(); + Promise<undefined> close(); +}; + + + +A {{FileSystemSyncAccessHandle}} has an associated \[[file]] +(a [=file entry=]). + +A {{FileSystemSyncAccessHandle}} has an associated \[[state]], +a string that may exclusively be "`open`" or "`closed`". + +A {{FileSystemSyncAccessHandle}} is an object that is capable of reading from/writing to, +as well as obtaining and changing the size of, a single file. + +The {{FileSystemSyncAccessHandle/read()}} and {{FileSystemSyncAccessHandle/write()}} methods are synchronous. +This allows for higher performance for critical methods on contexts where asynchronous +operations come with high overhead, e.g., WebAssembly. + +
+To create a new FileSystemSyncAccessHandle given a [=file entry=] |file| +in a [=/Realm=] |realm|, perform the following steps: + +1. Let |handle| be a [=new=] {{FileSystemSyncAccessHandle}} in |realm|. +1. Set |handle|.[=FileSystemSyncAccessHandle/[[file]]=] to |file|. +1. Set |handle|.[=FileSystemSyncAccessHandle/[[state]]=] to "`open`". +1. Return |handle|. + +
+ +### The {{FileSystemSyncAccessHandle/read()}} method ### {#api-filesystemsyncaccesshandle-read} + +
+ : |handle| . {{FileSystemSyncAccessHandle/read()|read}}(|buffer|) + : |handle| . {{FileSystemSyncAccessHandle/read()|read}}(|buffer|, { {{FileSystemReadWriteOptions/at}} }) + :: Reads the contents of the file associated with |handle| into |buffer|, optionally at a given offset. +
+ +// TODO(fivedots): Specify how Access Handles should react when reading from a file that has been modified externally. +
+The read(|buffer|, {{FileSystemReadWriteOptions}}: |options|) method, when invoked, must run +these steps: + +1. If [=this=].[=[[state]]=] is "`closed`", throw an {{InvalidStateError}}. +1. Let |bufferSize| be |buffer|'s [=byte length=]. +1. Let |fileContents| be [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=]. +1. Let |fileSize| be |fileContents|'s [=byte sequence/length=]. +1. Let |readStart| be |options|.{{FileSystemReadWriteOptions/at}}. +1. If |readStart| is larger than |fileSize|, return 0. +1. Let |readEnd| be |readStart| + (|bufferSize| − 1). +1. If |readEnd| is larger than |fileSize|, set |readEnd| to |fileSize|. +1. Let |bytes| be a [=byte sequence=] containing the bytes from |readStart| to |readEnd| of |fileContents|. +1. Let |result| be |bytes|'s [=byte sequence/length=]. +1. If the operations reading from |fileContents| in the previous steps failed: + 1. If there were partial reads and the number of bytes that were read into |bytes| is known, + set |result| to the number of read bytes. + 1. Otherwise set |result| to 0. +1. Let |arrayBuffer| be |buffer|'s [=underlying buffer=]. +1. [=ArrayBuffer/write|Write=] |bytes| into |arrayBuffer|. +1. Return |result|. + +
+ +### The {{FileSystemSyncAccessHandle/write()}} method ### {#api-filesystemsyncaccesshandle-write} + +
+ : |handle| . {{FileSystemSyncAccessHandle/write()|write}}(|buffer|) + : |handle| . {{FileSystemSyncAccessHandle/write()|write}}(|buffer|, { {{FileSystemReadWriteOptions/at}} }) + :: Writes the content of |buffer| into the file associated with |handle|, optionally at a given offset, and returns the number of written bytes. + Checking the returned number of written bytes allows callers to detect and handle errors and partial writes. +
+ +// TODO(fivedots): Figure out how to properly check the available storage quota (in this method and others) by passing the right storage shelf. +
+The write(|buffer|, {{FileSystemReadWriteOptions}}: |options|) method, when invoked, must run +these steps: + +1. If [=this=].[=[[state]]=] is "`closed`", throw a {{InvalidStateError}}. +1. Let |writePosition| be |options|.{{FileSystemReadWriteOptions/at}}. +1. Let |fileContents| be a copy of [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=]. +1. Let |oldSize| be |fileContents|'s [=byte sequence/length=]. +1. Let |bufferSize| be |buffer|'s [=byte length=]. +1. If |writePosition| is larger than |oldSize|, + append |writePosition| − |oldSize| 0x00 (NUL) bytes to the end of |fileContents|. + + Note: Implementations are expected to behave as if the skipped over file contents + are indeed filled with NUL bytes. That doesn't mean these bytes have to actually be + written to disk and take up disk space. Instead most file systems support so called + sparse files, where these NUL bytes don't take up actual disk space. + +1. Let |head| be a [=byte sequence=] containing the first |writePosition| bytes of |fileContents|. +1. Let |tail| be an empty [=byte sequence=]. +1. If |writePosition| + |bufferSize| is smaller than |oldSize|: + 1. Set |tail| to a [=byte sequence=] containing the last + |oldSize| − (|writePosition| + |bufferSize|) bytes of |fileContents|. +1. Let |newSize| be |head|'s [=byte sequence/length=] + |bufferSize| + |tail|'s [=byte sequence/length=]. +1. If |newSize| − |oldSize| exceeds the available [=storage quota=], throw a {{QuotaExceededError}}. +1. Set [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=] to the concatenation of |head|, the contents of |buffer| and |tail|. + + Note: The mechanism used to access buffer's contents is left purposely vague. + It is likely that implementations will choose to focus on performance by issuing + direct write calls to the host operating system (instead of creating a copy of buffer), + which prevents a detailed specification of the write order and the results of partial writes. + +1. If the operations modifying the [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=] in the previous steps + failed: + 1. If there were partial writes and the number of bytes that were written from |buffer| is known, + return the number of bytes that were written from |buffer|. + 1. Otherwise throw an {{InvalidStateError}}. +1. Return |bufferSize|. + +
+ +### The {{FileSystemSyncAccessHandle/truncate()}} method ### {#api-filesystemsyncaccesshandle-truncate} + +
+ : |handle| . {{FileSystemSyncAccessHandle/truncate()|truncate}}(|newSize|) + :: Resizes the file associated with stream to be |newSize| bytes long. If size is larger than the current file size this pads the file with null bytes; otherwise it truncates the file. +
+ +
+The truncate(|newSize|) method, when invoked, must run +these steps: + +1. If [=this=].[=[[state]]=] is "`closed`", return [=a promise rejected with=] an {{InvalidStateError}}. +1. Let |fileContents| be a copy of [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=]. +1. Let |p| be [=a new promise=] created in the [=relevant Realm=] of [=this=]. +1. Run the following steps [=in parallel=]: + 1. Let |oldSize| be the [=byte sequence/length=] of [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=]. + 1. If |newSize| is larger than |oldSize|: + 1. If |newSize| − |oldSize| exceeds the available [=storage quota=], [=/reject=] |p| + with a {{QuotaExceededError}} and abort. + 1. Set [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s to a [=byte sequence=] formed by concatenating + |fileContents| with a [=byte sequence=] + containing |newSize| − |oldSize| 0x00 bytes. + 1. If the operations modifying the [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=] in the previous steps + failed, [=/reject=] |p| with a {{InvalidStateError}} and abort. + 1. Otherwise, if |newSize| is smaller than |oldSize|: + 1. Set [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s to a [=byte sequence=] containing the first |newSize| bytes + in |fileContents|. + 1. If the operations modifying the [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=] in the previous steps + failed, [=/reject=] |p| with a {{InvalidStateError}} and abort. + 1. [=/Resolve=] |p|. +1. Return |p|. + +
+ +### The {{FileSystemSyncAccessHandle/getSize()}} method ### {#api-filesystemsyncaccesshandle-getsize} + +
+ : |handle| . {{FileSystemSyncAccessHandle/getSize()}} + :: Returns the size of the file associated with |handle| in bytes. +
+ +
+The getSize() method, when invoked, must run +these steps: + +1. If [=this=].[=[[state]]=] is "`closed`", return [=a promise rejected with=] an {{InvalidStateError}}. +1. Let |p| be [=a new promise=] created in the [=relevant Realm=] of [=this=]. +1. Run the following steps [=in parallel=]: + 1. Let |size| be the [=byte sequence/length=] of [=this=].[=FileSystemSyncAccessHandle/[[file]]=]'s [=file entry/binary data=]. + 1. [=/Resolve=] |p| with |size|. +1. Return |p|. + + +
+ +### The {{FileSystemSyncAccessHandle/flush()}} method ### {#api-filesystemsyncaccesshandle-flush} + +
+ : |handle| . {{FileSystemSyncAccessHandle/flush()}} + :: Ensures that the contents of the file associated with |handle| contain all the modifications done through {{FileSystemSyncAccessHandle/write()}}. +
+ +
+The flush() method, when invoked, must run +these steps: + +// TODO(fivedots): Fill in, after figuring out language to describe flushing at the OS level. + +
+ +### The {{FileSystemSyncAccessHandle/close()}} method ### {#api-filesystemsyncaccesshandle-close} + +
+ : |handle| . {{FileSystemSyncAccessHandle/close()}} + :: Flushes the access handle and then closes it. Closing an access handle disables any further operations on it and + [=file entry/lock/release|releases the lock=] on the [=FileSystemHandle/entry=] associated with |handle|. +
+ +//TODO(fivedots): Figure out language to describe flushing the file at the OS level before closing the handle. +
+The close() method, when invoked, must run +these steps: + +1. Let |p| be [=a new promise=] created in the [=relevant Realm=] of [=this=]. +1. Run the following steps [=in parallel=]: + 1. Set [=this=].[=[[state]]=] to "`closed`". + 1. [=/Resolve=] |p|. +1. Return |p|. + +
+ # Accessing the Origin Private File System # {#sandboxed-filesystem}