From 25ac6d4a5d0a51c8adbf871ac107e3998a3dbf92 Mon Sep 17 00:00:00 2001 From: Anne van Kesteren Date: Fri, 15 May 2020 13:17:31 +0200 Subject: [PATCH] Finally define storage infrastructure This defines the underpinnings of storage endpoints so there is a single coherent model for all storage needs of the web platform. Closes #18. --- storage.bs | 393 ++++++++++++++++++++++++++++++++++++++++++++--------- 1 file changed, 329 insertions(+), 64 deletions(-) diff --git a/storage.bs b/storage.bs index bafbdc9..1644e1f 100644 --- a/storage.bs +++ b/storage.bs @@ -7,6 +7,12 @@ Abstract: The Storage Standard defines an API for persistent storage and quota e Translation: ja https://triple-underscore.github.io/storage-ja.html +
+urlPrefix: https://tc39.github.io/ecma262/; spec: ECMASCRIPT
+    text: agent; url: #sec-agents; type: dfn
+    text: agent cluster; url: #sec-agent-clusters; type: dfn
+
+

Introduction

@@ -83,9 +89,9 @@ function retrieveNextChunk(nextChunkInfo) { -

Infrastructure

+

Lay of the land

-

A user agent has various kinds of semi-persistent state: +

A user agent has various kinds of semi-persistent state:

Credentials @@ -94,37 +100,293 @@ function retrieveNextChunk(nextChunkInfo) {

Permissions for various features, such as geolocation

Network

HTTP cache, cookies, authentication entries, TLS client certificates -

Storage +
Storage
Indexed DB, Cache API, service worker registrations, localStorage, history.pushState(), application caches, notifications, etc.
-

This standard primarily concerns itself with storage. +

This standard primarily concerns itself with storage. + + + +

Model

+ +

Standards defining local or session storage APIs will define a storage endpoint and +register it by changing this standard. They will use either +obtain a local storage bottle map or obtain a session storage bottle map, which +will give them: + +

+ +

If you are defining a standard for such an API, consider filing an issue agains this +standard for assistance and review. + +

To isolate this data this standard defines a storage shed which segments +storage shelves by a storage key. A storage shelf in turn consists of a +storage bucket and will likely consist of multiple storage buckets in the future to +allow for different storage policies. And lastly, a storage bucket consists of +storage bottles, one for each storage endpoint. + + +

Storage endpoints

+ +

A storage endpoint is a local or +session storage API that uses the infrastructure defined by this standard to keep track of +its storage needs. + +

A storage endpoint has an identifier, which is a +storage identifier. + +

A storage endpoint also has types, which is a +set of storage types. + +

A storage identifier is an ASCII string. + +

A storage type is "local" or "session". + +

+ +

The registered storage endpoints are a set of storage endpoints +defined by the following table: + + + + + + + + + +
Storage identifier + Storage types +
"caches" + « "local" » +
"indexedDB" + « "local" » +
"localStorage" + « "local" » +
"serviceWorkerRegistrations" + « "local" » +
"sessionStorage" + « "session" » +
+ +

As mentioned, standards can use these storage identifiers with +obtain a local storage bottle map and obtain a session storage bottle map. It is +anticipated that some APIs will be applicable to both storage types going forward. + + + +

Storage sheds

+ +

A storage key is an origin. [[HTML]] + +

This is expected to change, see +Client-Side Storage Partitioning. + +

A storage shed is a map of storage keys to storage shelves. +It is initially empty. + +

+ +

A user agent holds a storage shed, which is a +storage shed. A user agent's storage shed holds all +local storage data. + +

A browsing session holds a storage shed, which is a +storage shed. A browsing session's storage shed holds all +session storage data. + +

See +whatwg/html issue #4782 and +whatwg/html issue #5350 for defining +browsing session. It is roughly analogous to top-level browsing context except that it cannot +be replaced due to Cross-Origin-Opener-Policy or navigation. + + +

Storage shelves

+ +

A storage shelf exists for +each storage key within a storage shed. It holds a bucket map, +which is a map of strings to storage buckets. + +

For now "default" is the only key that exists in a +bucket map. See issue #2. It is +given a value when a storage shelf is +obtained for the first time. + +

To obtain a storage shelf, given a storage shed shed, an +environment settings object environment, and a storage type +type, run these steps: + +

    +

  1. Let key be environment's + origin. + +

  2. If key is an opaque origin, then return failure. + +

  3. If the user has disabled storage, then return failure. + +

  4. +

    If shed[key] does not exist, then: + +

      +

    1. Let shelf be a new storage shelf. + +

    2. Set shelf's bucket map["default"] to the result of + create a storage bucket with type. + +

    3. Set shed[key] to shelf. +

    + +

  5. Return shed[key]. +

+ +

To obtain a local storage shelf, given an environment settings object +environment, return the result of running obtain a storage shelf with the user +agent's storage shed, environment, and "local". + + +

Storage buckets

-

Storage consists of zero or more storage units. +

A storage bucket is a place for storage endpoints to store data. Whenever a +storage bucket is cleared by the user agent, it must be cleared in its entirety. + -

Each origin has an associated storage unit. A storage unit contains a -single bucket. [[HTML]] +

A storage bucket has a bottle map of storage identifiers to +storage bottles. +

-

Buckets

+

A local storage bucket is a storage bucket for +local storage APIs. -

A bucket has mode which is either -"best-effort" or "persistent". A -persistent bucket is a bucket whose -mode is "persistent". A -non-persistent bucket is a bucket whose -mode is not "persistent". +

A local storage bucket has a +mode, which is +"best-effort" or "persistent". It is initially "best-effort". -

A bucket is considered to be an atomic unit. Whenever a bucket is cleared by the -user agent, it must be cleared in its entirety. +

+ +

A session storage bucket is a storage bucket for session storage APIs. + +

+ +

To create a storage bucket, given a storage type type, run these +steps: + +

    +

  1. Let bucket be null. + +

  2. If type is "local", then set bucket to a new + local storage bucket. + +

  3. +

    Otherwise: + +

      +

    1. Assert: type is "session". + +

    2. Set bucket to a new session storage bucket. +

    + +

  4. For each endpoint of registered storage endpoints whose + types contain type, set bucket's + bottle map[endpoint's identifier] to a new + storage bottle. + +

  5. Return bucket. +

+ + +

Storage bottles

+ +

A storage bottle is a part of a storage bucket carved out for a single +storage endpoint. A storage bottle has a map, which is +initially an empty map. A storage bottle also has a +proxy map reference set, which is initially an empty +set. + +

A storage bottle's map is where the actual data meant to be +stored lives. User agents are expected to store this data, and make it available across agent +and even agent cluster boundaries, in an implementation-defined manner, so that this +standard and standards using this standard can access the contents. + +

+ +

To obtain a storage bottle map, given a storage type type, +environment settings object environment, and storage identifier +identifier, run these steps:

+ +
    +

  1. Let shed be null. + +

  2. If type is "local", then set shed to the user agent's + storage shed. + +

  3. +

    Otherwise: + +

      +

    1. Assert: type is "session". + +

    2. Set shed to environment's + browsing session's + storage shed. +

    + +

  4. Let shelf be the result of running obtain a storage shelf, with + shed, environment, and type. + +

  5. If shelf is failure, then return failure. + +

  6. Let bucket be shelf's bucket map["default"]. + +

  7. Let bottle be bucket's bottle map[identifier]. + +

  8. Let proxyMap be a new storage proxy map whose + backing map is bottle's map. + +

  9. Append proxyMap to bottle's + proxy map reference set. + +

  10. Return proxyMap. +

+ +

To obtain a local storage bottle map, given an +environment settings object environment and storage identifier +identifier, return the result of running obtain a storage bottle map with +"local", environment, and identifier. + +

To obtain a session storage bottle map, given an +environment settings object environment and storage identifier +identifier, return the result of running obtain a storage bottle map with +"session", environment, and identifier. + + +

Storage proxy maps

+ +

A storage proxy map is equivalent to a map, except that all operations +are instead performed on its backing map. + +

This allows for the backing map to be replaced. This +is needed for issue #4 and potentially the +Storage Access API.

Persistence permission

-

A bucket can only be turned into a persistent bucket if the user (or user agent -on behalf of the user) has granted permission to use the {{"persistent-storage"}} feature. +

A local storage bucket can only have its mode change to +"persistent" if the user (or user agent on behalf of the user) has granted permission +to use the {{"persistent-storage"}} feature.

When granted to an origin, the persistence permission can be used to protect storage from the user agent's clearing policies. The user agent cannot clear storage marked @@ -136,35 +398,41 @@ locally. powerful feature's permission-related flags, algorithms, and types are defaulted, except for:

-
permission state
-
{{"persistent-storage"}}'s permission state must have the same value for all - environment settings objects with a given origin.
- -
permission revocation algorithm
-
If {{"persistent-storage"}}'s permission state is not - {{"granted"}}, then set the current origin’s storage unit's bucket's - mode to "best-effort".
+
permission state +

{{"persistent-storage"}}'s permission state must have the same value for all + environment settings objects with a given origin. + +

permission revocation algorithm +
+
    +

  1. If {{"persistent-storage"}}'s permission state is {{"granted"}}, then return. + +

  2. Let shelf be the result of running obtain a local storage shelf with + current settings object. + +

  3. Set shelf's bucket map["default"]'s + mode to "best-effort". +

Usage and quota

-

The storage usage of an origin origin is a rough -estimate of the amount of bytes used in origin's storage unit. +

The storage usage of a storage shelf is a rough estimate of the amount +of bytes used by it.

This cannot be an exact amount as user agents might, and are encouraged to, use -deduplication, compression, and other techniques that obscure exactly how much bytes an -origin uses. +deduplication, compression, and other techniques that obscure exactly how much bytes a +storage shelf uses. -

The storage quota of an origin origin is a conservative -estimate of the amount of bytes available to origin's storage unit. This amount -should be less than the total available storage space on the device to give users some wiggle room. +

The storage quota of a storage shelf is a conservative estimate of the +amount of bytes available to it. This amount should be less than the total available storage space +on the device to give users some wiggle room. -

User agents are strongly encouraged to provide "popular" origins with a -larger storage quota. Factors such as navigation frequency, recency of visits, bookmarking, -and permission for {{"persistent-storage"}} can be used as indications of -"popularity". +

User agents are strongly encouraged to consider navigation frequency, recency of +visits, bookmarking, and permission for {{"persistent-storage"}} when +evaluating quotas. @@ -204,8 +472,8 @@ WorkerNavigator includes NavigatorStorage;

Each environment settings object has an associated {{StorageManager}} object. [[HTML]] -

The storage attribute's getter must return -context object's relevant settings object's {{StorageManager}} object. +

The storage getter steps are to return +this's relevant settings object's {{StorageManager}} object. 

 [SecureContext,
@@ -223,25 +491,24 @@ dictionary StorageEstimate {
 };
-

The persisted() method, when invoked, must run -these steps: +

The persisted() method steps are:

  1. Let promise be a new promise. -

  2. Let origin be context object's relevant settings object's - origin. +

  3. Let shelf be the result of running obtain a local storage shelf with + this's relevant settings object. -

  4. If origin is an opaque origin, then reject promise with a - {{TypeError}}. +

  5. If shelf is a failure, then reject promise with a {{TypeError}}.

  6. Otherwise, run these steps in parallel:

    1. -

      Let persisted be true if origin's storage unit's bucket - is a persistent bucket, and false otherwise. +

      Let persisted be true if shelf's + bucket map["default"]'s mode is + "persistent"; otherwise false.

      It will be false when there's an internal error. @@ -251,17 +518,15 @@ these steps:

    2. Return promise.

    -

    The persist() method, when invoked, must run -these steps: +

    The persist() method steps are:

    1. Let promise be a new promise. -

    2. Let origin be context object's relevant settings object's - origin. +

    3. Let shelf be the result of running obtain a local storage shelf with + this's relevant settings object. -

    4. If origin is an opaque origin, then reject promise with a - {{TypeError}}. +

    5. If shelf is a failure, then reject promise with a {{TypeError}}.

    6. Otherwise, run these steps in parallel: @@ -275,9 +540,11 @@ these steps: the same origin around the same time and this algorithm is not equipped to handle such a scenario. +

    7. Let bucket be shelf's bucket map["default"]. +

    8. -

      Let persisted be true, if origin's storage unit's bucket - is a persistent bucket, and false otherwise. +

      Let persisted be true if bucket's + mode is "persistent"; otherwise false.

      It will be false when there's an internal error. @@ -285,7 +552,7 @@ these steps:

      If persisted is false and permission is {{"granted"}}, then:

        -

      1. Set origin's storage unit's bucket's mode to +

      2. Set bucket's mode to "persistent".

      3. If there was no internal error, then set persisted to true. @@ -297,25 +564,23 @@ these steps:

      4. Return promise.

      -

      The estimate() method, when invoked, must run -these steps: +

      The estimate() method steps are:

      1. Let promise be a new promise. -

      2. Let origin be context object's relevant settings object's - origin. +

      3. Let shelf be the result of running obtain a local storage shelf with + this's relevant settings object. -

      4. If origin is an opaque origin, then reject promise with a - {{TypeError}}. +

      5. If shelf is a failure, then reject promise with a {{TypeError}}.

      6. Otherwise, run these steps in parallel:

          -

        1. Let usage be storage usage for origin. +

        2. Let usage be storage usage for shelf. -

        3. Let quota be storage quota for origin. +

        4. Let quota be storage quota for shelf.

        5. Let dictionary be a new {{StorageEstimate}} dictionary whose {{usage}} member is usage and {{quota}} member is quota.