From d0336de88e20485ca8d45c4f09dc23a2e2ccb042 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Mon, 19 Apr 2021 21:05:06 -0700 Subject: [PATCH 01/10] updates to make clear root can be catalog or collection --- catalog-spec/catalog-spec.md | 59 ++++++++++++------------------ collection-spec/collection-spec.md | 4 +- overview.md | 22 ++++++----- 3 files changed, 39 insertions(+), 46 deletions(-) diff --git a/catalog-spec/catalog-spec.md b/catalog-spec/catalog-spec.md index 573e848a..b3fa521c 100644 --- a/catalog-spec/catalog-spec.md +++ b/catalog-spec/catalog-spec.md @@ -1,8 +1,7 @@ -# STAC Catalog Specification +# STAC Catalog Specification - [Catalog fields](#catalog-fields) - [Additional Field Information](#additional-field-information) - - [stac_version](#stac_version) - [stac_extensions](#stac_extensions) - [Link Object](#link-object) - [Relation types](#relation-types) @@ -11,18 +10,15 @@ - [STAC Media Types](#stac-media-types) - [Extensions](#extensions) -This document explains the structure and content of a STAC **Catalog** object. A STAC Catalog object -represents a logical group of other Catalog, -[Collection](../collection-spec/collection-spec.md), and [Item](../item-spec/item-spec.md) objects. -These Items can be linked to directly from a Catalog, or the Catalog can link to other Catalogs (often called -sub-catalogs) that contain links to Collections and Items. The division of sub-catalogs is up to the implementor, -but is generally done to aid the ease of online browsing by people. +This document explains the structure and content of a STAC Catalog. A STAC Catalog is a +[Collection](../collection-spec/collection-spec.md) of [STAC Items](../item-spec/item-spec.md). These Items can be +linked to directly from a Catalog, or the Catalog can link to other Catalogs (often called sub-Catalogs) that contain +links to Items. The division of sub-catalogs is up to the implementor, but is generally done to aid the ease of +online browsing by people. -A Catalog object will typically be the entry point into a STAC catalog. Their -purpose is discovery: to be browsed by people or be crawled -by clients to build a searchable index. - -Any JSON object that contains all the required fields is a valid STAC Catalog object. +Catalogs are not intended to be queried. Their purpose is discovery: to be browsed by people and crawled +by machines to build a search index. A Catalog can be represented in JSON format. Any JSON object +that contains all the required fields is a valid STAC Catalog. - [Examples](../examples/) - See an example [catalog.json](../examples/catalog.json). The [collection.json](../examples/collection.json) is also a valid @@ -43,8 +39,8 @@ also a valid STAC Catalog. | Element | Type | Description | | --------------- | ------------- | ------------------------------------------------------------ | +| stac_version | string | **REQUIRED.** The STAC version the Catalog implements. STAC versions can be mixed, but please keep the [recommended best practices](../best-practices.md#mixing-stac-versions) in mind. | | type | string | **REQUIRED.** Set to `Catalog` if this Catalog only implements the Catalog spec. | -| stac_version | string | **REQUIRED.** The STAC version the Catalog implements. | | stac_extensions | \[string] | A list of extension identifiers the Catalog implements. | | id | string | **REQUIRED.** Identifier for the Catalog. | | title | string | A short descriptive one-line title for the Catalog. | @@ -54,29 +50,22 @@ also a valid STAC Catalog. ### Additional Field Information -#### stac_version - -In general, STAC versions can be mixed, but please keep the [recommended best practices](../best-practices.md#mixing-stac-versions) in mind. - #### stac_extensions -A list of extensions the Catalog implements. -The list consists of URLs to JSON Schema files that can be used for validation. -This list must only contain extensions that extend the Catalog specification itself, -see the 'Scope' for each of the extensions. -This must **not** declare the extensions that are only implemented in child Collection objects or child Item objects. +A list of extensions the Catalog implements. This does NOT declare the extensions of children or Items. The list contains URLs to the JSON Schema files it can be validated against. For [core extensions](../extensions/README.md#core-stac-extensions), a "shortcut" can be used. This means you can specify the folder name of the extension, for example `view` for the View extension. If the versions of the extension and the Catalog diverge, you can specify the URL of the JSON schema file. +This list must only contain extensions that extend the Catalog itself, see the the 'Scope' column in the list of extensions. ### Link Object This object describes a relationship with another entity. Data providers are advised to be liberal with links. -| Field Name | Type | Description | -| ---------- | ------ | ----------- | -| href | string | **REQUIRED.** The actual link in the format of an URL. Relative and absolute links are both allowed. | +| Field Name | Type | Description | +| ---------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------- | +| href | string | **REQUIRED.** The actual link in the format of an URL. Relative and absolute links are both allowed. | | rel | string | **REQUIRED.** Relationship between the current document and the linked document. See chapter ["Relation types"](#relation-types) for more information. | -| type | string | [Media type](#media-types) of the referenced entity. | -| title | string | A human readable title to be used in rendered displays of the link. | +| type | string | [Media type](#media-types) of the referenced entity. | +| title | string | A human readable title to be used in rendered displays of the link. | For a full discussion of the situations where relative and absolute links are recommended see the ['Use of links'](../best-practices.md#use-of-links) section of the STAC best practices. @@ -88,8 +77,8 @@ The following types are commonly used as `rel` types in the Link Object of a STA | Type | Description | | ------- | ----------- | | self | STRONGLY RECOMMENDED. *Absolute* URL to the location that the Catalog file can be found online, if available. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | -| root | STRONGLY RECOMMENDED. URL to the root STAC Catalog or [Collection](../collection-spec/README.md). Catalogs should include a link to their root, even if it's the root and points to itself. | -| parent | URL to the parent STAC Catalog or Collection. Non-root Catalogs should include a link to their parent. | +| root | STRONGLY RECOMMENDED. URL to the root STAC entity (Catalog or [Collection](../collection-spec/README.md)). Catalogs should include a link to their root, even if it's the root and points to itself. | +| parent | URL to the parent STAC entity (Catalog or Collection). Non-root Catalogs should include a link to their parent. | | child | URL to a child STAC Catalog or Collection. | | item | URL to a STAC Item. | @@ -117,11 +106,11 @@ A STAC Catalog is a JSON file ([RFC 8259](https://tools.ietf.org/html/rfc8259)), The following table lists the Media Types to use for STAC structures. -| Media Type | Description | -| ---------------------- | ---------------------------------------------------------- | -| `application/geo+json` | A STAC [Item](../item-spec/item-spec.md) | -| `application/json` | A STAC Catalog | -| `application/json` | A STAC [Collection](../collection-spec/collection-spec.md) | +| Media Type | Description | +| ------------------------------ | ------------------------------------------------------------ | +| `application/geo+json` | A STAC [Item](../item-spec/item-spec.md) | +| `application/json` | A STAC [Catalog](#stac-catalog-specification) | +| `application/json` | A STAC [Collection](../collection-spec/collection-spec.md) | ## Extensions diff --git a/collection-spec/collection-spec.md b/collection-spec/collection-spec.md index f4e331c7..280a948e 100644 --- a/collection-spec/collection-spec.md +++ b/collection-spec/collection-spec.md @@ -252,8 +252,8 @@ This is done where there is not a clear official option, or where STAC uses an o | Type | Description | | ------- | ------------------------------------------------------------ | | self | STRONGLY RECOMMENDED. *Absolute* URL to the location that the Collection file can be found online, if available. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | -| root | URL to the root STAC Catalog or Collection. Collections should include a link to their root, even if it's the root and points to itself. | -| parent | URL to the parent STAC Catalog or Collection. Non-root Collections should include a link to their parent. | +| root | URL to the root STAC entity (Catalog or Collection). Collections should include a link to their root, even if it's the root and points to itself. | +| parent | URL to the parent STAC entity (Catalog or Collection). Non-root Collections should include a link to their parent. | | child | URL to a child STAC Catalog or Collection. | | item | URL to a STAC Item. All Items linked from a Collection MUST refer back to its Collection with the [`collection` relation type](../item-spec/item-spec.md#relation-types). | | license | The license URL(s) for the Collection SHOULD be specified if the `license` field is set to `proprietary` or `various`. If there is no public license URL available, it is RECOMMENDED to put the license text in a separate file and link to this file. | diff --git a/overview.md b/overview.md index b498f7b5..68bef150 100644 --- a/overview.md +++ b/overview.md @@ -53,10 +53,11 @@ A Catalog is a very simple construct - it just provides links to Items or to oth The closest analog is a folder in a file structure, it is the container for Items, but it can also hold other containers (folders / catalogs). -The Collection specification shares some fields with the catalog spec but has a number of additional fields: +The Collection entity shares most fields with the Catalog entity but has a number of additional fields: license, extent (spatial and temporal), providers, keywords and summaries. Every Item in a Collection links back to their Collection, so clients can easily find fields like the license. Thus every Item implicitly -shares the fields described in their parent Collection. +shares the fields described in their parent Collection. Collection entities can be used just like Catalog +entities to provide structure, as they provide all the same options for linking and organizing. But what *should* go in a Collection, versus just in a Catalog? A Collection will generally consist of a set of assets that are defined with the same properties and share higher level metadata. In the @@ -78,10 +79,10 @@ provide multiple grouping paths, serving as a sort of faceted search. The second case is used when one wants to represent diverse data in a single place. If an organization has an internal catalog with Landsat 8, Sentinel 2, NAIP data and several commercial imagery providers -then they'd have a root catalog that would link to a number of different Collections. +then they'd have a root Catalog that would link to a number of different Collections. So in conclusion it's best to use Collections for what you want user to find as starting point, and then -catalogs are just for structuring and grouping the data. Future work includes a mechanism to actually +Catalogs are just for structuring and grouping the data. Future work includes a mechanism to actually search Collection-level data, hopefully in concert with other specifications. ## Catalog Overview @@ -91,14 +92,14 @@ points to [STAC Items](item-spec/README.md), or to other STAC catalogs. It provi linking structure that can be used recursively so that many Items can be included in a single Catalog, organized however the implementor desires. -STAC makes no formal distinction between a "root" catalog and the "child" catalogs. A root catalog -is simply the top-most catalog -- it has no parent. A nested catalog structure is useful (and +STAC makes no formal distinction between a "root" Catalog and the "child" Catalogs. A root Catalog +is simply the top-most Catalog -- it has no parent. A nested catalog structure is useful (and recommended) for breaking up massive numbers of catalog Items into logical groupings. For example, it might make sense to organize a catalog by date (year, month, day), or geography (continent, country, state/prov). See the [Catalog Layout](best-practices.md#catalog-layout) best practices section for more. -A simple Catalog structure might look like this: +A simple STAC structure might look like this: - catalog (root) - catalog @@ -137,6 +138,9 @@ root catalog might be a sub-catalog of someone else's structure. The goal is for information and links they want to, while also encouraging a natural web of information to arise as Catalogs and Items are linked to across the web. +*NOTE: The above examples all say Catalog, but those can all be Collections as well, as it has all the fields necessary to +serve as a Catalog* + ### Static and Dynamic Catalogs The Catalog specification is designed so it can be implemented as easily as possibly. This can be as simple as @@ -164,8 +168,8 @@ each Item and Catalog, as well as ways to achieve that. ## Collection Overview -A STAC Collection extends the core fields of the Catalog construct to provide additional metadata to describe the set of Items it -contains. The required fields are fairly +A STAC Collection includes the core fields of the Catalog entity and also provides additional metadata to describe +the set of Items it contains. The required fields are fairly minimal - it includes the 4 required Catalog fields (id, description, stac_version and links), and adds license and extents. But there are a number of other common fields defined in the spec, and more common fields are also defined in [STAC extensions](extensions/). These serve as basic metadata, and ideally Collections also link to From 2be5e0408834ad4e65a2fca0213c06ec56afb7da Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Mon, 19 Apr 2021 21:13:25 -0700 Subject: [PATCH 02/10] got rid of old changes that snuck in --- catalog-spec/catalog-spec.md | 59 +++++++++++++++++++++--------------- 1 file changed, 35 insertions(+), 24 deletions(-) diff --git a/catalog-spec/catalog-spec.md b/catalog-spec/catalog-spec.md index b3fa521c..573e848a 100644 --- a/catalog-spec/catalog-spec.md +++ b/catalog-spec/catalog-spec.md @@ -1,7 +1,8 @@ -# STAC Catalog Specification +# STAC Catalog Specification - [Catalog fields](#catalog-fields) - [Additional Field Information](#additional-field-information) + - [stac_version](#stac_version) - [stac_extensions](#stac_extensions) - [Link Object](#link-object) - [Relation types](#relation-types) @@ -10,15 +11,18 @@ - [STAC Media Types](#stac-media-types) - [Extensions](#extensions) -This document explains the structure and content of a STAC Catalog. A STAC Catalog is a -[Collection](../collection-spec/collection-spec.md) of [STAC Items](../item-spec/item-spec.md). These Items can be -linked to directly from a Catalog, or the Catalog can link to other Catalogs (often called sub-Catalogs) that contain -links to Items. The division of sub-catalogs is up to the implementor, but is generally done to aid the ease of -online browsing by people. +This document explains the structure and content of a STAC **Catalog** object. A STAC Catalog object +represents a logical group of other Catalog, +[Collection](../collection-spec/collection-spec.md), and [Item](../item-spec/item-spec.md) objects. +These Items can be linked to directly from a Catalog, or the Catalog can link to other Catalogs (often called +sub-catalogs) that contain links to Collections and Items. The division of sub-catalogs is up to the implementor, +but is generally done to aid the ease of online browsing by people. -Catalogs are not intended to be queried. Their purpose is discovery: to be browsed by people and crawled -by machines to build a search index. A Catalog can be represented in JSON format. Any JSON object -that contains all the required fields is a valid STAC Catalog. +A Catalog object will typically be the entry point into a STAC catalog. Their +purpose is discovery: to be browsed by people or be crawled +by clients to build a searchable index. + +Any JSON object that contains all the required fields is a valid STAC Catalog object. - [Examples](../examples/) - See an example [catalog.json](../examples/catalog.json). The [collection.json](../examples/collection.json) is also a valid @@ -39,8 +43,8 @@ also a valid STAC Catalog. | Element | Type | Description | | --------------- | ------------- | ------------------------------------------------------------ | -| stac_version | string | **REQUIRED.** The STAC version the Catalog implements. STAC versions can be mixed, but please keep the [recommended best practices](../best-practices.md#mixing-stac-versions) in mind. | | type | string | **REQUIRED.** Set to `Catalog` if this Catalog only implements the Catalog spec. | +| stac_version | string | **REQUIRED.** The STAC version the Catalog implements. | | stac_extensions | \[string] | A list of extension identifiers the Catalog implements. | | id | string | **REQUIRED.** Identifier for the Catalog. | | title | string | A short descriptive one-line title for the Catalog. | @@ -50,22 +54,29 @@ also a valid STAC Catalog. ### Additional Field Information +#### stac_version + +In general, STAC versions can be mixed, but please keep the [recommended best practices](../best-practices.md#mixing-stac-versions) in mind. + #### stac_extensions -A list of extensions the Catalog implements. This does NOT declare the extensions of children or Items. The list contains URLs to the JSON Schema files it can be validated against. For [core extensions](../extensions/README.md#core-stac-extensions), a "shortcut" can be used. This means you can specify the folder name of the extension, for example `view` for the View extension. If the versions of the extension and the Catalog diverge, you can specify the URL of the JSON schema file. -This list must only contain extensions that extend the Catalog itself, see the the 'Scope' column in the list of extensions. +A list of extensions the Catalog implements. +The list consists of URLs to JSON Schema files that can be used for validation. +This list must only contain extensions that extend the Catalog specification itself, +see the 'Scope' for each of the extensions. +This must **not** declare the extensions that are only implemented in child Collection objects or child Item objects. ### Link Object This object describes a relationship with another entity. Data providers are advised to be liberal with links. -| Field Name | Type | Description | -| ---------- | ------ | ----------------------------------------------------------------------------------------------------------------------------------- | -| href | string | **REQUIRED.** The actual link in the format of an URL. Relative and absolute links are both allowed. | +| Field Name | Type | Description | +| ---------- | ------ | ----------- | +| href | string | **REQUIRED.** The actual link in the format of an URL. Relative and absolute links are both allowed. | | rel | string | **REQUIRED.** Relationship between the current document and the linked document. See chapter ["Relation types"](#relation-types) for more information. | -| type | string | [Media type](#media-types) of the referenced entity. | -| title | string | A human readable title to be used in rendered displays of the link. | +| type | string | [Media type](#media-types) of the referenced entity. | +| title | string | A human readable title to be used in rendered displays of the link. | For a full discussion of the situations where relative and absolute links are recommended see the ['Use of links'](../best-practices.md#use-of-links) section of the STAC best practices. @@ -77,8 +88,8 @@ The following types are commonly used as `rel` types in the Link Object of a STA | Type | Description | | ------- | ----------- | | self | STRONGLY RECOMMENDED. *Absolute* URL to the location that the Catalog file can be found online, if available. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | -| root | STRONGLY RECOMMENDED. URL to the root STAC entity (Catalog or [Collection](../collection-spec/README.md)). Catalogs should include a link to their root, even if it's the root and points to itself. | -| parent | URL to the parent STAC entity (Catalog or Collection). Non-root Catalogs should include a link to their parent. | +| root | STRONGLY RECOMMENDED. URL to the root STAC Catalog or [Collection](../collection-spec/README.md). Catalogs should include a link to their root, even if it's the root and points to itself. | +| parent | URL to the parent STAC Catalog or Collection. Non-root Catalogs should include a link to their parent. | | child | URL to a child STAC Catalog or Collection. | | item | URL to a STAC Item. | @@ -106,11 +117,11 @@ A STAC Catalog is a JSON file ([RFC 8259](https://tools.ietf.org/html/rfc8259)), The following table lists the Media Types to use for STAC structures. -| Media Type | Description | -| ------------------------------ | ------------------------------------------------------------ | -| `application/geo+json` | A STAC [Item](../item-spec/item-spec.md) | -| `application/json` | A STAC [Catalog](#stac-catalog-specification) | -| `application/json` | A STAC [Collection](../collection-spec/collection-spec.md) | +| Media Type | Description | +| ---------------------- | ---------------------------------------------------------- | +| `application/geo+json` | A STAC [Item](../item-spec/item-spec.md) | +| `application/json` | A STAC Catalog | +| `application/json` | A STAC [Collection](../collection-spec/collection-spec.md) | ## Extensions From 5a994b7dac681b82529c152d5182f6f2099d29de Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 20 Apr 2021 08:35:29 -0700 Subject: [PATCH 03/10] Update overview.md Co-authored-by: Matthias Mohr --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 68bef150..4e9d84b5 100644 --- a/overview.md +++ b/overview.md @@ -93,7 +93,7 @@ linking structure that can be used recursively so that many Items can be include a single Catalog, organized however the implementor desires. STAC makes no formal distinction between a "root" Catalog and the "child" Catalogs. A root Catalog -is simply the top-most Catalog -- it has no parent. A nested catalog structure is useful (and +is simply the top-most Catalog or Collection -- it has no parent. A nested catalog structure is useful (and recommended) for breaking up massive numbers of catalog Items into logical groupings. For example, it might make sense to organize a catalog by date (year, month, day), or geography (continent, country, state/prov). See the [Catalog Layout](best-practices.md#catalog-layout) best practices From 60bf0bcc7e79b4d171cafea23eae864646ec8601 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 20 Apr 2021 08:36:50 -0700 Subject: [PATCH 04/10] more updates --- best-practices.md | 18 +++++++++++------- item-spec/item-spec.md | 2 +- 2 files changed, 12 insertions(+), 8 deletions(-) diff --git a/best-practices.md b/best-practices.md index 77e01e6b..164a8cef 100644 --- a/best-practices.md +++ b/best-practices.md @@ -71,8 +71,8 @@ So to enable all the great web tools (like [stacindex.org](http://stacindex.org) [Google Cloud Storage](https://cloud.google.com/storage/docs/cross-origin), or [Apache Server](https://enable-cors.org/server_apache.html). Many more are listed on [enable-cors.org](https://enable-cors.org/server.html). We recommend enabling CORS for all requests ('\*'), so that diverse online tools can access your data. If you aren't sure if your server has CORS enabled you can use -[test-cors.org](https://www.test-cors.org/). Enter the URL of your STAC root [Catalog](catalog-spec/catalog-spec.md) JSON -and make sure it gets a response. +[test-cors.org](https://www.test-cors.org/). Enter the URL of your STAC root [Catalog](catalog-spec/catalog-spec.md) or +[Collection](collection-spec/collection-spec.md) JSON and make sure it gets a response. ### STAC on the Web @@ -81,7 +81,7 @@ surprised that there is nothing about HTML in the entire specification. This is should be on web pages without ending up with very bad looking pages. But the importance of having web-accessible versions of every STAC Item is paramount. -The main recommendation is to have an HTML page for every single STAC Item and Catalog. They should be visually pleasing, +The main recommendation is to have an HTML page for every single STAC Item, Catalog and Collection. They should be visually pleasing, crawlable by search engines and ideally interactive. The current best practice is to use a tool in the STAC ecosystem called [STAC Browser](https://github.com/radiantearth/stac-browser/). It can crawl most any valid STAC implementation and generate unique web pages for each Item and Catalog (or Collection). While it has a default look and feel, the design can easily be @@ -393,6 +393,10 @@ file that just has the bands needed for display ## Catalog & Collection Practices +*Note: This section uses the term 'Catalog' (with an uppercase C) to refer to the JSON entity specified in the +[Catalog spec](catalog-spec/catalog-spec.md), and 'catalog' (with a lowercase c) to refer to any full STAC implementation, +which can be any mix of Catalogs Collections and Items.* + ### Static and Dynamic Catalogs As mentioned in the main [overview](overview.md), there are two main types of catalogs - static @@ -446,7 +450,7 @@ providers, and users could browse down to both. The leaf Items should just be li ### Catalog Layout -Creating a catalog involves a number of decisions as to what folder structure to use to represent sub-catalogs, items +Creating a catalog involves a number of decisions as to what folder structure to use to represent sub-catalogs, Items and assets, and how to name them. The specification leaves this totally open, and you can link things as you want. But it is recommended to be thoughtful about the organization of sub-catalogs, putting them into a structure that a person might reasonably browse (since they likely will with [STAC on the Web](#stac-on-the-web) recommendations). For example @@ -610,7 +614,7 @@ implement it. This is a self-contained catalog as described above, except it includes an absolute `self` link at the root catalog, to identify its online location. This is designed so that a self-contained catalog (of either type, with its assets or just metadata) can be 'published' online -by just adding one field (the self link) to its root catalog. All the other links should remain the same. The resulting catalog +by just adding one field (the self link) to its root (Catalog or Collection). All the other links should remain the same. The resulting catalog is no longer compliant with the self-contained catalog recommendations, but instead transforms into a 'relative published catalog'. With this, a client may resolve Item and sub-catalog self links by traversing parent and root links, but requires reading multiple sources to achieve this. @@ -632,8 +636,8 @@ a number of the common official relations that are used in production STAC imple | alternate | It is recommended that STAC Items are also available as HTML, and should use this rel with `"type" : "text/html"` to tell clients where they can get a version of the Item or Collection to view in a browser. See [STAC on the Web in Best Practices](#stac-on-the-web) for more information. | | canonical | The URL of the [canonical](https://en.wikipedia.org/wiki/Canonical_link_element) version of the Item or Collection. API responses and copies of catalogs should use this to inform users that they are direct copy of another STAC Item, using the canonical rel to refer back to the primary location. | | via | The URL of the source metadata that this STAC Item or Collection is created from. Used similarly to canonical, but refers back to a non-STAC record (Landsat MTL, Sentinel tileInfo.json, etc) | -| prev | Indicates that the link's context is a part of a series, and that the previous in the series is the link target. Typically used in STAC by API's, to return smaller groups of Items or Catalogs. | -| next | Indicates that the link's context is a part of a series, and that the next in the series is the link target. Typically used in STAC by API's, to return smaller groups of Items or Catalogs. | +| prev | Indicates that the link's context is a part of a series, and that the previous in the series is the link target. Typically used in STAC by API's, to return smaller groups of Items or Catalogs/Collections. | +| next | Indicates that the link's context is a part of a series, and that the next in the series is the link target. Typically used in STAC by API's, to return smaller groups of Items or Catalogs/Collections. | ### Versioning for Catalogs diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index 5f609c52..2ac20a66 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -84,7 +84,7 @@ It is important that an Item identifier is unique within a Collection, and that [Collection identifier](../collection-spec/collection-spec.md#id) in turn is unique globally. Then the two can be combined to give a globally unique identifier. Items are *[strongly recommended](#collections)* to have Collections, and not having one makes it more difficult to be used in the wider STAC ecosystem. -If an Item does not have a Collection, then the Item identifier should be unique within its root Catalog. +If an Item does not have a Collection, then the Item identifier should be unique within its root Catalog or root Collection. As most geospatial assets are already uniquely defined by some identification scheme from the data provider it is recommended to simply use that ID. From 6b24f7c9dc264178ee58da0b03777fa8d02bf296 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Tue, 20 Apr 2021 08:49:27 -0700 Subject: [PATCH 05/10] put note up top --- overview.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/overview.md b/overview.md index 4e9d84b5..306daf5f 100644 --- a/overview.md +++ b/overview.md @@ -87,6 +87,9 @@ search Collection-level data, hopefully in concert with other specifications. ## Catalog Overview +*NOTE: The below examples all say Catalog, but those can all be Collections as well, as it has all the fields necessary to +serve as a Catalog* + There are two required element types of a Catalog: Catalog and Item. A STAC Catalog points to [STAC Items](item-spec/README.md), or to other STAC catalogs. It provides a simple linking structure that can be used recursively so that many Items can be included in @@ -138,9 +141,6 @@ root catalog might be a sub-catalog of someone else's structure. The goal is for information and links they want to, while also encouraging a natural web of information to arise as Catalogs and Items are linked to across the web. -*NOTE: The above examples all say Catalog, but those can all be Collections as well, as it has all the fields necessary to -serve as a Catalog* - ### Static and Dynamic Catalogs The Catalog specification is designed so it can be implemented as easily as possibly. This can be as simple as From 233b757cb59948f1d5993b6299b20cd73577054b Mon Sep 17 00:00:00 2001 From: Brad Hards Date: Fri, 23 Apr 2021 17:40:14 +1000 Subject: [PATCH 06/10] Minor change to Collection usage wording --- overview.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/overview.md b/overview.md index 3d752aae..8c35e597 100644 --- a/overview.md +++ b/overview.md @@ -80,7 +80,7 @@ The second case is used when one wants to represent diverse data in a single pla has an internal catalog with Landsat 8, Sentinel 2, NAIP data and several commercial imagery providers then they'd have a root catalog that would link to a number of different Collections. -So in conclusion it's best to use Collections for what you want user to find as starting point, and then +So in conclusion it's best to use Collections for what you want a user to find as a starting point, and then catalogs are just for structuring and grouping the data. Future work includes a mechanism to actually search Collection-level data, hopefully in concert with other specifications. From 54d7be4957dd6ec92c7c63a14c359fc9426d1576 Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Fri, 23 Apr 2021 07:27:25 -0700 Subject: [PATCH 07/10] updates from review comments --- best-practices.md | 10 +++++----- catalog-spec/catalog-spec.md | 6 +++--- collection-spec/collection-spec.md | 2 +- 3 files changed, 9 insertions(+), 9 deletions(-) diff --git a/best-practices.md b/best-practices.md index 164a8cef..c04cf116 100644 --- a/best-practices.md +++ b/best-practices.md @@ -467,11 +467,11 @@ if you follow these recommendations. 1. Root documents (Catalogs / Collections) should be at the root of a directory tree containing the static catalog. 2. Catalogs should be named `catalog.json` and Collections should be named `collection.json`. 3. Items should be named `.json`. -4. Sub-Catalogs should be stored in subdirectories of their parent +4. Sub-Catalogs or sub-Collections should be stored in subdirectories of their parent (and only 1 subdirectory deeper than a document's parent, e.g. `.../sample/sub1/catalog.json`). -5. Items should be stored in subdirectories of their parent Catalog. +5. Items should be stored in subdirectories of their parent Catalog or Collection. This means that each Item and its assets are contained in a unique subdirectory. -6. Limit the number of Items in a Catalog or sub-Catalog, grouping / partitioning as relevant to the dataset. +6. Limit the number of Items in a Catalog or Collection, grouping / partitioning as relevant to the dataset. 7. Use structural elements (Catalog and Collection) consistently across each 'level' of your hierarchy. For example, if levels 2 and 4 of the hierarchy only contain Collections, don't add a Catalog at levels 2 and 4. @@ -487,7 +487,7 @@ different sub-catalog organization structures. For example one catalog could div by providers, and users could browse down to both. The leaf Items should just be linked to in a single canonical location (or at least use a rel link that indicates the location of the canonical one). It is recommended that dynamic catalogs provide multiple 'views' to allow users to navigate in a way that makes sense to them, providing multiple 'sub-catalogs' -from the root Catalog that enable different paths to browse (country/state, date/time, constellation/satellite, etc). But the +from the root that enable different paths to browse (country/state, date/time, constellation/satellite, etc). But the canonical 'rel' link should be used to designate the primary location of the Item to search engine crawlers. #### Mixing STAC Versions @@ -612,7 +612,7 @@ implement it. #### Relative Published Catalog This is a self-contained catalog as described above, except it includes an absolute `self` link at -the root catalog, to identify its online location. This is designed so that a self-contained catalog (of either type, with its +the root to identify its online location. This is designed so that a self-contained catalog (of either type, with its assets or just metadata) can be 'published' online by just adding one field (the self link) to its root (Catalog or Collection). All the other links should remain the same. The resulting catalog is no longer compliant with the self-contained catalog recommendations, but instead transforms into a 'relative published catalog'. diff --git a/catalog-spec/catalog-spec.md b/catalog-spec/catalog-spec.md index 573e848a..64d2409e 100644 --- a/catalog-spec/catalog-spec.md +++ b/catalog-spec/catalog-spec.md @@ -36,8 +36,8 @@ and fields to be compliant. This Catalog specification primarily defines a structure for information to be discoverable. Any use that is publishing a set of related spatiotemporal assets is strongly recommended to also use the STAC Collection specification to provide additional information about the set of Items -contained in a Catalog, in order to give contextual information to aid in discovery. Every STAC Collection is -also a valid STAC Catalog. +contained in a Catalog, in order to give contextual information to aid in discovery. Every STAC Collection contains all +the same fields as Catalog, though returns 'Collection' instead of 'Catalog' for the type field. ## Catalog fields @@ -93,7 +93,7 @@ The following types are commonly used as `rel` types in the Link Object of a STA | child | URL to a child STAC Catalog or Collection. | | item | URL to a STAC Item. | -**Note:** A link to at least one `item` or `child` Catalog is **REQUIRED**. +**Note:** A link to at least one `item` or `child` (Catalog or Collection) is **REQUIRED**. There are additional `rel` types in the [Using Relation Types](../best-practices.md#using-relation-types) best practice, but as they are more typically used in Collections, as Catalogs tend to just be used to structure STAC organization, so tend to just use diff --git a/collection-spec/collection-spec.md b/collection-spec/collection-spec.md index 280a948e..c1614b99 100644 --- a/collection-spec/collection-spec.md +++ b/collection-spec/collection-spec.md @@ -254,7 +254,7 @@ This is done where there is not a clear official option, or where STAC uses an o | self | STRONGLY RECOMMENDED. *Absolute* URL to the location that the Collection file can be found online, if available. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | | root | URL to the root STAC entity (Catalog or Collection). Collections should include a link to their root, even if it's the root and points to itself. | | parent | URL to the parent STAC entity (Catalog or Collection). Non-root Collections should include a link to their parent. | -| child | URL to a child STAC Catalog or Collection. | +| child | URL to a child STAC entity (Catalog or Collection). | | item | URL to a STAC Item. All Items linked from a Collection MUST refer back to its Collection with the [`collection` relation type](../item-spec/item-spec.md#relation-types). | | license | The license URL(s) for the Collection SHOULD be specified if the `license` field is set to `proprietary` or `various`. If there is no public license URL available, it is RECOMMENDED to put the license text in a separate file and link to this file. | | derived_from | URL to a STAC Collection that was used as input data in the creation of this Collection. See the note in [STAC Item](../item-spec/item-spec.md#derived_from) for more info. | From e257928222c33013e216f4850b0d2eb9f75e1578 Mon Sep 17 00:00:00 2001 From: Matthias Mohr Date: Fri, 23 Apr 2021 16:36:21 +0200 Subject: [PATCH 08/10] Couple of changes for better alignment between the specs + list formatting --- best-practices.md | 4 ++-- catalog-spec/catalog-spec.md | 9 +++++---- item-spec/item-spec.md | 4 ++-- 3 files changed, 9 insertions(+), 8 deletions(-) diff --git a/best-practices.md b/best-practices.md index c04cf116..9a33eade 100644 --- a/best-practices.md +++ b/best-practices.md @@ -470,11 +470,11 @@ if you follow these recommendations. 4. Sub-Catalogs or sub-Collections should be stored in subdirectories of their parent (and only 1 subdirectory deeper than a document's parent, e.g. `.../sample/sub1/catalog.json`). 5. Items should be stored in subdirectories of their parent Catalog or Collection. -This means that each Item and its assets are contained in a unique subdirectory. + This means that each Item and its assets are contained in a unique subdirectory. 6. Limit the number of Items in a Catalog or Collection, grouping / partitioning as relevant to the dataset. 7. Use structural elements (Catalog and Collection) consistently across each 'level' of your hierarchy. For example, if levels 2 and 4 of the hierarchy only contain Collections, -don't add a Catalog at levels 2 and 4. + don't add a Catalog at levels 2 and 4. #### Dynamic Catalog Layout diff --git a/catalog-spec/catalog-spec.md b/catalog-spec/catalog-spec.md index 64d2409e..325ea58f 100644 --- a/catalog-spec/catalog-spec.md +++ b/catalog-spec/catalog-spec.md @@ -36,8 +36,9 @@ and fields to be compliant. This Catalog specification primarily defines a structure for information to be discoverable. Any use that is publishing a set of related spatiotemporal assets is strongly recommended to also use the STAC Collection specification to provide additional information about the set of Items -contained in a Catalog, in order to give contextual information to aid in discovery. Every STAC Collection contains all -the same fields as Catalog, though returns 'Collection' instead of 'Catalog' for the type field. +contained in a Catalog, in order to give contextual information to aid in discovery. +STAC Collections all have the same fields as STAC Catalogs, but with different allowed +values for `type` and `stac_extensions`. ## Catalog fields @@ -89,8 +90,8 @@ The following types are commonly used as `rel` types in the Link Object of a STA | ------- | ----------- | | self | STRONGLY RECOMMENDED. *Absolute* URL to the location that the Catalog file can be found online, if available. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | | root | STRONGLY RECOMMENDED. URL to the root STAC Catalog or [Collection](../collection-spec/README.md). Catalogs should include a link to their root, even if it's the root and points to itself. | -| parent | URL to the parent STAC Catalog or Collection. Non-root Catalogs should include a link to their parent. | -| child | URL to a child STAC Catalog or Collection. | +| parent | URL to the parent STAC entity (Catalog or Collection). Non-root Catalogs should include a link to their parent. | +| child | URL to a child STAC entity (Catalog or Collection). | | item | URL to a STAC Item. | **Note:** A link to at least one `item` or `child` (Catalog or Collection) is **REQUIRED**. diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index 2ac20a66..641fd3da 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -192,8 +192,8 @@ This happens where there is not a clear official option, or where STAC uses an o | Type | Description | | ------------ | ------------------------------------------------------------ | | self | STRONGLY RECOMMENDED. *Absolute* URL to the Item if it is available at a public URL. This is particularly useful when in a download package that includes metadata, so that the downstream user can know where the data has come from. | -| root | URL to the root STAC Catalog or Collection. | -| parent | URL to the parent STAC Catalog or Collection. | +| root | URL to the root STAC entity (Catalog or Collection). | +| parent | URL to the parent STAC entity (Catalog or Collection). | | collection | STRONGLY RECOMMENDED. URL to a Collection. *Absolute* URLs should be used whenever possible. The referenced Collection is STRONGLY RECOMMENDED to implement the same STAC version as the Item. A link with this `rel` type is *required* if the `collection` field in properties is present. | | derived_from | URL to a STAC Item that was used as input data in the creation of this Item. | From 1f3b72b83e8b1135b01ccdc9526354a44a2cb72b Mon Sep 17 00:00:00 2001 From: Matthias Mohr Date: Fri, 23 Apr 2021 18:17:31 +0200 Subject: [PATCH 09/10] All timestamps in UTC #1095 --- CHANGELOG.md | 1 + collection-spec/collection-spec.md | 2 +- collection-spec/json-schema/collection.json | 3 ++- item-spec/common-metadata.md | 4 ++-- item-spec/json-schema/datetime.json | 15 ++++++++++----- 5 files changed, 16 insertions(+), 9 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 2e5a4424..68ce46e1 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -12,6 +12,7 @@ and this project adheres to [Semantic Versioning](http://semver.org/spec/v2.0.0. - The first extent in a Collection is always the overall extent, followed by more specific extents. ([#1064](https://github.com/radiantearth/stac-spec/issues/1064), [opengeospatial/ogcapi-features#520](https://github.com/opengeospatial/ogcapi-features/pull/520)) - Updated examples for automatic collection creation from code and validation ([#1080](https://github.com/radiantearth/stac-spec/pull/1080) - Clarified that stac_extensions should also list extensions that are used in Collection summaries. ([#1077](https://github.com/radiantearth/stac-spec/issues/1077)) +- `changed`, `created` (common metadata) and temporal extents (collections): Timestamps must be always in UTC ([#1095](https://github.com/radiantearth/stac-spec/issues/1095)) ## [v1.0.0-rc.2] - 2021-03-30 diff --git a/collection-spec/collection-spec.md b/collection-spec/collection-spec.md index f4e331c7..c76be328 100644 --- a/collection-spec/collection-spec.md +++ b/collection-spec/collection-spec.md @@ -197,7 +197,7 @@ Clients only interested in the overall extent will only need to access the first It is recommended to only use multiple temporal extents if a union of them would then include a large uncovered time span (e.g. only having data for the years 2000, 2010 and 2020). -Each inner array consists of exactly two dates and times. +Each inner array consists of exactly two dates and times, each in UTC. Each date and time MUST be formatted according to [RFC 3339, section 5.6](https://tools.ietf.org/html/rfc3339#section-5.6). The temporal reference system is the Gregorian calendar. diff --git a/collection-spec/json-schema/collection.json b/collection-spec/json-schema/collection.json index 422385f2..19bb9611 100644 --- a/collection-spec/json-schema/collection.json +++ b/collection-spec/json-schema/collection.json @@ -153,7 +153,8 @@ "string", "null" ], - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" } } } diff --git a/item-spec/common-metadata.md b/item-spec/common-metadata.md index 07df2234..d5326d76 100644 --- a/item-spec/common-metadata.md +++ b/item-spec/common-metadata.md @@ -43,8 +43,8 @@ Fields to provide additional temporal information such as ranges with a start an | Field Name | Type | Description | | ---------- | ------------ | ----------- | | datetime | string\|null | See the [Item Spec Fields](item-spec.md#properties-object) for more information. | -| created | string | Creation date and time of the corresponding data (see below). | -| updated | string | Date and time the corresponding data (see below) was updated last. | +| created | string | Creation date and time of the corresponding data (see below), in UTC. | +| updated | string | Date and time the corresponding data (see below) was updated last, in UTC. | All timestamps MUST be formatted according to [RFC 3339, section 5.6](https://tools.ietf.org/html/rfc3339#section-5.6). diff --git a/item-spec/json-schema/datetime.json b/item-spec/json-schema/datetime.json index 3a1480db..fa540a32 100644 --- a/item-spec/json-schema/datetime.json +++ b/item-spec/json-schema/datetime.json @@ -20,29 +20,34 @@ "title": "Date and Time", "description": "The searchable date/time of the assets, in UTC (Formatted in RFC 3339) ", "type": ["string", "null"], - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" }, "start_datetime": { "title": "Start Date and Time", "description": "The searchable start date/time of the assets, in UTC (Formatted in RFC 3339) ", "type": "string", - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" }, "end_datetime": { "title": "End Date and Time", "description": "The searchable end date/time of the assets, in UTC (Formatted in RFC 3339) ", "type": "string", - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" }, "created": { "title": "Creation Time", "type": "string", - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" }, "updated": { "title": "Last Update Time", "type": "string", - "format": "date-time" + "format": "date-time", + "pattern": "(\\+00:00|Z)$" } } } \ No newline at end of file From 14fec945cbd424f7323379ba36a699fe70493d8d Mon Sep 17 00:00:00 2001 From: Chris Holmes Date: Fri, 23 Apr 2021 09:36:24 -0700 Subject: [PATCH 10/10] More emphasis on UTC requirement --- item-spec/item-spec.md | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-) diff --git a/item-spec/item-spec.md b/item-spec/item-spec.md index 5f609c52..0518dd60 100644 --- a/item-spec/item-spec.md +++ b/item-spec/item-spec.md @@ -127,7 +127,7 @@ resources below. | Field Name | Type | Description | | ---------- | ------------ | ------------------------------------------------------------ | -| datetime | string\|null | **REQUIRED.** The searchable date and time of the assets, in UTC. It is formatted according to [RFC 3339, section 5.6](https://tools.ietf.org/html/rfc3339#section-5.6). `null` is allowed, but requires `start_datetime` and `end_datetime` from [common metadata](common-metadata.md#date-and-time-range) to be set. | +| datetime | string\|null | **REQUIRED.** The searchable date and time of the assets, which must be in UTC. It is formatted according to [RFC 3339, section 5.6](https://tools.ietf.org/html/rfc3339#section-5.6). `null` is allowed, but requires `start_datetime` and `end_datetime` from [common metadata](common-metadata.md#date-and-time-range) to be set. | #### datetime @@ -136,7 +136,8 @@ or representative time in the case of assets that are combined together. Though complex thing to capture, for this purpose keep in mind the STAC spec is primarily searching for data, so use whatever single date and time is most useful for a user to search for. STAC content extensions may further specify the meaning of the main `datetime` field, and many will also add more -datetime fields. +datetime fields. **All times in STAC metadata should be in [Coordinated Universal +Time](https://en.wikipedia.org/wiki/Coordinated_Universal_Time) (UTC).** If there's clearly no meaningful single 'nominal' time, it is allowed to use `null` instead. In this case it is **required** to specify a temporal interval with the fields `start_datetime` and `end_datetime` from [common metadata](common-metadata.md#date-and-time-range). For example, if