Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

DDF-5606 Standardized references to 'resources' in documentation #5608

Merged
merged 1 commit into from
Dec 2, 2019
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
Expand Up @@ -83,7 +83,6 @@ If no sort policy is defined for a particular search, the temporal policy will a

The ${branding} is used to catalog resources.
A Resource is a URI-addressable entity that is represented by a Metacard.
Resources may also be known as products or data.
Resources may exist either locally or on a remote data store.

.Examples of Resources
Expand All @@ -101,30 +100,16 @@ Resources may exist either locally or on a remote data store.
* ${branding} REST

The Query Service Endpoint, the Catalog Framework, and the `CatalogProvider` are key
components for processing a retrieve product request.
The Endpoint bundle contains a Web service that exposes the interface to retrieve products, also referred to as Resources.
components for processing a retrieve resource request.
The Endpoint bundle contains a Web service that exposes the interface to retrieve resources.
The Endpoint calls the `CatalogFramework` to execute the operations of its specification.
The `CatalogFramework` relies on the Sources to execute the actual product retrieval.
Optional PreResource and PostResource Catalog Plugins may be invoked by the `CatalogFramework` to modify the product retrieval request/response prior to the Catalog Provider processing the request and providing the response.
It is possible to retrieve products from specific remote Sources by specifying the site name(s) in the request.
The `CatalogFramework` relies on the Sources to execute the actual resource retrieval.
Optional `PreResource` and `PostResource` Catalog Plugins may be invoked by the `CatalogFramework` to modify the resource retrieval request/response prior to the Catalog Provider processing the request and providing the response.
It is possible to retrieve resources from specific remote Sources by specifying the site name(s) in the request.

.Product Caching
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

❓ Should all of the "Product" headers (Product Retrieval, Product Retrieval Services, Product Cacheing, etc.) be changed to "Resource" as well?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The configurations for these in the code use Product, so I left them for now, but it'd be nice to change them up as well

[NOTE]
====
Existing ${branding} clients are able to leverage product caching due to the product cache being implemented in the ${branding}.
Enabling the product cache is an administrator function.

Product Caching is enabled by default.
====

To configure product caching:

. Navigate to the *${admin-console}*.
. Select ${ddf-catalog}.
. Select *Configuration*.
. Select *Resource Download Settings*.

See <<{reference-prefix}ddf.catalog.resource.download.ReliableResourceDownloadManager,Resource Download Settings configurations>> for all possible configurations.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure. Can you explain?

Existing ${branding} clients are able to leverage product caching due to the product cache being implemented in the ${branding}.

.Product Retrieval Request
[ditaa,product_retrieval_request,png]
Expand Down Expand Up @@ -165,11 +150,11 @@ See <<{reference-prefix}ddf.catalog.resource.download.ReliableResourceDownloadMa

===== Notifications and Activities

${branding} can send/receive notifications of "Activities" occuring in the system.
${branding} can send/receive notifications of "Activities" occurring in the system.

====== Notifications

Currently, the notifications provide information about product retrieval only.
Currently, the notifications provide information about resource retrieval only.

====== Activities

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -165,8 +165,8 @@ It is possible to have only remote Sources configured and no local `CatalogProvi

===== Product Retrieval

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

✏️ I noticed Product Caching seems to be different, but I wasn't sure about these (166, 171, 172)

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't change any of the configurations (metatypes, for example) under the scope of this PR and they use "Product"


The Query Service Endpoint, the Catalog Framework, and the `CatalogProvider` are key components for processing a retrieve product request.
The Endpoint bundle contains a Web service that exposes the interface to retrieve products, also referred to as Resources.
The Query Service Endpoint, the Catalog Framework, and the `CatalogProvider` are key components for processing a resource retrieval request.
The Endpoint bundle contains a Web service that exposes the interface to retrieve resources.
The Endpoint calls the `CatalogFramework` to execute the operations of its specification.
The `CatalogFramework` relies on the Sources to execute the actual product retrieval.
Optional `PreResource` and `PostResource` Catalog Plugins may be invoked by the `CatalogFramework` to modify the product retrieval request/response prior to the Catalog Provider processing the request and providing the response. 
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ A `Metacard` is a container for metadata. 
=== Metacards

A metacard is a single instance of metadata in the Catalog (an instance of a metacard type) which
generally contains general information about the product, such as the title of the product, the product's geo-location, the date the product was created and/or modified, the owner or producer, and/or the security classification. 
generally contains general information about the resource, such as the title of the resource, the resource's geo-location, the date the resource was created and/or modified, the owner or producer, and/or the security classification. 

==== Metacard Type

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ The PDP that is being used returns a decision, and the metacard will either be f
How a metacard gets filtered is left up to any number of FilterStrategy implementations that might be installed.
Each FilterStrategy will return a result to the filter plugin that says whether or not it was able to process the metacard, along with the metacard or response itself.
This allows a metacard or entire response to be partially filtered to allow some data to pass back to the requester.
This could also include filtering any products sent back to a requester.
This could also include filtering any resources sent back to a requester.

The security attributes populated on the metacard are completely dependent on the type of the metacard.
Each type of metacard must have its own `PolicyPlugin` that reads the metadata being returned and then returns the appropriate attributes.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
:plugintypes: postquery
:summary: Updates the resource size attribute of a metacard.

This post-query plugin updates the resource size attribute of each metacard in the query results if there is a cached file for the product and it has a size greater than zero; otherwise, the resource size is unmodified and the original result is returned.
This post-query plugin updates the resource size attribute of each metacard in the query results if there is a cached file for the resource and it has a size greater than zero; otherwise, the resource size is unmodified and the original result is returned.

Use this post-query plugin as a convenience to return query results with accurate resource sizes for cached products. 

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ The plugin runs each metacard in the `CreateRequest` and `UpdateRequest` against

[NOTE]
====
This plugin can make it seem like ingested products are not successfully ingested if a user does not have permissions to access invalid metacards.
This plugin can make it seem like ingested resources are not successfully ingested if a user does not have permissions to access invalid metacards.
If an ingest did not fail, there are no errors in the ingest log, but the expected results do not show up after a query,
verify either that the ingested data is valid or that the <<_metacard_validity_filter_plugin,Metacard Validity Filter Plugin>> is configured to show warnings and/or errors.
====
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,6 @@
Resource components are used when working with resources

A resource is a URI-addressable entity that is represented by a metacard.
Resources may also be known as *products* or *data*.

Resources may exist either locally or on a remote data store.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,10 +6,10 @@
:order: 03
:summary: Resource Writers.

A resource writer stores a resource and produces a URI that can be used to retrieve the resource at a later time.
A resource writer stores a resource and produces a URI that can be used for retrieval.
The resource URI uniquely locates and identifies the resource.
Resource writers can interact with an underlying data store and store the resource in the proper place.
Each implementation can do this differently, providing flexibility in the data stores used to persist the resources.

Resource Writers should be used within the Content Framework if and when implementing a custom Storage Provider to store the product.
The default Storage Provider that comes with the ${branding} writes the products to the file system.
Resource Writers should be used within the Content Framework if and when implementing a custom Storage Provider to store data.
The default Storage Provider that comes with the ${branding} writes the resources to the file system.
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,9 @@ The `URLResourceReader` will connect to the provided Resource URL and read the

[WARNING]
====
When a resource linked using a file-based URL is in the product cache, the `URLResourceReader`&#8217;s rootResourceDirectories is not checked when downloading the product.
When a resource linked using a file-based URL is in the product cache, the ``URLResourceReader``'s `rootResourceDirectories` is not checked when downloading.
It is downloaded from the product cache which bypasses the `URLResourceReader`.
For example, if path `/my/valid/path` is configured in the `URLResourceReader`&#8217;s rootResourceDirectories and one downloads the product with resource-uri `file:///my/valid/path/product.txt` and then one removes `/my/valid/path` from the `URLResourceReader`&#8217;s `rootResourceDirectories` configuration, the product will still be accessible via the product cache.
For example, if path `/my/valid/path` is configured in the ``URLResourceReader``'s `rootResourceDirectories` and one downloads the product with resource-uri `file:///my/valid/path/product.txt` and then one removes `/my/valid/path` from the ``URLResourceReader``'s `rootResourceDirectories` configuration, the product will still be accessible via the product cache.
====

===== Installing the URL Resource Reader
Expand Down Expand Up @@ -54,7 +54,7 @@ See <<{reference-prefix}ddf.catalog.resource.impl.URLResourceReader,URL Resource
`URLResourceReader` will be used by the Catalog Framework to obtain a resource whose metacard is cataloged in the local data store.
This particular `ResourceReader` will be chosen by the `CatalogFramework` if the requested resource's URL has a protocol of `http`, `https`, or `file`.  

For example, requesting a resource with the following URL will make the Catalog Framework invoke the `URLResourceReader` to retrieve the product.
For example, requesting a resource with the following URL will make the Catalog Framework invoke the `URLResourceReader` for retrieval.

.Example
[source,http]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

The KML Metacard Transformer is responsible for translating a metacard into a KML-formatted document.
The KML will contain an HTML description that will display in the pop-up bubble in Google Earth.
The HTML contains links to the full metadata view as well as the product.
The HTML contains links to the full metadata view as well as the resource.

===== Installing the KML Metacard Transformer

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

The KML Query Response Transformer translates a query response into a KML-formatted document.
The KML will contain an HTML description for each metacard that will display in the pop-up bubble in Google Earth.
The HTML contains links to the full metadata view as well as the product.
The HTML contains links to the full metadata view as well as the resource.

===== Installing the KML Query Response Transformer

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@

The MimeTypeMapper is the entry point in ${branding} for resolving file extensions to mime types, and vice versa.

`MimeTypeMappers` are used by the `ResourceReader` to determine the file extension for a given mime type in aid of retrieving a product.
`MimeTypeMappers` are used by the `ResourceReader` to determine the file extension for a given mime type in aid of retrieving a resource.
`MimeTypeMappers` are also used by the `FileSystemProvider` in the Catalog Framework to read a file from the content file repository.

The `MimeTypeMapper` maintains a list of all of the `MimeTypeResolvers` in ${branding}.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@
:subtype: metacard
:status: published
:link: _resource_metacard_transformer
:summary: Retrieves the resource bytes of a metacard by returning the product associated with the metacard.
:summary: Retrieves the resource bytes of a metacard by returning the resource associated with the metacard.

The Resource Metacard Transformer retrieves a resource associated with a metacard.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ An action is a URL that, when invoked, provides a resource or executes intended
==== Action Component Naming Convention

For each Action, a title and description should be provided to describe what the action does.
The recommended naming convention is to use the verb 'Get' when retrieving a portion of a metacard, such as the metadata or thumbnail, or when downloading a product.
The recommended naming convention is to use the verb 'Get' when retrieving a portion of a metacard, such as the metadata or thumbnail, or when downloading a resource.
The verb 'Export' or the expression 'Export as' is recommended when the metacard is being exported in a different format or presented after going some transformation.

===== Action Component Taxonomy
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -79,7 +79,7 @@ Query requests process  `PolicyPlugin`, `AccessPlugin`, and `PreQueryPlugin` i

====== Resource Methods

Resource methods retrieve products from Sources.
Resource methods retrieve data resources from Sources.

.Example Resource Methods
[source,java,linenums]
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -52,7 +52,7 @@ create, update, and delete. 
|Get available `Source` information.

|Resource
|Retrieve products referenced in Metacards from Sources.
|Retrieve resources referenced in Metacards from Sources.

|Transform
|Convert common Catalog Framework data types to and from other data formats.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,7 @@
:order: 15
:summary: Creating a custom Resource Reader.

A `ResourceReader` is a class that retrieves a resource or product from a native/external source and returns it to ${branding}.
A `ResourceReader` is a class that retrieves a resource from a native/external source and returns it to ${branding}.
A simple example is that of a File `ResourceReader`.
It takes a file from the local file system and passes it back to ${branding}.
New implementations can be created in order to support obtaining Resources from various Resource data stores. 
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -11,7 +11,7 @@ Federation is the ability of the ${branding} to query other data sources, includ
By default, the ${branding} is able to federate using http://www.opensearch.org/Home[OpenSearch] and http://www.opengeospatial.org/standards/cat[CSW] protocols.
The minimum configuration necessary to configure those federations is a query address.

Federation enables constructing dynamic networks of data sources that can be queried individually or aggregated into specific configuration to enable a wider range of accessibility for data and data products.
Federation enables constructing dynamic networks of data sources that can be queried individually or aggregated into specific configuration to enable a wider range of accessibility for data and data resources.

Federation provides the capability to extend the ${branding} enterprise to include <<{managing-prefix}connecting_to_sources,Remote Sources>>, which may include other instances of ${branding}. 
The Catalog handles all aspects of federated queries as they are sent to the Catalog Provider and Remote Sources, as they are processed, and as the query results are returned.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

== {title}

Ingest is the process of bringing data products, metadata, or both into the catalog to enable search, sharing, and discovery.
Ingest is the process of bringing data resources, metadata, or both into the catalog to enable search, sharing, and discovery.
Ingested files are <<{integrating-prefix}transformers,transformed>> into a neutral format that can be searched against as well as migrated to other formats and systems.
See <<{managing-prefix}ingesting_data, Ingesting Data>> for the various methods of ingesting data.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

== {title}

In ${branding}, <<_introduction_to_resources,resources>> are the data products, files, reports, or documents of interest to users of the system.
In ${branding}, <<_introduction_to_resources,resources>> are the files, reports, or documents of interest to users of the system.

Metadata is information about those resources, organized into a schema to make search possible.
The ${ddf-catalog} stores this metadata and allows access to it.
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@

== {title}

Filtering is the process of evaluating security markings on data products, comparing them to the users permissions and protecting resources from inappropriate access.
Filtering is the process of evaluating security markings on data resources, comparing them to the users permissions and protecting resources from inappropriate access.

There are two options for processing filtering policies: internally, or through the use of a policy formatted in eXtensible Access Control Markup Language (XACML).
The procedure for setting up a policy differs depending on whether that policy is to be used internally or by the external XACML processing engine.
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,18 @@
:title: Configuring Product Caching
:type: configuration
:status: published
:parent: Configuring Data Management
:order: 026
:summary: Enabling or disabling product caching.

== {title}

To configure product caching:

. Navigate to the *${admin-console}*.
. Select ${ddf-catalog}.
. Select *Configuration*.
. Select *Resource Download Settings*.
. Select / Deselect *Enable Product Caching*.

See <<{reference-prefix}ddf.catalog.resource.download.ReliableResourceDownloadManager,Resource Download Settings configurations>> for all possible configurations.
Original file line number Diff line number Diff line change
Expand Up @@ -46,7 +46,10 @@ If a CDM configuration is deleted, then the corresponding permissions that were
.Content Directory Monitor Permissions
[IMPORTANT]
====
When configuring a Content Directory Monitor, make sure to set permissions on the new directory to allow ${branding} to access it. Setting permissions should be done *before* configuring a CDM. Also, don't forget to add permissions for products outside of the monitored directory. See <<{managing-prefix}configuring_permissions_for_the_content_directory_monitor,Configuring Permissions for the Content Directory Monitor>> for in-depth instructions on configuring permissions.
When configuring a Content Directory Monitor, make sure to set permissions on the new directory to allow ${branding} to access it.
Setting permissions should be done *before* configuring a CDM.
Also, don't forget to add permissions for resources outside of the monitored directory.
See <<{managing-prefix}configuring_permissions_for_the_content_directory_monitor,Configuring Permissions for the Content Directory Monitor>> for in-depth instructions on configuring permissions.
====

[NOTE]
Expand All @@ -57,7 +60,8 @@ If there's a metacard that points to a resource outside of the CDM, then you mus
.Monitoring Directories In Place
[WARNING]
====
If monitoring a directory in place, then the <<{developing-prefix}url_resource_reader, URL Resource Reader>> must be configured prior to configuring the CDM to allow reading from the configured directory. This allows the ${ddf-catalog} to download the products.
If monitoring a directory in place, then the <<{developing-prefix}url_resource_reader, URL Resource Reader>> must be configured prior to configuring the CDM to allow reading from the configured directory.
This allows the ${ddf-catalog} to download the resources.
====

Configure the CDM from the ${admin-console}:
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -41,7 +41,7 @@ a|Set OS File permissions under the `${home_directory}` directory (e.g. `/deploy
** Select the *${ddf-catalog}* application. +
** Select *Resource Download Settings*. +
** Uncheck the `Enable Product Caching` box. +
* Install ${ddf-security} to ensure only the appropriate users are accessing the products. +
* Install ${ddf-security} to ensure only the appropriate users are accessing the resources. +
** Navigate to the ${admin-console} +
** Select *Manage*. +
** Install the ${ddf-security} application, if applicable. +
Expand Down
Loading