Skip to content

Commit

Permalink
Revert "copy edits to fix spacing issues in docs (#6639)"
Browse files Browse the repository at this point in the history
This reverts commit 1932ef8.
  • Loading branch information
alexabird committed Sep 13, 2024
1 parent 186e621 commit 37eb420
Show file tree
Hide file tree
Showing 16 changed files with 93 additions and 105 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -37,22 +37,22 @@ The selected <<{managing-prefix}federation_strategy, federation strategy>> calls
Then, any Post-query or Post-ingest plug-ins are invoked.
Finally, the appropriate response is returned to the calling endpoint.

The Catalog Framework wires all Catalog components together.
The Catalog Framework wires all Catalog components together.

It is responsible for routing Catalog requests and responses to the appropriate target.
It is responsible for routing Catalog requests and responses to the appropriate target. 

<<{integrating-prefix}endpoints,Endpoints>> send Catalog requests to the Catalog Framework.
The Catalog Framework then invokes <<{architecture-prefix}catalog_plugins,Catalog Plugins>>, <<{architecture-prefix}transformers,Transformers>>, and <<{architecture-prefix}resources,Resource Components>> as needed before sending requests to the intended destination, such as one or more <<{managing-prefix}connecting_to_sources,Sources>>.
<<{integrating-prefix}endpoints,Endpoints>> send Catalog requests to the Catalog Framework.
The Catalog Framework then invokes <<{architecture-prefix}catalog_plugins,Catalog Plugins>>, <<{architecture-prefix}transformers,Transformers>>, and <<{architecture-prefix}resources,Resource Components>> as needed before sending requests to the intended destination, such as one or more <<{managing-prefix}connecting_to_sources,Sources>>. 

The Catalog Framework decouples clients from service implementations and provides integration points for Catalog Plugins and convenience methods for Endpoint developers.
The Catalog Framework decouples clients from service implementations and provides integration points for Catalog Plugins and convenience methods for Endpoint developers.

== Catalog API Design

The Catalog is composed of several components and an API that connects them together.
The ((Catalog API)) is central to ${branding}'s architectural qualities of extensibility and flexibility.
The Catalog API consists of Java interfaces that define Catalog functionality and specify interactions between components.
The ((Catalog API)) is central to ${branding}'s architectural qualities of extensibility and flexibility. 
The Catalog API consists of Java interfaces that define Catalog functionality and specify interactions between components. 
These interfaces provide the ability for components to interact without a dependency on a particular underlying implementation, thus allowing the possibility of alternate implementations that can maintain interoperability and share developed components.
As such, new capabilities can be developed independently, in a modular fashion, using the Catalog API interfaces and reused by other ${branding} installations.
As such, new capabilities can be developed independently, in a modular fashion, using the Catalog API interfaces and reused by other ${branding} installations.

=== Ensuring Compatibility

Expand Down Expand Up @@ -100,15 +100,15 @@ These examples are for illustrative purposes only and do not necessarily represe
The Ingest Service Endpoint, the Catalog Framework, and the Catalog Provider are key components of the Reference Implementation.
The Endpoint bundle implements a Web service that allows clients to create, update, and delete metacards.
The Endpoint calls the `CatalogFramework` to execute the operations of its specification.
The `CatalogFramework` routes the request through optional `PreIngest` and `PostIngest` Catalog Plugins, which may modify the ingest request/response before/after the Catalog Provider executes the ingest request and provides the response.
The `CatalogFramework` routes the request through optional `PreIngest` and `PostIngest` Catalog Plugins, which may modify the ingest request/response before/after the Catalog Provider executes the ingest request and provides the response. 
Note that a `CatalogProvider` must be present for any ingest requests to be successfully processed, otherwise a fault is returned.

This process is similar for updating catalog entries, with update requests calling the `update(UpdateRequest)` methods on the Endpoint, `CatalogFramework`, and Catalog Provider.
Similarly, for deletion of catalog entries, the delete requests call the `delete(DeleteRequest)` methods on the `Endpoint`, `CatalogFramework`, and `CatalogProvider`.

==== Error Handling

Any ingest attempts that fail inside the Catalog Framework (whether the failure comes from the Catalog Framework itself, pre-ingest plugin failures, or issues with the Catalog Provider) will be logged to a separate log file for ease of error handling.
Any ingest attempts that fail inside the Catalog Framework (whether the failure comes from the Catalog Framework itself, pre-ingest plugin failures, or issues with the Catalog Provider) will be logged to a separate log file for ease of error handling.
The file is located at `${home_directory}/data/log/ingest_error.log` and will log the Metacards that fail, their ID and Title name, and the stack trace associated with their failure.
By default, successful ingest attempts are not logged.
However, that functionality can be achieved by setting the log level of the `ingestLogger` to DEBUG (note that enabling DEBUG can cause a non-trivial performance hit).
Expand Down Expand Up @@ -158,11 +158,11 @@ log:set

The Query Service Endpoint, the Catalog Framework, and the `CatalogProvider` are key components for processing a query request as well.
The Endpoint bundle contains a Web service that exposes the interface to query for `Metacards`.
The Endpoint calls the `CatalogFramework` to execute the operations of its specification.
The Endpoint calls the `CatalogFramework` to execute the operations of its specification.
The `CatalogFramework` relies on the `CatalogProvider` to execute the actual query.
Optional PreQuery and PostQuery Catalog Plugins may be invoked by the `CatalogFramework` to modify the query request/response prior to the Catalog Provider processing the query request and providing the query response.
If a `CatalogProvider` is not configured and no other remote Sources are configured, a fault will be returned.
It is possible to have only remote Sources configured and no local `CatalogProvider` configured and be able to execute queries to specific remote Sources by specifying the site name(s) in the query request.
Optional PreQuery and PostQuery Catalog Plugins may be invoked by the `CatalogFramework` to modify the query request/response prior to the Catalog Provider processing the query request and providing the query response.
If a `CatalogProvider` is not configured and no other remote Sources are configured, a fault will be returned.
It is possible to have only remote Sources configured and no local `CatalogProvider` configured and be able to execute queries to specific remote Sources by specifying the site name(s) in the query request.

==== Product Caching

Expand All @@ -171,7 +171,7 @@ If caching is enabled, each time a retrieve product request is received, the Cat
If it has, the product is retrieved from the local site and returned to the client, providing a much quicker turnaround because remote product retrieval and network traffic was avoided.
If the requested product is not in the cache, the product is retrieved from the Source (local or remote) and cached locally while returning the product to the client.
The caching to a local file of the product and the streaming of the product to the client are done simultaneously so that the client does not have to wait for the caching to complete before receiving the product.
If errors are detected during the caching, caching of the product will be abandoned, and the product will be returned to the client.
If errors are detected during the caching, caching of the product will be abandoned, and the product will be returned to the client. 

The Catalog Framework attempts to detect any network problems during the product retrieval, e.g., long pauses where no bytes are read implying a network connection was dropped.
(The amount of time defined as a "long pause" is configurable, with the default value being five seconds.)
Expand All @@ -183,4 +183,4 @@ Otherwise, caching is canceled if the user cancels the product download.

==== Product Download Status

As part of the caching of products, the Catalog Framework also posts events to the OSGi notification framework. Information includes when the product download started, whether the download is retrying or failed (after the number of retrieval attempts configured for product caching has been exhausted), and when the download completes. These events are retrieved by the Search UI and presented to the user who initiated the download.
As part of the caching of products, the Catalog Framework also posts events to the OSGi notification framework. Information includes when the product download started, whether the download is retrying or failed (after the number of retrieval attempts configured for product caching has been exhausted), and when the download completes. These events are retrieved by the Search UI and presented to the user who initiated the download.
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
A ((Resource Reader)) retrieves resources associated with metacards via URIs.
Each resource reader must know how to interpret the resource's URI and how to interact with the data store to retrieve the resource.

There can be multiple resource readers in a Catalog instance.
The `Catalog Framework` selects the appropriate resource reader based on the scheme of the resource's URI.
There can be multiple resource readers in a Catalog instance.
The `Catalog Framework` selects the appropriate resource reader based on the scheme of the resource's URI. 

In order to make a resource reader available to the Catalog Framework, it must be exported to the OSGi Service Registry as a `${ddf-branding}.catalog.resource.ResourceReader`.
In order to make a resource reader available to the Catalog Framework, it must be exported to the OSGi Service Registry as a `${ddf-branding}.catalog.resource.ResourceReader`. 
Original file line number Diff line number Diff line change
Expand Up @@ -31,13 +31,11 @@ Configuring the URL Resource Reader to retrieve files requires adding Security M
After adding permission entries, a system restart is required for them to take effect.
====

----
grant codeBase "file:/org.apache.tika.core/catalog-core-urlresourcereader" {
//# Add permissions here
permission java.io.FilePermission "<DIRECTORY_PATH>", "read";
permission java.io.FilePermission "<OTHER_DIRECTORY_PATH>", "read";
}
----

Trailing slashes after <DIRECTORY_PATH> have no effect on the permissions granted. For example, adding a permission for "${/}test${/}path" and "${/}test${/}path${/}" are equivalent. The recursive forms "${/}test${/}path${/}-", and "${/}test${/}path${/}${/}-" are also equivalent.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -92,7 +92,7 @@ Does not support the acquisition of credentials.

|ddf.security.pdp.realm.AuthZRealm
|This realm performs the authorization decision and may or may not delegate out to the external XACML processing engine. It uses the incoming permissions to create a decision.
However, it is possible to extend this realm using the `ddf.security.policy.extension.PolicyExtension interface`.
However, it is possible to extend this realm using the ddf.security.policy.extension.PolicyExtension interface.
This interface allows an integrator to add additional policy information to the PDP that can't be covered via its generic matching policies.
This approach is often easier to configure for those that are not familiar with XACML.

Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ The Subject can also be used to perform permission checks to see if the calling
This class was made ${branding}-specific because the Shiro interface cannot be added to the Query Request property map.

.Implementations of Subject:
[cols="1m,1" options="header"]
[cols="2" options="header"]
|===

|Classname
Expand Down
Loading

0 comments on commit 37eb420

Please sign in to comment.