diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index a54ee93b..0139ef9f 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -10,19 +10,40 @@ The concept flow through the code is single threaded down a stack. What this tra What happens if new code cannot fit into the current concept flow? A refactoring of the concept flow will need to take place and all existing classes will have to be updated to the new structure. This should be resolved before any code changes take place. -Lastly, the code is divided into packages that very roughly group the functionality. It is not well defined and therefore much of the code is smeared about and any clean up of the smearing is welcome - always. Below is a rough definition for each package: - -gov.nasa.pds.api. - -model.xml - strictly autogenerated code by swagger from model/swagger.yml -registry - base package that contains interfaces to break circular dependencies and spring-boot content. -registry.business - supposed to be the PDS specific stuff but it is much more of a hodgepodge at this time. -registry.configuration - AWS, spring-boot, and swagger configuration information -registry.controllers - where the user request is bottled into a UserContext or should be but it currently has a lot of business loging in it. -registry.exceptions - all locally defined exceptions -registry.lexer - strictly autogenerated code by antlr -registry.opensearch - required opensearch configuration for a connection and a collection of stuff that binds business needs with opensearch -registry.serialize - once the business logic is done, these unit seralize them into the request output format (header Accept value). +The overall architecture/design is the MVC. In this insteance, the controller is the transmutation of uer inputs into the interfaces necessary for the model to fullfil the request. The model represents how the PDS data is stored internally. The view is the serialized output in its appropriate format since that is what the user sees or views. + +Lastly, the code is divided into packages that very roughly group the functionality that reflect the MVC architecture/design. While they may not perfectly enforced currently, the implementation is moving in this direction: + +**gov.nasa.pds.** + +**model** + - strictly autogenerated code by swagger from model/swagger.yml + +**api.registry** + - base package that contains + - interfaces to break circular dependencies and improve encapsulation + - spring-boot content. + +**api.registry.configuration** + - AWS, spring-boot, and swagger configuration information + +**api.registry.controller** + - *control* bidirectional transmutation between registry Java design/implementation and the swagger model (auto generated code) + +**api.registry.exceptions** + - all locally defined exceptions + +**api.registry.lexer** + - strictly autogenerated code by antlr + +**api.registry.model** + - supposed to be the model of how the data is stored and organized but it is much more of a hodgepodge at this time. + +**api.registry.search** + - required opensearch configuration for a connection and a collection of stuff that binds model needs with opensearch + +**api.registry.view** + - once the model has found the needed data, these unit seralize them into the request output format (header Accept value) to complete the request. All of the interfaces in registry are a Context. What does context mean here. It represents the conceptual flow of information. Many users can access the same endpoint with differnt parameters. These context items hold the state for those individual requests just a context is normally used with program threading. They help separate those classes that need state from that which is purely functional help. In the case of the registry, very little state is needed and is easily represented with these interfaces. diff --git a/README.md b/README.md index 3a7d5510..2850ee4f 100644 --- a/README.md +++ b/README.md @@ -19,9 +19,7 @@ Based on `docker` you can easily start all the prerequisites as configured in th git clone https://github.com/NASA-PDS/registry.git -Start the prerequisites: - - docker compose --profile=dev-api up +Start the prerequisites by following the [Quick Start Guide](https://github.com/NASA-PDS/registry/tree/main/docker#-quick-start-guide---with-default-configurations) ## Start the application from a released package @@ -53,8 +51,11 @@ Builds the application: cd service - mvn sprint-boot:run + mvn spring-boot:run + +## View Swagger UI +Go to http://localhost:8080 ## Tests diff --git a/docs/requirements/v1.0.1/REQUIREMENTS.md b/docs/requirements/v1.0.1/REQUIREMENTS.md new file mode 100644 index 00000000..aed7ecaa --- /dev/null +++ b/docs/requirements/v1.0.1/REQUIREMENTS.md @@ -0,0 +1,90 @@ + +Requirements Summary +==================== + +# default + +## As an API user, I want an average query response time of 1 second for q=* queries ([#69](https://github.com/NASA-PDS/registry-api/issues/69)) + + +This requirement is not impacted by the current version +## As an API user, I want to handle long-running queries that take >10 seconds. ([#68](https://github.com/NASA-PDS/registry-api/issues/68)) + + +This requirement is not impacted by the current version +## As an operator, I want to have a wrapper script for starting up the API service ([#67](https://github.com/NASA-PDS/registry-api/issues/67)) + + +This requirement is not impacted by the current version +## As a user, I want to know why my query syntax is invalid ([#66](https://github.com/NASA-PDS/registry-api/issues/66)) + + +This requirement is not impacted by the current version +## As a user, I want specific end points for products which are not collections or bundles ([#65](https://github.com/NASA-PDS/registry-api/issues/65)) + + +This requirement is not impacted by the current version +## As a user, I want the /products end point to work for any class of products ([#64](https://github.com/NASA-PDS/registry-api/issues/64)) + + +This requirement is not impacted by the current version +## As a user, I want the end-point /api to redirect to the API documentation ([#63](https://github.com/NASA-PDS/registry-api/issues/63)) + + +This requirement is not impacted by the current version +## As a user, I want to have singular urls when I should only expect a single element in the response ([#61](https://github.com/NASA-PDS/registry-api/issues/61)) + + +This requirement is not impacted by the current version +## As a user, I want to see the version of the API specification in the URL of the service ([#59](https://github.com/NASA-PDS/registry-api/issues/59)) + + +This requirement is not impacted by the current version +## As a user, I want to see the version of the API specification in the URL of the service ([#8](https://github.com/NASA-PDS/registry-api/issues/8)) + + +This requirement is not impacted by the current version +## As a user, I want the /products end point to work for any class of products ([#11](https://github.com/NASA-PDS/registry-api/issues/11)) + + +This requirement is not impacted by the current version +## As a user, I want specific end points for products which are not collections or bundles ([#12](https://github.com/NASA-PDS/registry-api/issues/12)) + + +This requirement is not impacted by the current version +## As a user, I want to know why my query syntax is invalid ([#13](https://github.com/NASA-PDS/registry-api/issues/13)) + + +This requirement is not impacted by the current version +## As an operator, I want to have a wrapper script for starting up the API service ([#14](https://github.com/NASA-PDS/registry-api/issues/14)) + + +This requirement is not impacted by the current version +## As an API user, I want to handle long-running queries that take >10 seconds. ([#15](https://github.com/NASA-PDS/registry-api/issues/15)) + + +This requirement is not impacted by the current version +## As an API user, I want an average query response time of 1 second for q=* queries ([#16](https://github.com/NASA-PDS/registry-api/issues/16)) + + +This requirement is not impacted by the current version +## As an API caller(user) I want to specify fields for endpoints given a lidvid ([#80](https://github.com/NASA-PDS/registry-api/issues/80)) + + +This requirement is not impacted by the current version +## the default proposed mime type in swagger-ui.html should be 'application/json' ([#88](https://github.com/NASA-PDS/registry-api/issues/88)) + + +This requirement is not impacted by the current version +## As a user, I want to have an administrator contact when I am getting an error 500 from the server ([#109](https://github.com/NASA-PDS/registry-api/issues/109)) + + +This requirement is not impacted by the current version +## As a user I want consistent /type/lidvlid/{child,parent}/{latest,all} ([#126](https://github.com/NASA-PDS/registry-api/issues/126)) + + +This requirement is not impacted by the current version +## As a user, I want to get the product identifiers in a application/kvp+json format ([#132](https://github.com/NASA-PDS/registry-api/issues/132)) + + +This requirement is not impacted by the current version \ No newline at end of file diff --git a/model/pom.xml b/model/pom.xml index ae68f2d7..e0560d44 100644 --- a/model/pom.xml +++ b/model/pom.xml @@ -120,27 +120,22 @@ io.springfox - springfox-swagger2 - ${springfox-version} - - - io.springfox - springfox-swagger-ui - ${springfox-version} - - + springfox-boot-starter + 3.0.0 + + org.springframework spring-web - 5.3.19 + 5.3.20 org.springframework spring-context - 5.3.19 + 5.3.20 diff --git a/model/swagger.yml b/model/swagger.yml index 8adcfa55..48e70c0d 100644 --- a/model/swagger.yml +++ b/model/swagger.yml @@ -17,13 +17,13 @@ servers: description: production server paths: - /products: + /gid/{group}: get: tags: - products summary: | - returns all PDS data products, including bundles, collections, documentation, context and observational products that meet all given constraints. - operationId: products + returns all PDS products of the given PDS product class that meet all given constraints. + operationId: group-list responses: '200': $ref: "#/components/responses/Plural" @@ -37,20 +37,20 @@ paths: $ref: "#/components/responses/Error" parameters: - $ref: "#/components/parameters/Fields" + - $ref: "#/components/parameters/Group" - $ref: "#/components/parameters/Keyword" - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Query" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /products/{identifier}: + /uid/{identifier}: get: tags: - products summary: | - returns one product with the identifier selected from all PDS data products, including bundles, collections, documentation, context and observational products. See identifier for details on how it determines what is returned. - operationId: products-by-lidvid + returns one PDS Product with the identifier selected from all PDS Products, including bundles, collections, data, documentation, context and observational products. See identifier for details on how it determines what is returned. + operationId: select-by-lidvid responses: '200': $ref: "#/components/responses/Singular" @@ -65,13 +65,13 @@ paths: parameters: - $ref: "#/components/parameters/Fields" - $ref: "#/components/parameters/Identifier" - /products/{identifier}/latest: + /uid/{identifier}/latest: get: tags: - products summary: | - returns one product with the identifier selected from all PDS data products, including bundles, collections, documentation, context and observational products. See identifier for details on how it determines what is returned. - operationId: products-by-lidvid-latest + returns one PDS Product with the identifier selected from all PDS Products. See identifier for details on how it determines what is returned. + operationId: select-by-lidvid-latest responses: '200': $ref: "#/components/responses/Singular" @@ -86,13 +86,15 @@ paths: parameters: - $ref: "#/components/parameters/Fields" - $ref: "#/components/parameters/Identifier" - /products/{identifier}/all: + /uid/{identifier}/all: get: tags: - products summary: | - returns all product versions with the identifier (lid) selected from all PDS data products, including bundles, collections, documentation, context and observational products. See identifier for details on how it determines what is returned. - operationId: products-by-lidvid-all + returns all product versions with the identifier (lid) selected from all PDS Products. See identifier for details on how it determines what is returned. + + note: if given a lidvid, it will be translated to a lid + operationId: select-by-lidvid-all responses: '200': $ref: "#/components/responses/Plural" @@ -110,14 +112,13 @@ paths: - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /products/{identifier}/bundles: + /gid/{group}/referencing/{identifier}: get: tags: - - products containing bundles + - product references summary: | - returns one or more bundles that contain or reference the product specified by identifier selected from all PDS data products, including bundles, collections, documentation, context and observational products. See identifier for details on how it determines what is returned. - operationId: bundles-containing-product + returns one or more PDS Products of the specificed class that contain or reference the specified product identifier. See identifier for details on how it determines what is returned. + operationId: group-referencing-id responses: '200': $ref: "#/components/responses/Plural" @@ -131,18 +132,18 @@ paths: $ref: "#/components/responses/Error" parameters: - $ref: "#/components/parameters/Fields" + - $ref: "#/components/parameters/Group" - $ref: "#/components/parameters/Identifier" - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /products/{identifier}/collections: + /gid/{group}/referencing/{identifier}/{versions}: get: tags: - - product containing collections + - product references summary: | - returns one or more collections that contain or reference the product specified by identifier selected from all PDS data products, including bundles, collections, documentation, context and observational products. See identifier for details on how it determines what is returned. - operationId: collections-containing-product + returns one or more PDS Products of the specificed class that reference the product identifier. See identifier for details on how it determines what is returned. + operationId: group-referencing-id-vers responses: '200': $ref: "#/components/responses/Plural" @@ -156,329 +157,19 @@ paths: $ref: "#/components/responses/Error" parameters: - $ref: "#/components/parameters/Fields" + - $ref: "#/components/parameters/Group" - $ref: "#/components/parameters/Identifier" - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections: + - $ref: "#/components/parameters/Versions" + /uid/{identifier}/referencing/{group}: get: tags: - - collections + - product references summary: | - returns all PDS collections that meet all given constraints. - operationId: get collection - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Keyword" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Query" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections/{identifier}: - get: - tags: - - collections - summary: | - returns one product with the identifier selected from all PDS collections. See identifier for details on how it determines what is returned. - operationId: collections-by-lidvid - responses: - '200': - $ref: "#/components/responses/Singular" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - /collections/{identifier}/latest: - get: - tags: - - collections - summary: | - returns one product with the identifier selected from all PDS collections. See identifier for details on how it determines what is returned. - operationId: collections-by-lidvid-latest - responses: - '200': - $ref: "#/components/responses/Singular" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - /collections/{identifier}/all: - get: - tags: - - collections - summary: | - returns all collection versions with the identifier (lid) selected from all PDS collections. See identifier for details on how it determines what is returned. - operationId: collections-by-lidvid-all - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections/{identifier}/bundles: - get: - tags: - - collection's containing bundles - summary: | - returns one or more bundles that contain or reference the product specified by identifier selected from all PDS collections. See identifier for details on how it determines what is returned. - operationId: bundles-containing-collection - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections/{identifier}/products: - get: - tags: - - collection's products - summary: | - returns one or more latest version products that are contained or referenced by the collection specified by identifier. See identifier for details on how it determines what is returned. - operationId: products-of-a-collection - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections/{identifier}/products/latest: - get: - tags: - - collection's products - summary: | - returns one or more latest version products that are contained or referenced by the collection specified by identifier. See identifier for details on how it determines what is returned. - operationId: products-of-a-collection-latest - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /collections/{identifier}/products/all: - get: - tags: - - collection's products - summary: | - returns one or more products that are contained or referenced by the collection specified by identifier. See identifier for details on how it determines what is returned. - operationId: products-of-a-collection-all - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles: - get: - tags: - - bundles - summary: | - returns all PDS bundles that meet all given constraints. - operationId: get bundles - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Keyword" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Query" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles/{identifier}: - get: - tags: - - bundles - summary: | - returns one product with the identifier selected from all PDS bundles. See identifier for details on how it determines what is returned. - operationId: bundle-by-lidvid - responses: - '200': - $ref: "#/components/responses/Singular" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - /bundles/{identifier}/latest: - get: - tags: - - bundles - summary: | - returns one product with the identifier selected from all PDS bundles. See identifier for details on how it determines what is returned. - operationId: bundle-by-lidvid-latest - responses: - '200': - $ref: "#/components/responses/Singular" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - /bundles/{identifier}/all: - get: - tags: - - bundles - summary: | - returns all product versions with the identifier (lid) selected from all PDS bundles. See identifier for details on how it determines what is returned. - operationId: bundle-by-lidvid-all - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles/{identifier}/collections: - get: - tags: - - bundle's collections - summary: | - returns one or more latest version collections that are contained or referenced by the bundle specified by identifier. See identifier for details on how it determines what is returned. - operationId: collections-of-a-bundle - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles/{identifier}/collections/latest: - get: - tags: - - bundle's collections - summary: | - returns one or more latest version collections that are contained or referenced by the bundle specified by identifier. See identifier for details on how it determines what is returned. - operationId: collections-of-a-bundle-latest + returns one or more PDS Products of the specificed class that the specified product identifier contains or references. See identifier for details on how it determines what is returned. + operationId: id-referencing-group responses: '200': $ref: "#/components/responses/Plural" @@ -492,18 +183,18 @@ paths: $ref: "#/components/responses/Error" parameters: - $ref: "#/components/parameters/Fields" + - $ref: "#/components/parameters/Group" - $ref: "#/components/parameters/Identifier" - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles/{identifier}/collections/all: + /uid/{identifier}/referencing/{group}/{versions}: get: tags: - - bundle's collections + - product references summary: | - returns one or more collections that are contained or referenced by the bundle specified by identifier. See identifier for details on how it determines what is returned. - operationId: collections-of-a-bundle-all + returns one or more PDS Products of the specificed class that the specified product identifier contains or references. See identifier for details on how it determines what is returned. + operationId: id-referencing-group--vers responses: '200': $ref: "#/components/responses/Plural" @@ -517,36 +208,12 @@ paths: $ref: "#/components/responses/Error" parameters: - $ref: "#/components/parameters/Fields" + - $ref: "#/components/parameters/Group" - $ref: "#/components/parameters/Identifier" - $ref: "#/components/parameters/Limit" - $ref: "#/components/parameters/Sort" - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" - /bundles/{identifier}/products: - get: - tags: - - bundle's products - summary: | - returns one or more latest version products that are contained or referenced by the bundle specified by identifier. See identifier for details on how it determines what is returned. - operationId: products-of-a-bundle - responses: - '200': - $ref: "#/components/responses/Plural" - '400': - $ref: "#/components/responses/Error" - '404': - $ref: "#/components/responses/Error" - '500': - $ref: "#/components/responses/Error" - '501': - $ref: "#/components/responses/Error" - parameters: - - $ref: "#/components/parameters/Fields" - - $ref: "#/components/parameters/Identifier" - - $ref: "#/components/parameters/Limit" - - $ref: "#/components/parameters/Sort" - - $ref: "#/components/parameters/Start" - - $ref: "#/components/parameters/SummaryOnly" + - $ref: "#/components/parameters/Versions" externalDocs: description: Find out more information on the PDS API Specification @@ -567,6 +234,17 @@ components: type: array items: type: string + Group: + name: group + in: path + description: | + syntax: one of the allowable enum values + + This is a shortcut for doing a query with ... + required: true + schema: + type: string + enum: [any,bundle,collection,product] Identifier: name: identifier in: path @@ -603,10 +281,12 @@ components: syntax: limit=10 behavior: maximum number of matching results returned, for pagination + + note: limit=0 returns just the summary required: false schema: type: integer - minimum: 1 + minimum: 0 default: 100 Query: name: q @@ -644,16 +324,17 @@ components: type: integer minimum: 0 default: 0 - SummaryOnly: - name: summary-only - in: query + Versions: + name: versions + in: path description: | - syntax: summary-only={true,false} - - behavior: only return the summary when a list is returned. Useful to get the list of available properties. + syntax: one of the allowable enum values + + This is a shortcut for doing a query with ... + required: true schema: - type: boolean - default: false + type: string + enum: [all,latest] responses: Error: description: Unsuccessful request diff --git a/service/docker/Dockerfile b/service/docker/Dockerfile index cd76e345..79026c41 100644 --- a/service/docker/Dockerfile +++ b/service/docker/Dockerfile @@ -80,7 +80,7 @@ WORKDIR /usr/local/registry-api-service EXPOSE 80 CMD [ \ "java", \ - "-jar", "/usr/local/registry-api-service/registry-api-service.jar", ] \ + "-jar", "/usr/local/registry-api-service/registry-api-service.jar", \ "gov.nasa.pds.api.registry.SpringBootMain" \ ] diff --git a/service/docker/Dockerfile.aws b/service/docker/Dockerfile.aws new file mode 100644 index 00000000..ff845876 --- /dev/null +++ b/service/docker/Dockerfile.aws @@ -0,0 +1,100 @@ +# Copyright © 2022, California Institute of Technology ("Caltech"). +# U.S. Government sponsorship acknowledged. +# +# All rights reserved. +# +# Redistribution and use in source and binary forms, with or without +# modification, are permitted provided that the following conditions are met: +# +# • Redistributions of source code must retain the above copyright notice, +# this list of conditions and the following disclaimer. +# • Redistributions must reproduce the above copyright notice, this list of +# conditions and the following disclaimer in the documentation and/or other +# materials provided with the distribution. +# • Neither the name of Caltech nor its operating division, the Jet Propulsion +# Laboratory, nor the names of its contributors may be used to endorse or +# promote products derived from this software without specific prior written +# permission. +# +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE +# POSSIBILITY OF SUCH DAMAGE. +# +# Dockerfile for the Registry API +# =============================== +# +# Depending on the build arguments, you could get a Dockerfile using local +# assets or one with built and released assets. +# +# +# Basis +# ----- +# +# Normally we'd prefer Alpine Linux, but JDK 11 isn't available with it, so +# we go with a slim Debian. Debian's good too. + +FROM openjdk:11-slim + + +# API JAR file +# ------------ +# +# Provide a `--build-arg` to tell the image builder where the Registry API +# `.jar` file may found. This can be a file on the filesystem (sent as part +# of the build context) or a URL to a remote file. + +ARG api_jar + + +# Layering +# -------- +# +# Add the API JAR file and the libtcnative-1 package—which is needed for +# some reason. + +ADD $api_jar /usr/local/registry-api-service/registry-api-service.jar + +RUN : &&\ + apt-get update --quiet --yes &&\ + apt-get install --quiet --yes libtcnative-1 &&\ + apt-get autoclean --quiet --yes &&\ + rm --recursive --force /var/lib/apt/lists/* &&\ + : + +# Copy in the AWS-specific application properties file. The contents of this file dictates that certain runtime +# values are obtained in th manner appropriate to AWS (e.g. the Opensearch login is obtained from an environment +# variable that has been set via the Secrets Manager.). +COPY src/main/resources/application.properties.aws /usr/local/registry-api-service/application.properties + + +# Image Morphology +# ---------------- +# +# External context. + +WORKDIR /usr/local/registry-api-service +EXPOSE 80 +CMD [ \ + "java", \ + "-jar", "/usr/local/registry-api-service/registry-api-service.jar", \ + "gov.nasa.pds.api.registry.SpringBootMain" \ +] + + +# Labels +# ------ +# +# `org.label-schema` is deprecated, but no one can figure out whatever +# replaced it, so we'll use it here. + +LABEL "org.label-schema.name" = "PDS Registry API" +LABEL "org.label-schema.description" = "Planetary Data System's Application Programmer's Interface for the Registry" +LABEL "org.label-schema.url" = "https://github.com/NASA-PDS/registry-api" diff --git a/service/docker/README.md b/service/docker/README.md index c4f35198..8d032464 100644 --- a/service/docker/README.md +++ b/service/docker/README.md @@ -28,6 +28,10 @@ Building an image from a released jar file: $ docker image build --build-arg api_jar=https://github.com/NASA-PDS/registry-api/releases/download/v1.0.0/registry-api-service-1.0.0.jar --tag nasapds/registry-api-service:1.0.0 --file service/docker/Dockerfile . ``` +## 📍 Dockerfile.aws + +This Dockerfile is used to make images for AWS deployments of the registry manager. It copies in an AWS-specific application.properties file which has been setup to inform the service to obtain certain properties as environment variable values that have been injected by the Elastic Container Service (ECS) runtime. An example of this is the Opensearch credentials from the Secrets Manager. + ## 📍 Dockerfile.local You can ignore `Dockerfile.local` unless you're @al-niessner. diff --git a/service/pom.xml b/service/pom.xml index 9ac33716..6ec275b6 100644 --- a/service/pom.xml +++ b/service/pom.xml @@ -1,369 +1,370 @@ + Copyright © 2021–2021, California Institute of Technology ("Caltech"). + U.S. Government sponsorship acknowledged. + + All rights reserved. + + Redistribution and use in source and binary forms, with or without + modification, are permitted provided that the following conditions are met: + + • Redistributions of source code must retain the above copyright notice, + this list of conditions and the following disclaimer. + • Redistributions must reproduce the above copyright notice, this list of + conditions and the following disclaimer in the documentation and/or other + materials provided with the distribution. + • Neither the name of Caltech nor its operating division, the Jet Propulsion + Laboratory, nor the names of its contributors may be used to endorse or + promote products derived from this software without specific prior written + permission. + + THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS "AS IS" + AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE + IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE + ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT OWNER OR CONTRIBUTORS BE + LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR + CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF + SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS + INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN + CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) + ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE + POSSIBILITY OF SUCH DAMAGE. + --> - 4.0.0 - gov.nasa.pds - registry-api-service - 1.0.2 - gov.nasa.pds.api.registry-service - registry api service contributing to the PDS federated API - - 11 - 11 - UTF-8 - 2.13.2 - 3.0.0 - - - - - org.springframework.boot - spring-boot-maven-plugin - - - - repackage - - - - - gov.nasa.pds.api.registry.SpringBootMain - - - - org.apache.maven.plugins - maven-site-plugin - 3.8.2 - - - org.apache.maven.plugins - maven-assembly-plugin - 3.1.1 - - - bin-release - package - - single - - - true - - src/main/assembly/tar-assembly.xml - src/main/assembly/zip-assembly.xml - - - - - - posix - - - - - - - - - - ossrh - https://oss.sonatype.org/content/repositories/snapshots - - - ossrh - https://oss.sonatype.org/service/local/staging/deploy/maven2/ - - - dummy URL to enabling doc staging - https://{project.artifactId} - - - - - https://github.com/NASA-PDS/registry-api-service - scm:git:git@github.com/NASA-PDS/registry-api-service.git - scm:git:git@github.com:NASA-PDS/registry-api-service.git + 4.0.0 + gov.nasa.pds + registry-api-service + 1.1.0-SNAPSHOT + gov.nasa.pds.api.registry-service + registry api service contributing to the PDS federated API + + 11 + 11 + UTF-8 + 2.13.2 + 3.0.0 + + + + + src/main/resources + true + + + + + org.springframework.boot + spring-boot-maven-plugin + + + + repackage + + + + + gov.nasa.pds.api.registry.SpringBootMain + + + + org.apache.maven.plugins + maven-site-plugin + 3.8.2 + + + org.apache.maven.plugins + maven-assembly-plugin + 3.1.1 + + + bin-release + package + + single + + + true + + src/main/assembly/tar-assembly.xml + src/main/assembly/zip-assembly.xml + + + + + + posix + + + + + + + + + + ossrh + https://oss.sonatype.org/content/repositories/snapshots + + + ossrh + https://oss.sonatype.org/service/local/staging/deploy/maven2/ + + + dummy URL to enabling doc staging + https://{project.artifactId} + + + + + https://github.com/NASA-PDS/registry-api-service + scm:git:git@github.com/NASA-PDS/registry-api-service.git + scm:git:git@github.com:NASA-PDS/registry-api-service.git HEAD - - - - - io.springfox - springfox-swagger2 - ${springfox-version} - - - io.springfox - springfox-swagger-ui - ${springfox-version} - - - - - org.springframework - spring-web - - - - - org.springframework - spring-context - - - - - org.springframework.boot - spring-boot-starter-web - - - - org.springframework.boot - spring-boot-starter-thymeleaf - - - - org.springframework - spring-jdbc - - - - - - org.springframework.boot - spring-boot-autoconfigure - - - - - - javax.servlet - servlet-api - 2.5 - provided - - - - - - javax.validation - validation-api - - - - - com.fasterxml.jackson.jaxrs - jackson-jaxrs-base - ${jackson-version} - - - com.fasterxml.jackson.core - jackson-core - ${jackson-version} - - - com.fasterxml.jackson.core - jackson-annotations - - - com.fasterxml.jackson.core - jackson-databind - ${jackson-version}.2 - - - com.fasterxml.jackson.jaxrs - jackson-jaxrs-json-provider - ${jackson-version} - - - - com.fasterxml.jackson.dataformat - jackson-dataformat-xml - ${jackson-version} - - - - - com.github.joschi.jackson - jackson-datatype-threetenbp - 2.12.5 - - - - - joda-time - joda-time - 2.10.6 - - - - com.sun.xml.bind - jaxb-core - 2.3.0.1 - - - javax.xml.bind - jaxb-api - 2.3.1 - - - com.sun.xml.bind - jaxb-impl - 2.3.1 - - - org.javassist - javassist - 3.25.0-GA - - - - - org.threeten - threetenbp - 1.4.4 - - - - gov.nasa.pds - registry-api-model - 1.0.1 - - - - gov.nasa.pds - registry-api-lexer - 1.1.1 - - - - org.antlr - antlr4-runtime - 4.9.2 - - - - org.opensearch.client - opensearch-rest-client - 1.2.4 - - - org.opensearch.client - opensearch-rest-high-level-client - 1.2.4 - - - - - org.apache.httpcomponents - httpclient - 4.5.13 - - - - - org.apache.commons - commons-collections4 - 4.2 - - - - - org.springframework.boot - spring-boot-starter-test - - - junit - junit - - - - - - - org.junit.jupiter - junit-jupiter-api - - - org.junit.jupiter - junit-jupiter-engine - - - org.springframework.boot - spring-boot-starter-validation - - - - com.amazonaws - aws-java-sdk-secretsmanager - 1.12.201 - - - - - com.google.guava - guava - 31.1-jre - - - - - + + + + + io.springfox + springfox-boot-starter + 3.0.0 + + + + + org.springframework + spring-web + + + + + org.springframework + spring-context + + + + + org.springframework.boot + spring-boot-starter-web + + + + org.springframework.boot + spring-boot-starter-thymeleaf + + + + org.springframework + spring-jdbc + + + + + + org.springframework.boot + spring-boot-autoconfigure + + + + + + javax.servlet + servlet-api + 2.5 + provided + + + + + + javax.validation + validation-api + + + + + com.fasterxml.jackson.jaxrs + jackson-jaxrs-base + ${jackson-version} + + + com.fasterxml.jackson.core + jackson-core + ${jackson-version} + + + com.fasterxml.jackson.core + jackson-annotations + + + com.fasterxml.jackson.core + jackson-databind + ${jackson-version}.2 + + + com.fasterxml.jackson.jaxrs + jackson-jaxrs-json-provider + ${jackson-version} + + + + com.fasterxml.jackson.dataformat + jackson-dataformat-xml + ${jackson-version} + + + + + com.github.joschi.jackson + jackson-datatype-threetenbp + 2.12.5 + + + + + joda-time + joda-time + 2.10.6 + + + + com.sun.xml.bind + jaxb-core + 2.3.0.1 + + + javax.xml.bind + jaxb-api + 2.3.1 + + + com.sun.xml.bind + jaxb-impl + 2.3.1 + + + org.javassist + javassist + 3.25.0-GA + + + + + org.threeten + threetenbp + 1.4.4 + + + + gov.nasa.pds + registry-api-model + 1.0.1 + + + + gov.nasa.pds + registry-api-lexer + 1.1.1 + + + + org.antlr + antlr4-runtime + 4.9.2 + + + + org.opensearch.client + opensearch-rest-client + 1.2.4 + + + org.opensearch.client + opensearch-rest-high-level-client + 1.2.4 + + + + + org.apache.httpcomponents + httpclient + 4.5.13 + + + + + org.apache.commons + commons-collections4 + 4.2 + + + + + org.springframework.boot + spring-boot-starter-test + + + junit + junit + + + + + + + org.junit.jupiter + junit-jupiter-api + + + org.junit.jupiter + junit-jupiter-engine + + + org.springframework.boot + spring-boot-starter-validation + + + + com.amazonaws + aws-java-sdk-secretsmanager + 1.12.201 + + + + + com.google.guava + guava + 31.1-jre + + + + + - - org.springframework.boot - spring-boot-dependencies - 2.6.6 - pom - import - + + org.springframework.boot + spring-boot-dependencies + 2.7.0 + pom + import + - - - - - oss.sonatype.org-snapshot - https://oss.sonatype.org/content/repositories/snapshots - - true - - - true - - - + + + + + oss.sonatype.org-snapshot + https://oss.sonatype.org/content/repositories/snapshots + + true + + + true + + + diff --git a/service/src/main/java/gov/nasa/pds/api/registry/GroupConstraint.java b/service/src/main/java/gov/nasa/pds/api/registry/GroupConstraint.java new file mode 100644 index 00000000..f7eebae9 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/GroupConstraint.java @@ -0,0 +1,23 @@ +package gov.nasa.pds.api.registry; + +import java.util.List; +import java.util.Map; + +import com.google.errorprone.annotations.Immutable; + +@Immutable +public interface GroupConstraint +{ + /** + * A set of PDS keywords/values that ALL must be true to define just PDS items that make up this Group. + */ + public Map> all(); + /** + * A set of PDS keywords/values that ANY must be true to define just PDS items that make up this Group. + */ + public Map> any(); + /** + * A set of PDS keywords/values that NONE must be true to define just PDS items that make up this Group. + */ + public Map> not(); +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/LidvidsContext.java b/service/src/main/java/gov/nasa/pds/api/registry/LidvidsContext.java new file mode 100644 index 00000000..11bffa9a --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/LidvidsContext.java @@ -0,0 +1,8 @@ +package gov.nasa.pds.api.registry; + +public interface LidvidsContext +{ + public String getLidVid(); + public Integer getLimit(); + public Integer getStart(); +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/ReferencingLogic.java b/service/src/main/java/gov/nasa/pds/api/registry/ReferencingLogic.java new file mode 100644 index 00000000..8c255617 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/ReferencingLogic.java @@ -0,0 +1,31 @@ +package gov.nasa.pds.api.registry; + +import java.io.IOException; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.model.RequestAndResponseContext; + +@Immutable +public interface ReferencingLogic +{ + /** + * Map the set of PDS constraints that define just PDS items that make up this Group. + */ + public GroupConstraint constraints(); + + /** + * Find all of the PDS items of the given Group that reference the specified ID. + */ + public RequestAndResponseContext find(ControlContext context, UserContext input) + throws ApplicationTypeException, IOException, LidVidNotFoundException, UnknownGroupNameException; + + /** + * Find all of the PDS items of the given ID that reference the specified Group. + */ + public RequestAndResponseContext given(ControlContext context, UserContext input) + throws ApplicationTypeException, IOException, LidVidNotFoundException, UnknownGroupNameException; +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/RequestBuildContext.java b/service/src/main/java/gov/nasa/pds/api/registry/RequestBuildContext.java index 98e3b8d0..5e5c77e5 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/RequestBuildContext.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/RequestBuildContext.java @@ -1,10 +1,9 @@ package gov.nasa.pds.api.registry; import java.util.List; -import java.util.Map; public interface RequestBuildContext { public List getFields(); // must not return null but an empty list - public Map getPresetCriteria(); // must not return null but an empty list + public GroupConstraint getPresetCriteria(); // must not return null but an empty list } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/SpringBootMain.java b/service/src/main/java/gov/nasa/pds/api/registry/SpringBootMain.java index d707213d..9d36d34f 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/SpringBootMain.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/SpringBootMain.java @@ -10,7 +10,7 @@ @SpringBootApplication @EnableSwagger2 -@ComponentScan(basePackages = { "gov.nasa.pds.api.registry.business", "gov.nasa.pds.api.registry.configuration ", "gov.nasa.pds.api.registry.controllers", "gov.nasa.pds.api.registry.opensearch"}) +@ComponentScan(basePackages = { "gov.nasa.pds.api.registry.configuration ", "gov.nasa.pds.api.registry.controller", "gov.nasa.pds.api.registry.model", "gov.nasa.pds.api.registry.search"}) public class SpringBootMain implements CommandLineRunner { diff --git a/service/src/main/java/gov/nasa/pds/api/registry/UserContext.java b/service/src/main/java/gov/nasa/pds/api/registry/UserContext.java index 5eca0112..4a05c346 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/UserContext.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/UserContext.java @@ -2,17 +2,17 @@ import java.util.List; -import gov.nasa.pds.api.registry.business.ProductVersionSelector; +import gov.nasa.pds.api.registry.model.ProductVersionSelector; -public interface UserContext +public interface UserContext extends LidvidsContext { + public String getAccept(); public List getFields(); + public String getGroup(); public String getIdentifier(); public List getKeywords(); - public Integer getLimit(); public String getQuery(); public ProductVersionSelector getSelector(); public List getSort(); - public Integer getStart(); - public Boolean getSummanryOnly(); + public String getVersion(); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/BundleDAO.java b/service/src/main/java/gov/nasa/pds/api/registry/business/BundleDAO.java deleted file mode 100644 index 255e76d0..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/BundleDAO.java +++ /dev/null @@ -1,174 +0,0 @@ -package gov.nasa.pds.api.registry.business; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.Arrays; -import java.util.HashMap; -import java.util.List; -import java.util.Map; -import java.util.Set; -import java.util.TreeSet; - -import org.opensearch.action.search.SearchRequest; -import org.opensearch.client.RequestOptions; -import org.opensearch.search.SearchHit; -import org.opensearch.search.SearchHits; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; - -import gov.nasa.pds.api.registry.ControlContext; -import gov.nasa.pds.api.registry.RequestBuildContext; -import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; - -/** - * Bundle Data Access Object (DAO). - * Provides methods to get bundle information from opensearch. - * - * @author karpenko - */ -public class BundleDAO -{ - @SuppressWarnings("unused") - private static final Logger log = LoggerFactory.getLogger(BundleDAO.class); - - static public Map searchConstraints() - { - Map preset = new HashMap(); - preset.put("product_class", "Product_Bundle"); - return preset; - } - - /** - * Get collections of a bundle by bundle LIDVID. - * If a bundle has LIDVID collection references, then those collections are returned. - * If a bundle has LID collection references, then the latest versions of collections are returned. - * @return a list of collection LIDVIDs - * @throws IOException IO exception - * @throws LidVidNotFoundException LIDVID not found exception - */ - static public List getBundleCollectionLidVids( - String bundleLidVid, - ControlContext ctlContext, - RequestBuildContext reqBuildContext) - throws IOException, LidVidNotFoundException - { - // Fetch collection references only. - List fields = new ArrayList( - Arrays.asList("ref_lidvid_collection","ref_lidvid_collection_secondary", - "ref_lid_collection", "ref_lid_collection_secondary")); - - // Get bundle by lidvid. - SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(bundleLidVid), ctlContext.getConnection()) - .build(RequestBuildContextFactory.given(fields, BundleDAO.searchConstraints()), ctlContext.getConnection().getRegistryIndex()); - - // Call opensearch - SearchHit hit; - SearchHits hits = ctlContext.getConnection().getRestHighLevelClient().search(request, RequestOptions.DEFAULT).getHits(); - if(hits == null || hits.getTotalHits() == null || hits.getTotalHits().value != 1) - throw new LidVidNotFoundException(bundleLidVid); - else hit = hits.getAt(0); - - // Get fields - // LidVid references (e.g., OREX bundle) - Map fieldMap = hit.getSourceAsMap(); - List primaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lidvid_collection"); - List secondaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lidvid_collection_secondary"); - - List lidVids = new ArrayList(); - if(primaryIds != null) lidVids.addAll(primaryIds); - if(secondaryIds != null) lidVids.addAll(secondaryIds); - - // !!! NOTE !!! - // Harvest converts LIDVID references to LID references and stores them in - // "ref_lid_collection" and "ref_lid_collection_secondary" fields. - // To get "real" LID references, we have to exclude LIDVID references from these fields. - Set lidsToRemove = new TreeSet(); - for(String lidVid: lidVids) - { - int idx = lidVid.indexOf("::"); - if(idx > 0) - { - String lid = lidVid.substring(0, idx); - lidsToRemove.add(lid); - } - } - - // Lid references (e.g., Kaguya bundle) plus LIDVID references converted by Harvest - primaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lid_collection"); - secondaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lid_collection_secondary"); - - List lids = new ArrayList(); - if(primaryIds != null) lids.addAll(primaryIds); - if(secondaryIds != null) lids.addAll(secondaryIds); - - // Get "real" LIDs - if(!lidsToRemove.isEmpty()) - { - lids.removeAll(lidsToRemove); - } - - // Get the latest versions of LIDs - if(!lids.isEmpty()) - { - List latestLidVids = LidVidUtils.getLatestLidVidsByLids(ctlContext, - RequestBuildContextFactory.given(reqBuildContext.getFields(), CollectionDAO.searchConstraints()), lids); - lidVids.addAll(latestLidVids); - } - - return lidVids; - } - - - /** - * Get all versions of bundle's collections by bundle LIDVID. - * @param bundleLidVid bundle LIDVID (Could not pass LID here) - * @return a list of collection LIDVIDs - * @throws IOException IO exception - * @throws LidVidNotFoundException LIDVID not found exception - */ - static public List getAllBundleCollectionLidVids( - String bundleLidVid, - ControlContext ctlContext, - RequestBuildContext reqBuildContext) - throws IOException, LidVidNotFoundException - { - // Fetch collection references only. - List fields = new ArrayList( - Arrays.asList("ref_lid_collection", "ref_lid_collection_secondary")); - - // Get bundle by lidvid. - SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(bundleLidVid), ctlContext.getConnection()) - .build(RequestBuildContextFactory.given(fields, BundleDAO.searchConstraints()), ctlContext.getConnection().getRegistryIndex()); - - // Call opensearch - SearchHit hit; - SearchHits hits = ctlContext.getConnection().getRestHighLevelClient().search(request, RequestOptions.DEFAULT).getHits(); - if (hits == null || hits.getTotalHits() == null || hits.getTotalHits().value != 1) - throw new LidVidNotFoundException(bundleLidVid); - else hit = hits.getAt(0); - - // Get fields - Map fieldMap = hit.getSourceAsMap(); - - // Lid references (e.g., Kaguya bundle) - List primaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lid_collection"); - List secondaryIds = ResponseUtils.getFieldValues(fieldMap, "ref_lid_collection_secondary"); - - List ids = new ArrayList(); - if(primaryIds != null) ids.addAll(primaryIds); - if(secondaryIds != null) ids.addAll(secondaryIds); - - if(!ids.isEmpty()) - { - // Get the latest versions of LIDs (Return LIDVIDs) - ids = LidVidUtils.getAllLidVidsByLids(ctlContext, reqBuildContext, ids); - return ids; - } - - return new ArrayList(0); - } - -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/CollectionDAO.java b/service/src/main/java/gov/nasa/pds/api/registry/business/CollectionDAO.java deleted file mode 100644 index 968dd73a..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/CollectionDAO.java +++ /dev/null @@ -1,14 +0,0 @@ -package gov.nasa.pds.api.registry.business; - -import java.util.HashMap; -import java.util.Map; - -public class CollectionDAO -{ - static public Map searchConstraints() - { - Map preset = new HashMap(); - preset.put("product_class", "Product_Collection"); - return preset; - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductDAO.java b/service/src/main/java/gov/nasa/pds/api/registry/business/ProductDAO.java deleted file mode 100644 index e97af05c..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductDAO.java +++ /dev/null @@ -1,13 +0,0 @@ -package gov.nasa.pds.api.registry.business; - -import java.util.HashMap; -import java.util.Map; - -public class ProductDAO -{ - static public Map searchConstraints() - { - Map preset = new HashMap(); - return preset; - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ResponseUtils.java b/service/src/main/java/gov/nasa/pds/api/registry/business/ResponseUtils.java deleted file mode 100644 index 99731ba6..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ResponseUtils.java +++ /dev/null @@ -1,29 +0,0 @@ -package gov.nasa.pds.api.registry.business; - -import java.util.Arrays; -import java.util.List; -import java.util.Map; - - -public class ResponseUtils -{ - @SuppressWarnings({"unchecked", "rawtypes"}) - public static List getFieldValues(Map fieldMap, String fieldName) - { - if(fieldMap == null || fieldName == null) return null; - - Object obj = fieldMap.get(fieldName); - if(obj == null) return null; - - if(obj instanceof String) - { - return Arrays.asList((String)obj); - } - else if(obj instanceof List) - { - return (List)obj; - } - - return null; - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/configuration/ApiDocumentation.java b/service/src/main/java/gov/nasa/pds/api/registry/configuration/ApiDocumentation.java index 86fab0db..ff76ffc5 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/configuration/ApiDocumentation.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/configuration/ApiDocumentation.java @@ -22,7 +22,7 @@ public String index() { String contextPath = this.contextPath.endsWith("/")?this.contextPath:this.contextPath+"/"; - System.out.println(contextPath+"swagger-ui.html"); - return "redirect:"+contextPath+"swagger-ui.html"; + System.out.println(contextPath+"swagger-ui/index.html"); + return "redirect:"+contextPath+"swagger-ui/index.html"; } } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/configuration/SwaggerDocumentationConfig.java b/service/src/main/java/gov/nasa/pds/api/registry/configuration/SwaggerDocumentationConfig.java index 02737986..e479d12c 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/configuration/SwaggerDocumentationConfig.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/configuration/SwaggerDocumentationConfig.java @@ -1,5 +1,6 @@ package gov.nasa.pds.api.registry.configuration; +import org.springframework.beans.factory.annotation.Value; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; @@ -15,14 +16,17 @@ @Configuration public class SwaggerDocumentationConfig { - ApiInfo apiInfo() { + @Value("${registry.service.version:undefined}") + private String version; + + ApiInfo apiInfo() { return new ApiInfoBuilder() .title("PDS federated API") .description("Federated PDS API which provides actionable end points standardized between the different nodes. ") .license("Apache 2.0") .licenseUrl("http://www.apache.org/licenses/LICENSE-2.0.html") .termsOfServiceUrl("") - .version("0.0.1") + .version(this.version) .contact(new Contact("","", "pds_operator@jpl.nasa.gov")) .build(); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/configuration/WebMVCConfig.java b/service/src/main/java/gov/nasa/pds/api/registry/configuration/WebMVCConfig.java index 91a27352..2f0dd222 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/configuration/WebMVCConfig.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/configuration/WebMVCConfig.java @@ -14,27 +14,27 @@ import org.springframework.web.servlet.config.annotation.ResourceHandlerRegistry; import org.springframework.web.servlet.config.annotation.WebMvcConfigurer; -import gov.nasa.pds.api.registry.serializer.CsvErrorMessageSerializer; -import gov.nasa.pds.api.registry.serializer.CsvPluralSerializer; -import gov.nasa.pds.api.registry.serializer.CsvSingularSerializer; -import gov.nasa.pds.api.registry.serializer.HtmlErrorMessageSerializer; -import gov.nasa.pds.api.registry.serializer.JsonErrorMessageSerializer; -import gov.nasa.pds.api.registry.serializer.JsonPluralSerializer; -import gov.nasa.pds.api.registry.serializer.JsonProductSerializer; -import gov.nasa.pds.api.registry.serializer.JsonSingularSerializer; -import gov.nasa.pds.api.registry.serializer.Pds4JsonProductSerializer; -import gov.nasa.pds.api.registry.serializer.Pds4JsonProductsSerializer; -import gov.nasa.pds.api.registry.serializer.Pds4XmlProductSerializer; -import gov.nasa.pds.api.registry.serializer.Pds4XmlProductsSerializer; -import gov.nasa.pds.api.registry.serializer.PdsProductTextHtmlSerializer; -import gov.nasa.pds.api.registry.serializer.PdsProductXMLSerializer; -import gov.nasa.pds.api.registry.serializer.PdsProductsTextHtmlSerializer; -import gov.nasa.pds.api.registry.serializer.PdsProductsXMLSerializer; -import gov.nasa.pds.api.registry.serializer.XmlErrorMessageSerializer; +import gov.nasa.pds.api.registry.view.CsvErrorMessageSerializer; +import gov.nasa.pds.api.registry.view.CsvPluralSerializer; +import gov.nasa.pds.api.registry.view.CsvSingularSerializer; +import gov.nasa.pds.api.registry.view.HtmlErrorMessageSerializer; +import gov.nasa.pds.api.registry.view.JsonErrorMessageSerializer; +import gov.nasa.pds.api.registry.view.JsonPluralSerializer; +import gov.nasa.pds.api.registry.view.JsonProductSerializer; +import gov.nasa.pds.api.registry.view.JsonSingularSerializer; +import gov.nasa.pds.api.registry.view.Pds4JsonProductSerializer; +import gov.nasa.pds.api.registry.view.Pds4JsonProductsSerializer; +import gov.nasa.pds.api.registry.view.Pds4XmlProductSerializer; +import gov.nasa.pds.api.registry.view.Pds4XmlProductsSerializer; +import gov.nasa.pds.api.registry.view.PdsProductTextHtmlSerializer; +import gov.nasa.pds.api.registry.view.PdsProductXMLSerializer; +import gov.nasa.pds.api.registry.view.PdsProductsTextHtmlSerializer; +import gov.nasa.pds.api.registry.view.PdsProductsXMLSerializer; +import gov.nasa.pds.api.registry.view.XmlErrorMessageSerializer; @Configuration @EnableWebMvc -@ComponentScan(basePackages = { "gov.nasa.pds.api.registry.configuration ", "gov.nasa.pds.api.registry.controllers", "gov.nasa.pds.api.registry.opensearch"}) +@ComponentScan(basePackages = { "gov.nasa.pds.api.registry.configuration ", "gov.nasa.pds.api.registry.controller", "gov.nasa.pds.api.registry.search"}) public class WebMVCConfig implements WebMvcConfigurer { private static final Logger log = LoggerFactory.getLogger(WebMVCConfig.class); @@ -42,9 +42,6 @@ public class WebMVCConfig implements WebMvcConfigurer @Override public void addResourceHandlers(ResourceHandlerRegistry registry) { - registry.addResourceHandler("swagger-ui.html") - .addResourceLocations("classpath:/META-INF/resources/"); - registry.addResourceHandler("/webjars/**") .addResourceLocations("classpath:/META-INF/resources/webjars/"); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controller/EndpointHandler.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/EndpointHandler.java new file mode 100644 index 00000000..9a4c8493 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/EndpointHandler.java @@ -0,0 +1,18 @@ +package gov.nasa.pds.api.registry.controller; + +import java.io.IOException; + +import org.springframework.http.ResponseEntity; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.UserContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; + +interface EndpointHandler +{ + public ResponseEntity transmute (ControlContext control, UserContext content) + throws ApplicationTypeException,IOException,LidVidNotFoundException,NothingFoundException,UnknownGroupNameException; +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controller/GroupReferencingId.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/GroupReferencingId.java new file mode 100644 index 00000000..21c9b596 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/GroupReferencingId.java @@ -0,0 +1,28 @@ +package gov.nasa.pds.api.registry.controller; + +import java.io.IOException; + +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.UserContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.model.ReferencingLogicTransmuter; +import gov.nasa.pds.api.registry.model.RequestAndResponseContext; + +class GroupReferencingId implements EndpointHandler +{ + @Override + public ResponseEntity transmute(ControlContext control, UserContext content) + throws ApplicationTypeException, IOException, LidVidNotFoundException, NothingFoundException, + UnknownGroupNameException + { + RequestAndResponseContext context = ReferencingLogicTransmuter.getBySwaggerGroup(content.getGroup()).impl() + .find(control, content); + return new ResponseEntity(context.getResponse(), HttpStatus.OK); + } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controller/IdReferencingGroup.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/IdReferencingGroup.java new file mode 100644 index 00000000..4c4cbcd0 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/IdReferencingGroup.java @@ -0,0 +1,28 @@ +package gov.nasa.pds.api.registry.controller; + +import java.io.IOException; + +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.UserContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.model.ReferencingLogicTransmuter; +import gov.nasa.pds.api.registry.model.RequestAndResponseContext; + +class IdReferencingGroup implements EndpointHandler +{ + @Override + public ResponseEntity transmute(ControlContext control, UserContext content) + throws ApplicationTypeException, IOException, LidVidNotFoundException, NothingFoundException, + UnknownGroupNameException + { + RequestAndResponseContext context = ReferencingLogicTransmuter.getBySwaggerGroup(content.getGroup()).impl() + .given(control, content); + return new ResponseEntity(context.getResponse(), HttpStatus.OK); + } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controller/Standard.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/Standard.java new file mode 100644 index 00000000..50dc1154 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/Standard.java @@ -0,0 +1,32 @@ +package gov.nasa.pds.api.registry.controller; + +import java.io.IOException; + +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.UserContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.model.ReferencingLogicTransmuter; +import gov.nasa.pds.api.registry.model.RequestAndResponseContext; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; + +class Standard implements EndpointHandler +{ + @Override + public ResponseEntity transmute(ControlContext control, UserContext content) + throws ApplicationTypeException, IOException, LidVidNotFoundException, NothingFoundException, UnknownGroupNameException + { + RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext + (control, content, ReferencingLogicTransmuter.getBySwaggerGroup (content.getGroup()).impl().constraints()); + context.setResponse(control.getConnection().getRestHighLevelClient(), + new SearchRequestFactory(context, control.getConnection()) + .build(context, control.getConnection().getRegistryIndex())); + return new ResponseEntity(context.getResponse(), HttpStatus.OK); + } + +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controller/SwaggerJavaTransmuter.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/SwaggerJavaTransmuter.java new file mode 100644 index 00000000..c94802e7 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/SwaggerJavaTransmuter.java @@ -0,0 +1,254 @@ +package gov.nasa.pds.api.registry.controller; + +import java.io.IOException; +import java.net.MalformedURLException; +import java.net.URL; +import java.util.List; + +import javax.servlet.http.HttpServletRequest; +import javax.validation.Valid; +import javax.validation.constraints.Min; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; +import org.springframework.beans.factory.annotation.Autowired; +import org.springframework.beans.factory.annotation.Value; +import org.springframework.http.HttpStatus; +import org.springframework.http.ResponseEntity; +import org.springframework.stereotype.Controller; + +import com.fasterxml.jackson.databind.ObjectMapper; + +import gov.nasa.pds.api.base.GidApi; +import gov.nasa.pds.api.base.UidApi; +import gov.nasa.pds.api.registry.ConnectionContext; +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.model.ErrorFactory; + +@Controller +public class SwaggerJavaTransmuter implements ControlContext, GidApi, UidApi +{ + private static final Logger log = LoggerFactory.getLogger(SwaggerJavaTransmuter.class); + private final ObjectMapper objectMapper; + private final HttpServletRequest request; + + @Value("${server.contextPath}") + protected String contextPath; + + @Autowired + protected HttpServletRequest context; + + @Autowired + ConnectionContext connection; + + @org.springframework.beans.factory.annotation.Autowired + public SwaggerJavaTransmuter(ObjectMapper objectMapper, HttpServletRequest context) + { + this.objectMapper = objectMapper; + this.request = context; + } + + @Override + public URL getBaseURL() + { + try + { + SwaggerJavaTransmuter.log.debug("contextPath is: " + this.contextPath); + URL baseURL; + + if (this.proxyRunsOnDefaultPort()) + { + baseURL = new URL(this.context.getScheme(), this.context.getServerName(), this.contextPath); + } + else + { + baseURL = new URL(this.context.getScheme(), this.context.getServerName(), this.context.getServerPort(), this.contextPath); + } + + log.debug("baseUrl is " + baseURL.toString()); + return baseURL; + + } + catch (MalformedURLException e) + { + log.error("Server URL was not retrieved"); + return null; + } + } + + @Override + public ConnectionContext getConnection() { return this.connection; } + + @Override + public ObjectMapper getObjectMapper() + { return this.objectMapper; } + + @Override + public ResponseEntity groupList( + String group, + @Valid List fields, + @Valid List keywords, + @Min(0) @Valid Integer limit, + @Valid String q, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new Standard(), new URIParameters() + .setGroup(group) + .setFields(fields) + .setKeywords(keywords) + .setLimit(limit) + .setQuery(q) + .setSort(sort) + .setStart(start)); + } + + @Override + public ResponseEntity groupReferencingId( + String group, + String identifier, + @Valid List fields, + @Min(0) @Valid Integer limit, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new GroupReferencingId(), new URIParameters() + .setGroup(group) + .setIdentifier(identifier) + .setFields(fields) + .setLimit(limit) + .setSort(sort) + .setStart(start)); + } + + @Override + public ResponseEntity groupReferencingIdVers( + String group, + String identifier, + String versions, + @Valid List fields, + @Min(0) @Valid Integer limit, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new GroupReferencingId(), new URIParameters() + .setGroup(group) + .setIdentifier(identifier) + .setVersion(versions) + .setFields(fields) + .setLimit(limit) + .setSort(sort) + .setStart(start)); + } + + @Override + public ResponseEntity idReferencingGroup( + String group, + String identifier, + @Valid List fields, + @Min(0) @Valid Integer limit, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new IdReferencingGroup(), new URIParameters() + .setGroup(group) + .setIdentifier(identifier) + .setFields(fields) + .setLimit(limit) + .setSort(sort) + .setStart(start)); + } + + @Override + public ResponseEntity idReferencingGroupVers( + String group, + String identifier, + String versions, + @Valid List fields, + @Min(0) @Valid Integer limit, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new IdReferencingGroup(), new URIParameters() + .setGroup(group) + .setIdentifier(identifier) + .setVersion(versions) + .setFields(fields) + .setLimit(limit) + .setSort(sort) + .setStart(start)); + } + + private ResponseEntity processs (EndpointHandler handler, URIParameters parameters) + { + try { return handler.transmute(this, parameters.setAccept(this.request.getHeader("Accept")).setLidVid(this)); } + catch (ApplicationTypeException e) + { + log.error("Application type not implemented", e); + return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); + } + catch (IOException e) + { + log.error("Couldn't get or serialize response for content type " + this.request.getHeader("Accept"), e); + return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.INTERNAL_SERVER_ERROR); + } + catch (LidVidNotFoundException e) + { + log.warn("Could not find lid(vid) in database: " + parameters.getIdentifier()); + return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); + } + catch (NothingFoundException e) + { + log.warn("Could not find any matching reference(s) in database."); + return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); + } + catch (UnknownGroupNameException e) + { + log.error("Group name not implemented", e); + return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); + } + } + + private boolean proxyRunsOnDefaultPort() + { + return (("https".equals(this.context.getScheme()) && (this.context.getServerPort() == 443)) + || ("http".equals(this.context.getScheme()) && (this.context.getServerPort() == 80))); + } + @Override + public ResponseEntity selectByLidvid(String identifier, @Valid List fields) + { + return this.processs(new Standard(), new URIParameters() + .setIdentifier(identifier) + .setFields(fields)); + } + + @Override + public ResponseEntity selectByLidvidAll( + String identifier, + @Valid List fields, + @Min(0)@Valid Integer limit, + @Valid List sort, + @Min(0) @Valid Integer start) + { + return this.processs(new Standard(), new URIParameters() + .setIdentifier(identifier) + .setFields(fields) + .setLimit(limit) + .setSort(sort) + .setStart(start) + .setVersion("all")); + } + + @Override + public ResponseEntity selectByLidvidLatest(String identifier, @Valid List fields) + { + return this.processs(new Standard(), new URIParameters() + .setIdentifier(identifier) + .setFields(fields) + .setVersion("latest")); + } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controllers/URIParameters.java b/service/src/main/java/gov/nasa/pds/api/registry/controller/URIParameters.java similarity index 66% rename from service/src/main/java/gov/nasa/pds/api/registry/controllers/URIParameters.java rename to service/src/main/java/gov/nasa/pds/api/registry/controller/URIParameters.java index 1104b414..706fff41 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/controllers/URIParameters.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/controller/URIParameters.java @@ -1,10 +1,15 @@ -package gov.nasa.pds.api.registry.controllers; +package gov.nasa.pds.api.registry.controller; +import java.io.IOException; import java.util.ArrayList; import java.util.List; +import gov.nasa.pds.api.registry.ControlContext; import gov.nasa.pds.api.registry.UserContext; -import gov.nasa.pds.api.registry.business.ProductVersionSelector; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.model.LidVidUtils; +import gov.nasa.pds.api.registry.model.ProductVersionSelector; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; /* * Maybe not the most obvious properties class or bean or whatever name but @@ -25,25 +30,34 @@ */ class URIParameters implements UserContext { + private String accept = "applicaation/json"; private List fields = new ArrayList(); + private String group = ""; private String identifier = ""; private List keywords = new ArrayList(); + private String lidvid = ""; private Integer limit = Integer.valueOf(0); private String query = ""; private ProductVersionSelector selector = ProductVersionSelector.LATEST; private List sort = new ArrayList(); private Integer start = Integer.valueOf(-1); - private Boolean summanryOnly = Boolean.valueOf(false); - + private String version = "latest"; + + @Override + public String getAccept() { return accept; } @Override public List getFields() { return fields; } @Override + public String getGroup() { return group; } + @Override public String getIdentifier() { return identifier; } @Override public List getKeywords() { return keywords; } @Override public Integer getLimit() { return limit; } @Override + public String getLidVid() { return lidvid; } + @Override public String getQuery() { return query; } @Override public ProductVersionSelector getSelector() { return selector; } @@ -52,59 +66,67 @@ class URIParameters implements UserContext @Override public Integer getStart() { return start; } @Override - public Boolean getSummanryOnly() { return summanryOnly; } + public String getVersion() { return version; } + public URIParameters setAccept(String accept) + { + if (accept != null) this.accept = accept; + return this; + } public URIParameters setFields(List fields) { if (fields != null) this.fields = fields; return this; } - + public URIParameters setGroup(String group) + { + if (group != null) this.group = group; + return this; + } public URIParameters setIdentifier(String identifier) { if (identifier != null) this.identifier = identifier; return this; } - public URIParameters setKeywords(List keywords) { if (keywords != null) this.keywords = keywords; return this; } - public URIParameters setLimit(Integer limit) { if (limit != null) this.limit = limit; return this; } - - public URIParameters setQuery(String query) + public URIParameters setLidVid(ControlContext control) throws IOException, LidVidNotFoundException { - if (query != null) this.query = query; + this.lidvid = LidVidUtils.resolve(this.getIdentifier(), ProductVersionSelector.TYPED, + control, RequestBuildContextFactory.empty()); return this; } - - public URIParameters setSelector(ProductVersionSelector selector) + public URIParameters setQuery(String query) { - if (selector != null) this.selector = selector; + if (query != null) this.query = query; return this; } - public URIParameters setSort(List sort) { if (sort != null) this.sort = sort; return this; } - public URIParameters setStart(Integer start) { if (start != null) this.start = start; return this; } - - public URIParameters setSummanryOnly(Boolean summanryOnly) + public URIParameters setVersion(String version) { - if (summanryOnly != null) this.summanryOnly = summanryOnly; + if (version != null) + { + this.version = version; + if ("all".equalsIgnoreCase(version)) this.selector = ProductVersionSelector.ALL; + else this.selector = ProductVersionSelector.LATEST; + } return this; } } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyBundlesApiController.java b/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyBundlesApiController.java deleted file mode 100644 index 7b367093..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyBundlesApiController.java +++ /dev/null @@ -1,334 +0,0 @@ -package gov.nasa.pds.api.registry.controllers; - - -import gov.nasa.pds.api.base.BundlesApi; -import gov.nasa.pds.api.registry.business.BundleDAO; -import gov.nasa.pds.api.registry.business.CollectionDAO; -import gov.nasa.pds.api.registry.business.ErrorFactory; -import gov.nasa.pds.api.registry.business.ProductDAO; -import gov.nasa.pds.api.registry.business.ProductVersionSelector; -import gov.nasa.pds.api.registry.business.RequestAndResponseContext; -import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; -import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.exceptions.NothingFoundException; -import gov.nasa.pds.api.registry.opensearch.HitIterator; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; - -import com.fasterxml.jackson.databind.ObjectMapper; -import io.swagger.annotations.*; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; -import org.springframework.stereotype.Controller; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestParam; - -import javax.validation.Valid; -import javax.servlet.http.HttpServletRequest; -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - - -@javax.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2021-02-16T16:35:42.434-08:00[America/Los_Angeles]") -@Controller -public class MyBundlesApiController extends MyProductsApiBareController implements BundlesApi { - - private static final Logger log = LoggerFactory.getLogger(MyBundlesApiController.class); - - - @org.springframework.beans.factory.annotation.Autowired - public MyBundlesApiController(ObjectMapper objectMapper, HttpServletRequest request) - { super(objectMapper, request); } - - @Override - public ResponseEntity bundleByLidvid( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields - ) - { - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - BundleDAO.searchConstraints()); - } - - - @Override - public ResponseEntity bundleByLidvidLatest( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields) - { - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - BundleDAO.searchConstraints()); - } - - - @Override - public ResponseEntity bundleByLidvidAll( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return this.getAllProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSelector(ProductVersionSelector.ALL) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - BundleDAO.searchConstraints()); - } - - - @Override - public ResponseEntity getBundles( - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: keyword=keyword1,keyword2,... behaviro: free text search on title and description (if set q is ignored notes: is this implemented? ") @Valid @RequestParam(value = "keyword", required = false) List keywords, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: q=\"vid eq 13.0\" behaviro: query uses eq,ne,gt,ge,lt,le,(,),not,and,or operators. Properties are named as in 'properties' attributes, literals are strings between quotes, like \"animal\", or numbers. Detailed query specification is available at https://bit.ly/3h3D54T note: ignored when keyword is present ") @Valid @RequestParam(value = "q", required = false) String q, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return this.getProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setKeywords(keywords) - .setLimit(limit) - .setQuery(q) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - BundleDAO.searchConstraints()); - } - - - @Override - public ResponseEntity collectionsOfABundle( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getBundlesCollectionsEntity(new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - - @Override - public ResponseEntity collectionsOfABundleAll( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getBundlesCollectionsEntity(new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSelector(ProductVersionSelector.ALL) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - - @Override - public ResponseEntity collectionsOfABundleLatest( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getBundlesCollectionsEntity(new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - private void getBundleCollections(RequestAndResponseContext context) - throws IOException, LidVidNotFoundException - { - MyBundlesApiController.log.info("Get bundle's collections. Bundle LIDVID = " + context.getLIDVID()); - - List clidvids = null; - if(context.getSelector() == ProductVersionSelector.ALL) - { clidvids = BundleDAO.getAllBundleCollectionLidVids(context.getLIDVID(), this, context); } - else - { clidvids = BundleDAO.getBundleCollectionLidVids(context.getLIDVID(), this, context); } - - int size = clidvids.size(); - if (size > 0 && context.getStart() < size && context.getLimit() > 0) - { - int end = context.getStart() + context.getLimit(); - if(end > size) end = size; - - List ids = clidvids.subList(context.getStart(), end); - this.fillProductsFromLidvids(context, - RequestBuildContextFactory.given(context.getFields(), CollectionDAO.searchConstraints()), - ids, -1); - } - else - { - log.warn("Did not find any collections for bundle lidvid: " + context.getLIDVID()); - } - } - - - private ResponseEntity getBundlesCollectionsEntity(URIParameters parameters) - { - String accept = this.request.getHeader("Accept"); - MyBundlesApiController.log.info("accept value is " + accept); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, CollectionDAO.searchConstraints(), BundleDAO.searchConstraints(), accept); - this.getBundleCollections(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + parameters.getIdentifier()); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - - @Override - public ResponseEntity productsOfABundle( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fileds=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - String accept = this.request.getHeader("Accept"); - MyBundlesApiController.log.info("accept value is " + accept); - URIParameters parameters = new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, ProductDAO.searchConstraints(), BundleDAO.searchConstraints(), accept); - this.getProductChildren(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + identifier); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - - private void getProductChildren(RequestAndResponseContext context) throws IOException,LidVidNotFoundException - { - MyBundlesApiController.log.info("request bundle lidvid, children of products: " + context.getLIDVID()); - - int iteration=0; - List clidvids = BundleDAO.getBundleCollectionLidVids(context.getLIDVID(), this, context); - - List plidvids = new ArrayList(); - List wlidvids = new ArrayList(); - - if (0 < clidvids.size()) - { - for (final Map hit : - new HitIterator(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory (RequestConstructionContextFactory.given ("collection_lidvid", clidvids, false), this.connectionContext) - .build(RequestBuildContextFactory.given ("product_lidvid"), this.connectionContext.getRegistryRefIndex()))) - { - wlidvids.clear(); - - if (hit.get("product_lidvid") instanceof String) - { wlidvids.add(hit.get("product_lidvid").toString()); } - else - { - @SuppressWarnings("unchecked") - List plids = (List)hit.get("product_lidvid"); - wlidvids.addAll(plids); - } - - if ((context.getStart() <= iteration || context.getStart() < iteration+wlidvids.size()) && plidvids.size() <= context.getLimit()) - { plidvids.addAll(wlidvids.subList(context.getStart() <= iteration ? 0 : context.getStart()-iteration, wlidvids.size())); } - - iteration = iteration + wlidvids.size(); - } - } - else MyBundlesApiController.log.warn ("Did not find any collections for bundle lidvid: " + context.getLIDVID()); - - if (plidvids.size() > 0 && context.getLimit() > 0) - { - this.fillProductsFromLidvids(context, RequestBuildContextFactory.given(context.getFields()), - plidvids.subList(0, plidvids.size() < context.getLimit() ? plidvids.size() : context.getLimit()), iteration); - } - else MyBundlesApiController.log.warn ("Did not find any products for bundle lidvid: " + context.getLIDVID()); - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyCollectionsApiController.java b/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyCollectionsApiController.java deleted file mode 100644 index e71a7ffb..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyCollectionsApiController.java +++ /dev/null @@ -1,323 +0,0 @@ -package gov.nasa.pds.api.registry.controllers; - - -import gov.nasa.pds.api.base.CollectionsApi; -import gov.nasa.pds.api.registry.business.BundleDAO; -import gov.nasa.pds.api.registry.business.CollectionDAO; -import gov.nasa.pds.api.registry.business.ErrorFactory; -import gov.nasa.pds.api.registry.business.LidVidUtils; -import gov.nasa.pds.api.registry.business.ProductDAO; -import gov.nasa.pds.api.registry.business.ProductVersionSelector; -import gov.nasa.pds.api.registry.business.RequestAndResponseContext; -import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; -import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.exceptions.NothingFoundException; -import gov.nasa.pds.api.registry.opensearch.HitIterator; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; - -import com.fasterxml.jackson.databind.ObjectMapper; -import io.swagger.annotations.*; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; -import org.springframework.stereotype.Controller; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestParam; - -import javax.validation.Valid; -import javax.servlet.http.HttpServletRequest; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - - - -@Controller -public class MyCollectionsApiController extends MyProductsApiBareController implements CollectionsApi { - - private static final Logger log = LoggerFactory.getLogger(MyCollectionsApiController.class); - - - public MyCollectionsApiController(ObjectMapper objectMapper, HttpServletRequest request) - { super(objectMapper, request); } - - - @Override - public ResponseEntity collectionsByLidvid(String identifier, - List fields - ) - { - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - CollectionDAO.searchConstraints()); - } - - @Override - public ResponseEntity collectionsByLidvidLatest( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields - ) - { - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - CollectionDAO.searchConstraints()); - } - - - @Override - public ResponseEntity collectionsByLidvidAll( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getAllProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSelector(ProductVersionSelector.ALL) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - CollectionDAO.searchConstraints()); - } - - - @Override - public ResponseEntity getCollection( - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: keyword=keyword1,keyword2,... behaviro: free text search on title and description (if set q is ignored notes: is this implemented? ") @Valid @RequestParam(value = "keyword", required = false) List keywords, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: q=\"vid eq 13.0\" behaviro: query uses eq,ne,gt,ge,lt,le,(,),not,and,or operators. Properties are named as in 'properties' attributes, literals are strings between quotes, like \"animal\", or numbers. Detailed query specification is available at https://bit.ly/3h3D54T note: ignored when keyword is present ") @Valid @RequestParam(value = "q", required = false) String q, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return this.getProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setKeywords(keywords) - .setLimit(limit) - .setQuery(q) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - CollectionDAO.searchConstraints()); - } - - - @Override - public ResponseEntity productsOfACollection( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getProductsOfACollectionResponseEntity( - new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - - @Override - public ResponseEntity productsOfACollectionLatest( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getProductsOfACollectionResponseEntity(new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - - @Override - public ResponseEntity productsOfACollectionAll( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return getProductsOfACollectionResponseEntity(new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSelector(ProductVersionSelector.ALL) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly)); - } - - - protected ResponseEntity getProductsOfACollectionResponseEntity(URIParameters parameters) - { - String accept = this.request.getHeader("Accept"); - MyCollectionsApiController.log.info("Get productsOfACollection"); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, ProductDAO.searchConstraints(), CollectionDAO.searchConstraints(), accept); - this.getProductChildren(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (LidVidNotFoundException e) - { - log.error("Couldn't find the lidvid " + e.getMessage()); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - - private void getProductChildren(RequestAndResponseContext context) throws IOException, LidVidNotFoundException - { - // FIXME: this does not do latest/all correctly but need a dataset to test with - MyCollectionsApiController.log.info("request collection lidvid, collections children: " + context.getLIDVID()); - - int iteration=0,wsize=0; - List productLidvids = new ArrayList(); - List pageOfLidvids = new ArrayList(); - - for (final Map kvp : new HitIterator(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(RequestConstructionContextFactory.given ("collection_lidvid", context.getLIDVID(), true), this.connectionContext) - .build (RequestBuildContextFactory.given ("product_lidvid"), this.connectionContext.getRegistryRefIndex()))) - { - pageOfLidvids.clear(); - wsize = 0; - - if (kvp.get("product_lidvid") instanceof String) - { pageOfLidvids.add(LidVidUtils.resolveLIDVID (kvp.get("product_lidvid").toString(), ProductVersionSelector.LATEST, this, context)); } - else - { - @SuppressWarnings("unchecked") - List clids = (List)kvp.get("product_lidvid"); - - // if we are working with data that we care about (between start and start + limit) then record them - if (context.getStart() <= iteration || context.getStart() < iteration+clids.size()) {pageOfLidvids.addAll(clids); } - // else just modify the counter to skip them without wasting CPU cycles processing them - else { wsize = clids.size(); } - } - - // if any data from the pages then add them to the complete roster - if (context.getStart() <= iteration || context.getStart() < iteration+pageOfLidvids.size()) - { productLidvids.addAll(pageOfLidvids.subList(context.getStart() <= iteration ? 0 : context.getStart()-iteration, pageOfLidvids.size())); } - - // if the limit of data has been found then break out of the loop - //if (limit <= productLidvids.size()) { break; } - // otherwise update all of hte indices for the next iteration - //else { iteration = iteration + pageOfLidvids.size() + wsize; } - iteration = iteration + pageOfLidvids.size() + wsize; - } - - if (productLidvids.size() > 0 && context.getLimit() > 0) - { - this.fillProductsFromLidvids(context, RequestBuildContextFactory.given(context.getFields()), - productLidvids.subList(0, productLidvids.size() < context.getLimit() ? productLidvids.size() : context.getLimit()), iteration); - } - else MyCollectionsApiController.log.warn("Did not find any products for collection lidvid: " + context.getLIDVID()); - } - - - @Override - public ResponseEntity bundlesContainingCollection( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - String accept = this.request.getHeader("Accept"); - MyCollectionsApiController.log.info("accept value is " + accept); - URIParameters parameters = new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly); - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, BundleDAO.searchConstraints(), CollectionDAO.searchConstraints(), accept); - this.getContainingBundle(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + identifier); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - private void getContainingBundle(RequestAndResponseContext context) throws IOException,LidVidNotFoundException - { - MyCollectionsApiController.log.info("find all bundles containing the collection lidvid: " + context.getLIDVID()); - MyCollectionsApiController.log.info("find all bundles containing the collection lid: " + LidVidUtils.extractLidFromLidVid(context.getLIDVID())); - - context.setResponse(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(RequestConstructionContextFactory.given("ref_lid_collection", - LidVidUtils.extractLidFromLidVid(context.getLIDVID()), true), this.connectionContext) - .build(context, this.connectionContext.getRegistryIndex())); - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiBareController.java b/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiBareController.java deleted file mode 100644 index 1af1ed8e..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiBareController.java +++ /dev/null @@ -1,228 +0,0 @@ -package gov.nasa.pds.api.registry.controllers; - -import java.io.IOException; -import java.net.MalformedURLException; -import java.net.URL; -import java.util.List; -import java.util.Map; - -import javax.servlet.http.HttpServletRequest; - -import org.antlr.v4.runtime.misc.ParseCancellationException; -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.beans.factory.annotation.Autowired; -import org.springframework.beans.factory.annotation.Value; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; -import org.springframework.stereotype.Component; - -import com.fasterxml.jackson.databind.ObjectMapper; - -import gov.nasa.pds.api.registry.ControlContext; -import gov.nasa.pds.api.registry.ConnectionContext; -import gov.nasa.pds.api.registry.RequestBuildContext; -import gov.nasa.pds.api.registry.business.ErrorFactory; -import gov.nasa.pds.api.registry.business.RequestAndResponseContext; -import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; -import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.exceptions.NothingFoundException; -import gov.nasa.pds.api.registry.opensearch.HitIterator; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; - -@Component -public class MyProductsApiBareController implements ControlContext -{ - private static final Logger log = LoggerFactory.getLogger(MyProductsApiBareController.class); - protected final ObjectMapper objectMapper; - protected final HttpServletRequest request; - - @Value("${server.contextPath}") - protected String contextPath; - - @Autowired - protected HttpServletRequest context; - - @Autowired - ConnectionContext connectionContext; - - public MyProductsApiBareController(ObjectMapper objectMapper, HttpServletRequest context) { - this.objectMapper = objectMapper; - this.request = context; - } - - protected void fillProductsFromLidvids (RequestAndResponseContext context, RequestBuildContext buildContext, List lidvids, int real_total) throws IOException - { - context.setResponse(new HitIterator(lidvids.size(), - this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(RequestConstructionContextFactory.given("lidvid", lidvids, true), this.connectionContext) - .build(buildContext, this.connectionContext.getRegistryIndex())), - real_total); - } - - - protected void getProducts(RequestAndResponseContext context) throws IOException - { - context.setResponse(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(context, this.connectionContext).build(context, this.connectionContext.getRegistryIndex())); - } - - - protected ResponseEntity getProductsResponseEntity(URIParameters parameters, Map preset) - { - String accept = this.request.getHeader("Accept"); - log.debug("accept value is " + accept); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, preset, accept); - this.getProducts(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + parameters.getIdentifier()); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (ParseCancellationException pce) - { - log.error("Could not parse the query string: " + parameters.getQuery(), pce); - return new ResponseEntity(ErrorFactory.build(pce, this.request), HttpStatus.BAD_REQUEST); - } - } - - - protected ResponseEntity getAllProductsResponseEntity(URIParameters parameters, Map preset) - { - String accept = this.request.getHeader("Accept"); - log.debug("accept value is " + accept); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, preset, accept); - this.getProductsByLid(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + parameters.getIdentifier()); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (ParseCancellationException pce) - { - log.error("", pce); - return new ResponseEntity(ErrorFactory.build(pce, this.request), HttpStatus.BAD_REQUEST); - } - } - - - public void getProductsByLid(RequestAndResponseContext context) throws IOException - { - context.setResponse(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(context, this.connectionContext).build(context, this.connectionContext.getRegistryIndex())); - } - - - protected ResponseEntity getLatestProductResponseEntity(URIParameters parameters, Map preset) - { - String accept = request.getHeader("Accept"); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, preset, accept); - context.setResponse(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(context, this.connectionContext).build (context, this.connectionContext.getRegistryIndex())); - - if (context.getResponse() == null) - { - log.warn("Could not find any matches for LIDVID: " + context.getLIDVID()); - return new ResponseEntity(HttpStatus.NOT_FOUND); - } - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't get or serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + parameters.getIdentifier()); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - - private boolean proxyRunsOnDefaultPort() { - return (((this.context.getScheme() == "https") && (this.context.getServerPort() == 443)) - || ((this.context.getScheme() == "http") && (this.context.getServerPort() == 80))); - } - - @Override - public URL getBaseURL() { - try { - MyProductsApiBareController.log.debug("contextPath is: " + this.contextPath); - - URL baseURL; - if (this.proxyRunsOnDefaultPort()) { - baseURL = new URL(this.context.getScheme(), this.context.getServerName(), this.contextPath); - } - else { - baseURL = new URL(this.context.getScheme(), this.context.getServerName(), this.context.getServerPort(), this.contextPath); - } - - log.debug("baseUrl is " + baseURL.toString()); - return baseURL; - - } catch (MalformedURLException e) { - log.error("Server URL was not retrieved"); - return null; - } - } - - @Override - public ObjectMapper getObjectMapper() { return this.objectMapper; } - @Override - public ConnectionContext getConnection() { return this.connectionContext; } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiController.java b/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiController.java deleted file mode 100644 index 2265c6af..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/controllers/MyProductsApiController.java +++ /dev/null @@ -1,278 +0,0 @@ -package gov.nasa.pds.api.registry.controllers; - -import java.io.IOException; -import java.util.ArrayList; -import java.util.List; -import java.util.Map; - -import javax.servlet.http.HttpServletRequest; -import javax.validation.Valid; - -import org.slf4j.Logger; -import org.slf4j.LoggerFactory; -import org.springframework.http.HttpStatus; -import org.springframework.http.ResponseEntity; -import org.springframework.stereotype.Controller; -import org.springframework.web.bind.annotation.PathVariable; -import org.springframework.web.bind.annotation.RequestParam; - -import com.fasterxml.jackson.databind.ObjectMapper; - -import gov.nasa.pds.api.base.ProductsApi; -import gov.nasa.pds.api.registry.business.BundleDAO; -import gov.nasa.pds.api.registry.business.CollectionDAO; -import gov.nasa.pds.api.registry.business.ErrorFactory; -import gov.nasa.pds.api.registry.business.ProductDAO; -import gov.nasa.pds.api.registry.business.ProductVersionSelector; -import gov.nasa.pds.api.registry.business.RequestAndResponseContext; -import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; -import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.exceptions.NothingFoundException; -import gov.nasa.pds.api.registry.opensearch.HitIterator; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; -import io.swagger.annotations.ApiParam; - - -@javax.annotation.Generated(value = "io.swagger.codegen.v3.generators.java.SpringCodegen", date = "2020-10-29T11:01:11.991-07:00[America/Los_Angeles]") -@Controller -public class MyProductsApiController extends MyProductsApiBareController implements ProductsApi { - - private static final Logger log = LoggerFactory.getLogger(MyProductsApiController.class); - - @org.springframework.beans.factory.annotation.Autowired - public MyProductsApiController(ObjectMapper objectMapper, HttpServletRequest request) - { super(objectMapper, request); } - - @Override - public ResponseEntity products( - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: keyword=keyword1,keyword2,... behaviro: free text search on title and description (if set q is ignored notes: is this implemented? ") @Valid @RequestParam(value = "keyword", required = false) List keywords, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: q=\"vid eq 13.0\" behaviro: query uses eq,ne,gt,ge,lt,le,(,),not,and,or operators. Properties are named as in 'properties' attributes, literals are strings between quotes, like \"animal\", or numbers. Detailed query specification is available at https://bit.ly/3h3D54T note: ignored when keyword is present ") @Valid @RequestParam(value = "q", required = false) String q, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - return this.getProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setKeywords(keywords) - .setLimit(limit) - .setQuery(q) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - ProductDAO.searchConstraints()); - } - - @Override - public ResponseEntity productsByLidvid( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields - ) - { - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - ProductDAO.searchConstraints()); - } - - - @Override - public ResponseEntity productsByLidvidLatest( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields - ) - { - // FIXME: add fields - return this.getLatestProductResponseEntity( - new URIParameters().setFields(fields).setIdentifier(identifier), - ProductDAO.searchConstraints()); - } - - - @Override - public ResponseEntity productsByLidvidAll( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - // FIXME: add fields, sort, summary-only - return getAllProductsResponseEntity( - new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSelector(ProductVersionSelector.ALL) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly), - ProductDAO.searchConstraints()); - } - - - @Override - public ResponseEntity bundlesContainingProduct( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - String accept = this.request.getHeader("Accept"); - MyProductsApiController.log.info("accept value is " + accept); - URIParameters parameters = new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, BundleDAO.searchConstraints(), ProductDAO.searchConstraints(), accept); - this.getContainingBundle(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + identifier); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - private void getContainingBundle(RequestAndResponseContext context) throws IOException,LidVidNotFoundException - { - MyProductsApiController.log.info("find all bundles containing the product lidvid: " + context.getLIDVID()); - - List collectionLIDs = this.getCollectionLidvids(context.getLIDVID(), true); - - if (0 < collectionLIDs.size()) - { - context.setResponse(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(RequestConstructionContextFactory.given("ref_lid_collection", collectionLIDs, false), this.connectionContext) - .build(context, this.connectionContext.getRegistryIndex())); - } - else MyProductsApiController.log.warn ("No parent collection for product LIDVID: " + context.getLIDVID()); - } - - @Override - public ResponseEntity collectionsContainingProduct( - @ApiParam(value = "syntax: lidvid or lid behavior (lid): returns one or more items whose lid matches this lid exactly. If the endpoint ends with the identifier or /latest then a signle result is returned and it is the highest version. If the endpoint ends with /all then all versions of the lid are returned. behavior (lidvid): returns one and only one item whose lidvid matches this lidvid exactly. note: the current lid/lidvid resolution will match all the lids that start with lid. In other words, it acts like a glob of foobar*. It behaves this way from first character to the last note: simple sorting of the lidvid is being done to select the latest from the end of the list. However, the versions 1.0, 2.0, and 13.0 will sort to 1.0, 13.0, and 2.0 so the end of the list may not be the latest. ",required=true) @PathVariable("identifier") String identifier, - @ApiParam(value = "syntax: fields=field1,field2,... behavior: this parameter and the headder Accept: type determine what content is packaged for the result. While the types application/csv, application/kvp+json, and text/csv return only the fields requesteted, all of the other types have a minimal set of fields that must be returned. Duplicating a minimally required field in this parameter has not effect. The types vnd.nasa.pds.pds4+json and vnd.nasa.pds.pds4+xml have a complete set of fields that must be returned; meaning this parameter does not impact their content. When fields is not used, then the minimal set of fields, or all when minimal is an empty set, is returned. notes: the blob fields are blocked unless specifically requrested and only for the *_/csv and application/kvp+csv types. ") @Valid @RequestParam(value = "fields", required = false) List fields, - @ApiParam(value = "syntax: limit=10 behavior: maximum number of matching results returned, for pagination ", defaultValue = "100") @Valid @RequestParam(value = "limit", required = false, defaultValue="100") Integer limit, - @ApiParam(value = "syntax: sort=asc(field0),desc(field1),... behavior: is this implemented? ") @Valid @RequestParam(value = "sort", required = false) List sort, - @ApiParam(value = "syntax: start=12 behaviro: offset in matching result list, for pagination ", defaultValue = "0") @Valid @RequestParam(value = "start", required = false, defaultValue="0") Integer start, - @ApiParam(value = "syntax: summary-only={true,false} behavior: only return the summary when a list is returned. Useful to get the list of available properties. ", defaultValue = "false") @Valid @RequestParam(value = "summary-only", required = false, defaultValue="false") Boolean summaryOnly - ) - { - String accept = this.request.getHeader("Accept"); - MyProductsApiController.log.info("accept value is " + accept); - URIParameters parameters = new URIParameters() - .setFields(fields) - .setIdentifier(identifier) - .setLimit(limit) - .setSort(sort) - .setStart(start) - .setSummanryOnly(summaryOnly); - - try - { - RequestAndResponseContext context = RequestAndResponseContext.buildRequestAndResponseContext(this, parameters, CollectionDAO.searchConstraints(), ProductDAO.searchConstraints(), accept); - this.getContainingCollection(context); - return new ResponseEntity(context.getResponse(), HttpStatus.OK); - } - catch (ApplicationTypeException e) - { - log.error("Application type not implemented", e); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_ACCEPTABLE); - } - catch (IOException e) - { - log.error("Couldn't serialize response for content type " + accept, e); - return new ResponseEntity(ErrorFactory.build(this.request), HttpStatus.INTERNAL_SERVER_ERROR); - } - catch (LidVidNotFoundException e) - { - log.warn("Could not find lid(vid) in database: " + identifier); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - catch (NothingFoundException e) - { - log.warn("Could not find any matching reference(s) in database."); - return new ResponseEntity(ErrorFactory.build(e, this.request), HttpStatus.NOT_FOUND); - } - } - - - private List getCollectionLidvids (String lidvid, boolean noVer) throws IOException - { - List fields = new ArrayList(), lidvids = new ArrayList(); - String field = noVer ? "collection_lid" : "collection_lidvid"; - fields.add(field); - - for (final Map kvp : new HitIterator(this.connectionContext.getRestHighLevelClient(), - new SearchRequestFactory(RequestConstructionContextFactory.given("product_lidvid", lidvid, true), this.connectionContext) - .build (RequestBuildContextFactory.given(fields), this.connectionContext.getRegistryRefIndex()))) - { - if (kvp.get(field) instanceof String) - { lidvids.add(kvp.get(field).toString()); } - else - { - @SuppressWarnings("unchecked") - List clids = (List)kvp.get(field); - for (String clid : clids) { lidvids.add(clid); } - } - } - - return lidvids; - } - - - private void getContainingCollection(RequestAndResponseContext context) throws IOException,LidVidNotFoundException - { - MyProductsApiController.log.info("find all bundles containing the product lidvid: " + context.getLIDVID()); - - List collectionLidvids = this.getCollectionLidvids(context.getLIDVID(), false); - - int size = collectionLidvids.size(); - if (size > 0 && context.getLimit() > 0 && context.getStart() < size) - { - int end = context.getStart() + context.getLimit(); - if(end > size) end = size; - List ids = collectionLidvids.subList(context.getStart(), end); - - this.fillProductsFromLidvids(context, - RequestBuildContextFactory.given(context.getFields(), CollectionDAO.searchConstraints()), - ids, collectionLidvids.size()); - } - else - { - MyProductsApiController.log.warn("Did not find a product with lidvid: " + context.getLIDVID()); - } - } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/exceptions/UnknownGroupNameException.java b/service/src/main/java/gov/nasa/pds/api/registry/exceptions/UnknownGroupNameException.java new file mode 100644 index 00000000..26da7388 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/exceptions/UnknownGroupNameException.java @@ -0,0 +1,10 @@ +package gov.nasa.pds.api.registry.exceptions; + +import java.util.Set; + +public class UnknownGroupNameException extends Exception +{ + private static final long serialVersionUID = -5630215762959235121L; + public UnknownGroupNameException (String group, Setknown) + { super("Unknown group '" + group + "'. All known groups: " + String.join(", ", known)); } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/Antlr4SearchListener.java b/service/src/main/java/gov/nasa/pds/api/registry/model/Antlr4SearchListener.java similarity index 99% rename from service/src/main/java/gov/nasa/pds/api/registry/business/Antlr4SearchListener.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/Antlr4SearchListener.java index 43f51e84..4f48b363 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/Antlr4SearchListener.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/Antlr4SearchListener.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import org.slf4j.Logger; import org.slf4j.LoggerFactory; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/BlobUtil.java b/service/src/main/java/gov/nasa/pds/api/registry/model/BlobUtil.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/business/BlobUtil.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/BlobUtil.java index 94e0c8af..8dbf80a7 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/BlobUtil.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/BlobUtil.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.io.ByteArrayInputStream; import java.io.ByteArrayOutputStream; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/EntityProduct.java b/service/src/main/java/gov/nasa/pds/api/registry/model/EntityProduct.java similarity index 98% rename from service/src/main/java/gov/nasa/pds/api/registry/business/EntityProduct.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/EntityProduct.java index 146d7602..cb729365 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/EntityProduct.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/EntityProduct.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import com.fasterxml.jackson.annotation.JsonFormat; import com.fasterxml.jackson.annotation.JsonProperty; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ErrorFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ErrorFactory.java similarity index 93% rename from service/src/main/java/gov/nasa/pds/api/registry/business/ErrorFactory.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/ErrorFactory.java index 45a55c54..206e4478 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ErrorFactory.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ErrorFactory.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import javax.servlet.http.HttpServletRequest; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/LidVidUtils.java b/service/src/main/java/gov/nasa/pds/api/registry/model/LidVidUtils.java similarity index 84% rename from service/src/main/java/gov/nasa/pds/api/registry/business/LidVidUtils.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/LidVidUtils.java index 8fe39849..539adff5 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/LidVidUtils.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/LidVidUtils.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.io.IOException; import java.util.ArrayList; @@ -17,9 +17,9 @@ import gov.nasa.pds.api.registry.ControlContext; import gov.nasa.pds.api.registry.RequestBuildContext; import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; -import gov.nasa.pds.api.registry.opensearch.RequestConstructionContextFactory; -import gov.nasa.pds.api.registry.opensearch.SearchRequestFactory; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.search.RequestConstructionContextFactory; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; /** @@ -30,7 +30,7 @@ public class LidVidUtils { private static final String LIDVID_SEPARATOR = "::"; private static final Logger log = LoggerFactory.getLogger(LidVidUtils.class); - + public static String extractLidFromLidVid(String identifier) { if (identifier == null) return null; @@ -46,7 +46,7 @@ public static List getLatestLidVidsByLids( RequestBuildContext reqContext, Collection lids) throws IOException,LidVidNotFoundException { - List lidvids = new ArrayList(lids.size()); + List lidvids = new ArrayList(); for (String lid : lids) { @@ -93,14 +93,17 @@ public static List getAllLidVidsByLids( { List lidvids = new ArrayList(); - ctlContext.getConnection().getRestHighLevelClient().search( - new SearchRequestFactory(RequestConstructionContextFactory.given("lid", new ArrayList(lids), true), ctlContext.getConnection()) - .build(reqContext, ctlContext.getConnection().getRegistryIndex()), RequestOptions.DEFAULT) - .getHits().forEach((hit) -> { lidvids.add(hit.getId()); }); + if (0 < lids.size()) + { + ctlContext.getConnection().getRestHighLevelClient().search( + new SearchRequestFactory(RequestConstructionContextFactory.given("lid", new ArrayList(lids), true), ctlContext.getConnection()) + .build(reqContext, ctlContext.getConnection().getRegistryIndex()), RequestOptions.DEFAULT) + .getHits().forEach((hit) -> { lidvids.add(hit.getId()); }); + } return lidvids; } - - public static String resolveLIDVID ( + + public static String resolve ( String identifier, ProductVersionSelector scope, ControlContext ctlContext, diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/Pagination.java b/service/src/main/java/gov/nasa/pds/api/registry/model/Pagination.java new file mode 100644 index 00000000..bb66090d --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/Pagination.java @@ -0,0 +1,12 @@ +package gov.nasa.pds.api.registry.model; + +import java.util.List; + +public interface Pagination +{ + public int limit(); // maximum number of items T in a page + public List page(); // actual page of items T + public int size(); // number of items T in List returned by page() + public int start(); // first index of page is this index in all possible items T over all pages + public int total(); // total number of items T over all pages +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/PaginationLidvidBuilder.java b/service/src/main/java/gov/nasa/pds/api/registry/model/PaginationLidvidBuilder.java new file mode 100644 index 00000000..3c7ca508 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/PaginationLidvidBuilder.java @@ -0,0 +1,63 @@ +package gov.nasa.pds.api.registry.model; + +import java.util.ArrayList; +import java.util.Arrays; +import java.util.List; + +import gov.nasa.pds.api.registry.LidvidsContext; + +class PaginationLidvidBuilder implements Pagination +{ + final private int limit; + final private int start; + final private List page = new ArrayList(); + + private int total = 0; + + PaginationLidvidBuilder (LidvidsContext bounds) + { + this.limit = bounds.getLimit(); + this.start = bounds.getStart(); + } + + void addAll (List data) + { + int remaining = this.limit - this.page.size(); + int skip = this.start - this.total; + + if (0 < remaining && this.start <= this.total + data.size() - 1) + { + if (this.start <= this.total && data.size() <= remaining) page.addAll(data); + else if (this.start <= this.total) page.addAll(data.subList(0, remaining)); + else if (data.size() - skip <= remaining) page.addAll(data.subList(skip, data.size())); + else page.addAll(data.subList(skip, remaining)); + } + this.total += data.size(); + } + + void add (Object sourceMapValue) { this.addAll(this.convert(sourceMapValue)); } + + List convert (Object sourceMapValue) + { + @SuppressWarnings("unchecked") + List values = sourceMapValue instanceof List ? (List)sourceMapValue : + (sourceMapValue == null ? new ArrayList(): Arrays.asList((String)sourceMapValue)); + return values; + } + + @Override + public int limit() { return this.limit; } + + @Override + public List page() { return this.page; } + + @Override + public int size() { return this.page.size(); } + + @Override + public int start() { return this.start; } + + @Override + public int total() { return this.total; } + +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductBusinessObject.java b/service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductBusinessObject.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductBusinessObject.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductBusinessObject.java index 40c62d0b..578b8f8d 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductBusinessObject.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductBusinessObject.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.net.URL; import java.util.ArrayList; @@ -12,7 +12,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; -import gov.nasa.pds.api.registry.opensearch.HitIterator; +import gov.nasa.pds.api.registry.search.HitIterator; import gov.nasa.pds.model.Pds4Product; import gov.nasa.pds.model.Pds4Products; import gov.nasa.pds.model.Summary; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductFactory.java similarity index 99% rename from service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductFactory.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductFactory.java index 97111172..9ad12cf2 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/Pds4ProductFactory.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/Pds4ProductFactory.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.util.ArrayList; import java.util.List; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/PdsProductBusinessObject.java b/service/src/main/java/gov/nasa/pds/api/registry/model/PdsProductBusinessObject.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/business/PdsProductBusinessObject.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/PdsProductBusinessObject.java index f14782b2..46ff5980 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/PdsProductBusinessObject.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/PdsProductBusinessObject.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.net.URL; import java.util.ArrayList; @@ -12,7 +12,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; -import gov.nasa.pds.api.registry.opensearch.HitIterator; +import gov.nasa.pds.api.registry.search.HitIterator; import gov.nasa.pds.model.PdsProduct; import gov.nasa.pds.model.PdsProducts; import gov.nasa.pds.model.PropertyArrayValues; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessLogic.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessLogic.java similarity index 87% rename from service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessLogic.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessLogic.java index ea7803b1..9942ac3c 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessLogic.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessLogic.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.net.URL; import java.util.List; @@ -8,7 +8,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; -import gov.nasa.pds.api.registry.opensearch.HitIterator; +import gov.nasa.pds.api.registry.search.HitIterator; import gov.nasa.pds.model.Summary; public interface ProductBusinessLogic diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessObject.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessObject.java similarity index 74% rename from service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessObject.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessObject.java index 88b3f023..9114350c 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductBusinessObject.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductBusinessObject.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.util.ArrayList; import java.util.HashMap; @@ -52,51 +52,44 @@ public static Map getFilteredProperties( Map filteredMapJsonProperties = new HashMap(); - if ((included_fields == null) || (included_fields.size() ==0)) { - + if ((included_fields == null) || (included_fields.size() == 0)) + { String apiProperty; - for (Map.Entry entry : sourceAsMap.entrySet()) { - try { + for (Map.Entry entry : sourceAsMap.entrySet()) + { + try + { apiProperty = SearchUtil.openPropertyToJsonProperty(entry.getKey()); - if ((excluded_fields == null) - || !excluded_fields.contains(apiProperty)) - filteredMapJsonProperties.put( + if ((excluded_fields == null) || !excluded_fields.contains(apiProperty)) + filteredMapJsonProperties.put( apiProperty, - ProductBusinessObject.object2PropertyValue(entry.getValue()) - ); - } catch (UnsupportedSearchProperty e) { - log.warn("openSearch property " + entry.getKey() + " is not supported, ignored"); + ProductBusinessObject.object2PropertyValue(entry.getValue())); } + catch (UnsupportedSearchProperty e) + { log.warn("openSearch property " + entry.getKey() + " is not supported, ignored"); } } - } - else { - + else + { String esField; - for (String field : included_fields) { - + for (String field : included_fields) + { esField = SearchUtil.jsonPropertyToOpenProperty(field); - - if (sourceAsMap.containsKey(esField)) { + + if (sourceAsMap.containsKey(esField)) + { filteredMapJsonProperties.put( field, - ProductBusinessObject.object2PropertyValue(sourceAsMap.get(esField)) - ); + ProductBusinessObject.object2PropertyValue(sourceAsMap.get(esField))); } - else { + else + { filteredMapJsonProperties.put( field, - ProductBusinessObject.object2PropertyValue(ProductBusinessObject.DEFAULT_NULL_VALUE) - ); + ProductBusinessObject.object2PropertyValue(ProductBusinessObject.DEFAULT_NULL_VALUE)); } - } - } - - return filteredMapJsonProperties; - - } } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductQueryBuilderUtil.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductQueryBuilderUtil.java similarity index 87% rename from service/src/main/java/gov/nasa/pds/api/registry/business/ProductQueryBuilderUtil.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/ProductQueryBuilderUtil.java index a81f740b..a0933af0 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductQueryBuilderUtil.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductQueryBuilderUtil.java @@ -1,8 +1,7 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.util.ArrayList; import java.util.List; -import java.util.Map; import java.util.StringTokenizer; import javax.annotation.PostConstruct; @@ -23,6 +22,7 @@ import org.springframework.beans.factory.annotation.Value; import org.springframework.stereotype.Component; +import gov.nasa.pds.api.registry.GroupConstraint; import gov.nasa.pds.api.registry.lexer.SearchLexer; import gov.nasa.pds.api.registry.lexer.SearchParser; @@ -65,7 +65,7 @@ public void init() * @param presetCriteria preset criteria * @return a query */ - public static QueryBuilder createPqlQuery(String queryString, List fields, Map presetCriteria) + public static QueryBuilder createPqlQuery(String queryString, List fields, GroupConstraint presetCriteria) { BoolQueryBuilder boolQuery = QueryBuilders.boolQuery(); @@ -99,13 +99,21 @@ public static void addArchiveStatusFilter(BoolQueryBuilder boolQuery) } - public static void addPresetCriteria(BoolQueryBuilder boolQuery, Map presetCriteria) + public static void addPresetCriteria(BoolQueryBuilder boolQuery, GroupConstraint presetCriteria) { if(presetCriteria != null) { - presetCriteria.forEach((key, value) -> + presetCriteria.all().forEach((key, list) -> { - boolQuery.filter(QueryBuilders.termQuery(key, value)); + list.forEach(value -> { boolQuery.must(QueryBuilders.termQuery(key, value)); }); + }); + presetCriteria.any().forEach((key, list) -> + { + list.forEach(value -> { boolQuery.filter(QueryBuilders.termQuery(key, value)); }); + }); + presetCriteria.not().forEach((key, list) -> + { + list.forEach(value -> { boolQuery.mustNot(QueryBuilders.termQuery(key, value)); }); }); } } @@ -117,7 +125,7 @@ public static void addPresetCriteria(BoolQueryBuilder boolQuery, Map presetCriteria) + public static QueryBuilder createKeywordQuery(String keyword, GroupConstraint presetCriteria) { // Lucene query QueryStringQueryBuilder luceneQuery = QueryBuilders.queryStringQuery(keyword); diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductVersionSelector.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductVersionSelector.java similarity index 92% rename from service/src/main/java/gov/nasa/pds/api/registry/business/ProductVersionSelector.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/ProductVersionSelector.java index d2c8c2fe..5d338527 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/ProductVersionSelector.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ProductVersionSelector.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; /** * Used by API calls, such as "/bundles/{lidvid}/collections", diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicAny.java b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicAny.java new file mode 100644 index 00000000..f235205b --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicAny.java @@ -0,0 +1,76 @@ +package gov.nasa.pds.api.registry.model; + +import java.io.IOException; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.ReferencingLogic; +import gov.nasa.pds.api.registry.UserContext; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.search.QuickSearch; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.util.GroupConstraintImpl; + +@Immutable +class RefLogicAny implements ReferencingLogic +{ + @Override + public GroupConstraint constraints() + { return GroupConstraintImpl.empty(); } + + private boolean isGrandchild (ReferencingLogicTransmuter idType) + { return !(idType == ReferencingLogicTransmuter.Bundle || idType == ReferencingLogicTransmuter.Collection); } + + private ReferencingLogicTransmuter resolveID (ControlContext context, UserContext input) + throws IOException, LidVidNotFoundException, UnknownGroupNameException + { + return ReferencingLogicTransmuter.getByProductClass( + QuickSearch.getValue(context.getConnection(), + LidVidUtils.resolve(input.getIdentifier(), ProductVersionSelector.TYPED, context, RequestBuildContextFactory.empty()), + "product_class")); + } + + private RequestAndResponseContext search(ControlContext context, UserContext input, boolean isIdToGroup) + throws ApplicationTypeException, IOException, LidVidNotFoundException, UnknownGroupNameException + { + // find all of the given group that reference the specified ID + ReferencingLogicTransmuter groupType = ReferencingLogicTransmuter.getBySwaggerGroup(input.getGroup()); + ReferencingLogicTransmuter idType = this.resolveID(context, input); + + if (idType == ReferencingLogicTransmuter.Bundle && groupType == ReferencingLogicTransmuter.Collection) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicBundle.children(context, input.getSelector(), input)); + if (idType == ReferencingLogicTransmuter.Bundle && this.isGrandchild(groupType)) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicBundle.grandchildren(context, input.getSelector(), input)); + if (idType == ReferencingLogicTransmuter.Collection && groupType == ReferencingLogicTransmuter.Bundle) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicCollection.parents(context, input.getSelector(), input)); + if (idType == ReferencingLogicTransmuter.Collection && this.isGrandchild(groupType)) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicCollection.children(context, input.getSelector(), input)); + if (this.isGrandchild(idType) && groupType == ReferencingLogicTransmuter.Bundle) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicProduct.grandparents(context, input.getSelector(), input)); + if (this.isGrandchild(idType) && groupType == ReferencingLogicTransmuter.Collection) + return RequestAndResponseContext.buildRequestAndResponseContext + (context, input, RefLogicProduct.parents(context, input.getSelector(), input)); + + throw new IOException("Waiting on implementation until references are figured out in database"); + // FIXME: return isIdToGroup ? this.idToGroup() : this.groupToId(); + } + + @Override + public RequestAndResponseContext find(ControlContext context, UserContext input) + throws ApplicationTypeException, IOException, LidVidNotFoundException, UnknownGroupNameException + { return this.search(context, input, false); } + + @Override + public RequestAndResponseContext given(ControlContext context, UserContext input) + throws ApplicationTypeException, IOException, LidVidNotFoundException, UnknownGroupNameException + { return this.search(context, input, true); } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicBundle.java b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicBundle.java new file mode 100644 index 00000000..2ad59626 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicBundle.java @@ -0,0 +1,179 @@ +package gov.nasa.pds.api.registry.model; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.TreeSet; + +import org.opensearch.action.search.SearchRequest; +import org.opensearch.client.RequestOptions; +import org.opensearch.search.SearchHit; +import org.opensearch.search.SearchHits; +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.LidvidsContext; +import gov.nasa.pds.api.registry.ReferencingLogic; +import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.search.RequestConstructionContextFactory; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; +import gov.nasa.pds.api.registry.util.GroupConstraintImpl; + +/** + * Bundle Data Access Object (DAO). + * Provides methods to get bundle information from opensearch. + * + * @author karpenko + */ +@Immutable +class RefLogicBundle extends RefLogicAny implements ReferencingLogic +{ + private static final Logger log = LoggerFactory.getLogger(RefLogicBundle.class); + + @Override + public GroupConstraint constraints() + { + Map> preset = new HashMap>(); + preset.put("product_class", Arrays.asList("Product_Bundle")); + return GroupConstraintImpl.buildAll(preset); + } + + static Pagination children (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws ApplicationTypeException, IOException, LidVidNotFoundException + { + log.info("Find children of a bundle"); + return selection == ProductVersionSelector.ALL ? + getAllBundleCollectionLidVids(uid, control) : + getBundleCollectionLidVids(uid, control); + } + + static Pagination grandchildren (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws ApplicationTypeException, IOException, LidVidNotFoundException + { + log.info("Find grandchildren of a bundle"); + PaginationLidvidBuilder ids = new PaginationLidvidBuilder(uid); + for (String cid : getBundleCollectionLidVids(new Unlimited(uid.getLidVid()), control).page()) + { ids.addAll(RefLogicCollection.children (control, selection, new Unlimited(cid)).page()); } + return ids; + } + + /** + * Get collections of a bundle by bundle LIDVID. + * If a bundle has LIDVID collection references, then those collections are returned. + * If a bundle has LID collection references, then the latest versions of collections are returned. + * @return a list of collection LIDVIDs + * @throws IOException IO exception + * @throws LidVidNotFoundException LIDVID not found exception + */ + static private Pagination getBundleCollectionLidVids( + LidvidsContext idContext, + ControlContext ctlContext) + throws IOException, LidVidNotFoundException + { + // Fetch collection references only. + List fields = Arrays.asList("ref_lidvid_collection","ref_lidvid_collection_secondary", + "ref_lid_collection", "ref_lid_collection_secondary"); + + // Get bundle by lidvid. + SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(idContext.getLidVid()), ctlContext.getConnection()) + .build(RequestBuildContextFactory.given(fields, + ReferencingLogicTransmuter.Bundle.impl().constraints()), + ctlContext.getConnection().getRegistryIndex()); + + // Call opensearch + SearchHit hit; + SearchHits hits = ctlContext.getConnection().getRestHighLevelClient().search(request, RequestOptions.DEFAULT).getHits(); + if(hits == null || hits.getTotalHits() == null || hits.getTotalHits().value != 1) + throw new LidVidNotFoundException(idContext.getLidVid()); + else hit = hits.getAt(0); + + // Get fields + // LidVid references (e.g., OREX bundle) + List ids = new ArrayList(); + Map fieldMap = hit.getSourceAsMap(); + PaginationLidvidBuilder lidvids = new PaginationLidvidBuilder(idContext); + + ids.addAll(lidvids.convert(fieldMap.get("ref_lidvid_collection"))); + ids.addAll(lidvids.convert(fieldMap.get("ref_lidvid_collection_secondary"))); + + // !!! NOTE !!! + // Harvest converts LIDVID references to LID references and stores them in + // "ref_lid_collection" and "ref_lid_collection_secondary" fields. + // To get "real" LID references, we have to exclude LIDVID references from these fields. + Set lidsToRemove = new TreeSet(); + for(String id: ids) + { + int idx = id.indexOf("::"); + if(idx > 0) + { + String lid = id.substring(0, idx); + lidsToRemove.add(lid); + } + } + + // Lid references (e.g., Kaguya bundle) plus LIDVID references converted by Harvest + List lids = new ArrayList(); + lids.addAll(lidvids.convert(fieldMap.get("ref_lid_collection"))); + lids.addAll(lidvids.convert(fieldMap.get("ref_lid_collection_secondary"))); + + // Get "real" LIDs + if(!lidsToRemove.isEmpty()) + { lids.removeAll(lidsToRemove); } + + // Get the latest versions of LIDs + lidvids.addAll(LidVidUtils.getLatestLidVidsByLids(ctlContext, + RequestBuildContextFactory.given("lid", + ReferencingLogicTransmuter.Collection.impl().constraints()), lids)); + return lidvids; + } + + + /** + * Get all versions of bundle's collections by bundle LIDVID. + */ + static private Pagination getAllBundleCollectionLidVids( + LidvidsContext idContext, + ControlContext ctlContext) + throws IOException, LidVidNotFoundException + { + // Fetch collection references only. + List fields = Arrays.asList("ref_lid_collection", "ref_lid_collection_secondary"); + + // Get bundle by lidvid. + SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(idContext.getLidVid()), ctlContext.getConnection()) + .build(RequestBuildContextFactory.given(fields, ReferencingLogicTransmuter.Bundle.impl().constraints()), ctlContext.getConnection().getRegistryIndex()); + + // Call opensearch + SearchHit hit; + SearchHits hits = ctlContext.getConnection().getRestHighLevelClient().search(request, RequestOptions.DEFAULT).getHits(); + if (hits == null || hits.getTotalHits() == null || hits.getTotalHits().value != 1) + throw new LidVidNotFoundException(idContext.getLidVid()); + else hit = hits.getAt(0); + + // Get fields + Map fieldMap = hit.getSourceAsMap(); + + // Lid references (e.g., Kaguya bundle) + List ids = new ArrayList(); + PaginationLidvidBuilder lidvids = new PaginationLidvidBuilder(idContext); + + ids.addAll(lidvids.convert(fieldMap.get("ref_lid_collection"))); + ids.addAll(lidvids.convert(fieldMap.get("ref_lid_collection_secondary"))); + ids = LidVidUtils.getAllLidVidsByLids(ctlContext, + RequestBuildContextFactory.given("lidvid", ReferencingLogicTransmuter.Collection.impl().constraints()), + ids); + lidvids.addAll(ids); + return lidvids; + } + +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicCollection.java b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicCollection.java new file mode 100644 index 00000000..870125e1 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicCollection.java @@ -0,0 +1,109 @@ +package gov.nasa.pds.api.registry.model; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.LidvidsContext; +import gov.nasa.pds.api.registry.ReferencingLogic; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.search.HitIterator; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.search.RequestConstructionContextFactory; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; +import gov.nasa.pds.api.registry.util.GroupConstraintImpl; + +@Immutable +class RefLogicCollection extends RefLogicAny implements ReferencingLogic +{ + private static final Logger log = LoggerFactory.getLogger(RefLogicCollection.class); + + @Override + public GroupConstraint constraints() + { + Map> preset = new HashMap>(); + preset.put("product_class", Arrays.asList("Product_Collection")); + return GroupConstraintImpl.buildAll(preset); + } + + static Pagination children (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + log.info("Find children of a collection -- both all and latest"); + return selection == ProductVersionSelector.ALL ? RefLogicCollection.childrenAll(control, uid) : RefLogicCollection.childrenLatest(control, uid); + } + + private static Pagination childrenAll (ControlContext control, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + PaginationLidvidBuilder productLidvids = new PaginationLidvidBuilder(uid); + + for (final Map kvp : new HitIterator(control.getConnection().getRestHighLevelClient(), + new SearchRequestFactory(RequestConstructionContextFactory.given ("collection_lidvid", uid.getLidVid(), true), control.getConnection()) + .build (RequestBuildContextFactory.given ("product_lid"), control.getConnection().getRegistryRefIndex()))) + { + productLidvids.addAll(LidVidUtils.getAllLidVidsByLids(control, + RequestBuildContextFactory.given("lidvid", ReferencingLogicTransmuter.Product.impl().constraints()), + productLidvids.convert(kvp.get("product_lid")))); + } + return productLidvids; +} + + private static Pagination childrenLatest (ControlContext control, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + PaginationLidvidBuilder productLidvids = new PaginationLidvidBuilder(uid); + + for (final Map kvp : new HitIterator(control.getConnection().getRestHighLevelClient(), + new SearchRequestFactory(RequestConstructionContextFactory.given ("collection_lidvid", uid.getLidVid(), true), control.getConnection()) + .build (RequestBuildContextFactory.given ("product_lidvid"), control.getConnection().getRegistryRefIndex()))) + { productLidvids.add(kvp.get("product_lidvid")); } + return productLidvids; + } + + static Pagination parents (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + List keys = Arrays.asList("ref_lid_collection", "ref_lid_collection_secondary", + "ref_lidvid_collection", "ref_lidvid_collection_secondary"); + List sortedLids; + Set lids = new HashSet(); + String lid = LidVidUtils.extractLidFromLidVid(uid.getLidVid()); + PaginationLidvidBuilder bundleLidvids = new PaginationLidvidBuilder(uid); + + log.info("Find parents of a colletion -- both all and latest"); + log.info("Find parents of collenction: " + uid.getLidVid() + " --- " + LidVidUtils.extractLidFromLidVid(uid.getLidVid())); + for (String key : keys) + { + for (final Map kvp : new HitIterator(control.getConnection().getRestHighLevelClient(), + new SearchRequestFactory(RequestConstructionContextFactory.given(key, lid, true), control.getConnection()) + .build(RequestBuildContextFactory.given("lid", ReferencingLogicTransmuter.Bundle.impl().constraints()), + control.getConnection().getRegistryIndex()))) + { + lids.addAll(bundleLidvids.convert(kvp.get("lid"))); + } + } + sortedLids = new ArrayList(lids); + Collections.sort(sortedLids); + + if (selection == ProductVersionSelector.ALL) + bundleLidvids.addAll (LidVidUtils.getAllLidVidsByLids(control, RequestBuildContextFactory.empty(), sortedLids)); + else + bundleLidvids.addAll (LidVidUtils.getLatestLidVidsByLids(control, RequestBuildContextFactory.empty(), sortedLids)); + + return bundleLidvids; + } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicProduct.java b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicProduct.java new file mode 100644 index 00000000..947c98d3 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/RefLogicProduct.java @@ -0,0 +1,82 @@ +package gov.nasa.pds.api.registry.model; + +import java.io.IOException; +import java.util.ArrayList; +import java.util.Arrays; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; + +import org.slf4j.Logger; +import org.slf4j.LoggerFactory; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.LidvidsContext; +import gov.nasa.pds.api.registry.ReferencingLogic; +import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; +import gov.nasa.pds.api.registry.search.HitIterator; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.search.RequestConstructionContextFactory; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; +import gov.nasa.pds.api.registry.util.GroupConstraintImpl; + +@Immutable +class RefLogicProduct extends RefLogicAny implements ReferencingLogic +{ + private static final Logger log = LoggerFactory.getLogger(RefLogicProduct.class); + + @Override + public GroupConstraint constraints() + { + Map> preset = new HashMap>(); + preset.put("product_class", Arrays.asList("Product_Bundle","Product_Collection")); + return GroupConstraintImpl.buildNot(preset); + } + + static Pagination grandparents (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + log.info ("Find the grandparents of a product -- both all and latest"); + List parents = RefLogicProduct.parents(control, ProductVersionSelector.LATEST, new Unlimited(uid.getLidVid())) + .page(); + PaginationLidvidBuilder grandparents = new PaginationLidvidBuilder(uid); + for (String parent : parents) + { + log.info("Find all the parents of collection: " + parent); + grandparents.addAll(RefLogicCollection.parents(control, selection, new Unlimited(parent)).page()); + log.info("Find grandparents size: " + String.valueOf(grandparents.size())); + for (String gp : grandparents.page()) { log.info(" grandparent: " + gp); } + } + return grandparents; + } + + static Pagination parents (ControlContext control, ProductVersionSelector selection, LidvidsContext uid) + throws IOException, LidVidNotFoundException + { + List sorted_lids; + PaginationLidvidBuilder parents = new PaginationLidvidBuilder(uid); + Set lids = new HashSet(); + + log.info("Find the parents of a product -- both all and latest"); + for (final Map kvp : new HitIterator(control.getConnection().getRestHighLevelClient(), + new SearchRequestFactory(RequestConstructionContextFactory.given("product_lidvid", uid.getLidVid(), true), control.getConnection()) + .build (RequestBuildContextFactory.given("collection_lid"), control.getConnection().getRegistryRefIndex()))) + { lids.addAll(parents.convert(kvp.get("collection_lid"))); } + sorted_lids = new ArrayList(lids); + Collections.sort(sorted_lids); + + if (selection == ProductVersionSelector.ALL) + parents.addAll (LidVidUtils.getAllLidVidsByLids(control, RequestBuildContextFactory.empty(), sorted_lids)); + else + parents.addAll(LidVidUtils.getLatestLidVidsByLids(control, RequestBuildContextFactory.empty(), sorted_lids)); + + return parents; + } + +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/ReferencingLogicTransmuter.java b/service/src/main/java/gov/nasa/pds/api/registry/model/ReferencingLogicTransmuter.java new file mode 100644 index 00000000..d71076f6 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/ReferencingLogicTransmuter.java @@ -0,0 +1,70 @@ +package gov.nasa.pds.api.registry.model; + +import java.util.HashSet; +import java.util.Set; + +import gov.nasa.pds.api.registry.ReferencingLogic; +import gov.nasa.pds.api.registry.exceptions.NothingFoundException; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; + +public enum ReferencingLogicTransmuter +{ + Any(new RefLogicAny(), "", "any"), + Bundle(new RefLogicBundle(), "Product_Bundle", "bundle"), + Collection(new RefLogicCollection(), "Product_Collection", "collection"), + Product(new RefLogicProduct(), "", "product"); + + final private ReferencingLogic refLogic; + final private String pds_name; + final private String swagger_name; + private ReferencingLogicTransmuter (ReferencingLogic impl, String pds_name, String swagger_name) + { + this.refLogic = impl; + this.pds_name = pds_name; + this.swagger_name = swagger_name; + } + + private static ReferencingLogicTransmuter get (String name, boolean usingPDSName) throws NothingFoundException + { + ReferencingLogicTransmuter resultant = null; + + if (name.length() == 0) return ReferencingLogicTransmuter.Any; + + for (ReferencingLogicTransmuter pc : ReferencingLogicTransmuter.values()) + { + if (name.equals(usingPDSName ? pc.pds_name : pc.swagger_name)) + { + resultant = pc; + break; + } + } + + if (resultant == null) throw new NothingFoundException(); + return resultant; + } + + public static ReferencingLogicTransmuter getByProductClass (String name) throws UnknownGroupNameException + { + try { return ReferencingLogicTransmuter.get(name, true); } + catch (NothingFoundException nfe) + { + return ReferencingLogicTransmuter.Any; +// Set known = new HashSet(); +// for (ReferencingLogicTransmuter pc : ReferencingLogicTransmuter.values()) known.add(pc.pds_name); +// throw new UnknownGroupNameException(name, known); + } + } + + public static ReferencingLogicTransmuter getBySwaggerGroup (String name) throws UnknownGroupNameException + { + try { return ReferencingLogicTransmuter.get(name, false); } + catch (NothingFoundException nfe) + { + Set known = new HashSet(); + for (ReferencingLogicTransmuter pc : ReferencingLogicTransmuter.values()) known.add(pc.swagger_name); + throw new UnknownGroupNameException(name, known); + } + + } + public ReferencingLogic impl() { return this.refLogic; } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/RequestAndResponseContext.java b/service/src/main/java/gov/nasa/pds/api/registry/model/RequestAndResponseContext.java similarity index 81% rename from service/src/main/java/gov/nasa/pds/api/registry/business/RequestAndResponseContext.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/RequestAndResponseContext.java index 40830366..6ed7f103 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/RequestAndResponseContext.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/RequestAndResponseContext.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.io.IOException; import java.util.ArrayList; @@ -17,14 +17,18 @@ import com.ibm.icu.util.StringTokenizer; import gov.nasa.pds.api.registry.ControlContext; +import gov.nasa.pds.api.registry.GroupConstraint; import gov.nasa.pds.api.registry.RequestBuildContext; import gov.nasa.pds.api.registry.RequestConstructionContext; import gov.nasa.pds.api.registry.UserContext; import gov.nasa.pds.api.registry.exceptions.ApplicationTypeException; import gov.nasa.pds.api.registry.exceptions.LidVidNotFoundException; import gov.nasa.pds.api.registry.exceptions.NothingFoundException; -import gov.nasa.pds.api.registry.opensearch.HitIterator; -import gov.nasa.pds.api.registry.opensearch.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.exceptions.UnknownGroupNameException; +import gov.nasa.pds.api.registry.search.HitIterator; +import gov.nasa.pds.api.registry.search.RequestBuildContextFactory; +import gov.nasa.pds.api.registry.search.RequestConstructionContextFactory; +import gov.nasa.pds.api.registry.search.SearchRequestFactory; import gov.nasa.pds.model.Summary; public class RequestAndResponseContext implements RequestBuildContext,RequestConstructionContext @@ -40,8 +44,7 @@ public class RequestAndResponseContext implements RequestBuildContext,RequestCon final private List sort; final private int start; final private int limit; - final private Map presetCriteria; - final private boolean summaryOnly; + final private GroupConstraint presetCriteria; final private ProductVersionSelector selector; final private String format; final private Map formatters; @@ -49,24 +52,37 @@ public class RequestAndResponseContext implements RequestBuildContext,RequestCon static public RequestAndResponseContext buildRequestAndResponseContext( ControlContext connection, // webby criteria UserContext parameters, - Map outPreset, // when first and last node of the endpoint criteria are the same - String output_format // the accept statement of the request that informs the output type + Pagination lidvids) throws ApplicationTypeException,LidVidNotFoundException,IOException,UnknownGroupNameException + { + GroupConstraint any = ReferencingLogicTransmuter.Any.impl().constraints(); + GroupConstraint preset = ReferencingLogicTransmuter.getBySwaggerGroup(parameters.getGroup()).impl().constraints(); + RequestAndResponseContext response = new RequestAndResponseContext (connection, parameters, preset, any); + SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(lidvids.page()), connection.getConnection()) + .build(response, connection.getConnection().getRegistryIndex()); + request.source().size(lidvids.size()); + response.setResponse(connection.getConnection().getRestHighLevelClient().search + (request, RequestOptions.DEFAULT).getHits(), null, lidvids.total()); + return response; + } + + static public RequestAndResponseContext buildRequestAndResponseContext( + ControlContext connection, // webby criteria + UserContext parameters, + GroupConstraint outPreset // when first and last node of the endpoint criteria are the same ) throws ApplicationTypeException,LidVidNotFoundException,IOException - { return new RequestAndResponseContext(connection, parameters, outPreset, outPreset, output_format); } + { return new RequestAndResponseContext(connection, parameters, outPreset, outPreset); } static public RequestAndResponseContext buildRequestAndResponseContext( ControlContext connection, // webby criteria UserContext parameters, - Map outPreset, MapresPreset, // criteria for defining last node (outPreset) and first node (resOutput) for any endpoint - String output_format // the accept statement of the request that informs the output type + GroupConstraint outPreset, GroupConstraint resPreset // criteria for defining last node (outPreset) and first node (resOutput) for any endpoint ) throws ApplicationTypeException,LidVidNotFoundException,IOException - { return new RequestAndResponseContext(connection, parameters, outPreset, resPreset, output_format); } + { return new RequestAndResponseContext(connection, parameters, outPreset, resPreset); } private RequestAndResponseContext( ControlContext controlContext,// webby criteria UserContext parameters, - Map outPreset, MapresPreset, // criteria for defining last node (outPreset) and first node (resOutput) for any endpoint - String output_format // the accept statement of the request that informs the output type + GroupConstraint outPreset, GroupConstraint resPreset // criteria for defining last node (outPreset) and first node (resOutput) for any endpoint ) throws ApplicationTypeException,LidVidNotFoundException,IOException { Map formatters = new HashMap(); @@ -82,12 +98,12 @@ private RequestAndResponseContext( formatters.put("text/xml", new PdsProductBusinessObject()); this.controlContext = controlContext; this.formatters = formatters; - this.format = this.find_match(output_format); + this.format = this.find_match(parameters.getAccept()); this.queryString = parameters.getQuery(); this.keywords = parameters.getKeywords(); this.fields = new ArrayList(); this.fields.addAll(this.add_output_needs (parameters.getFields())); - this.lidvid = LidVidUtils.resolveLIDVID( + this.lidvid = LidVidUtils.resolve( parameters.getIdentifier(), outPreset.equals(resPreset) ? parameters.getSelector() : ProductVersionSelector.TYPED, controlContext, @@ -95,7 +111,6 @@ private RequestAndResponseContext( this.limit = parameters.getLimit(); this.sort = parameters.getSort(); this.start = parameters.getStart(); - this.summaryOnly = parameters.getSummanryOnly(); this.presetCriteria = outPreset; this.selector = parameters.getSelector(); } @@ -112,11 +127,11 @@ private RequestAndResponseContext( public int getLimit() { return this.limit; } @Override public String getQueryString() { return this.queryString; } - public final Map getPresetCriteria() { return this.presetCriteria; }; + public final GroupConstraint getPresetCriteria() { return this.presetCriteria; }; public ProductVersionSelector getSelector() { return this.selector; } public boolean isSingular() { return this.getStart() == -1 && this.getLimit() == 0; } - public boolean isSummaryOnly() { return this.summaryOnly; } + public boolean isSummaryOnly() { return this.limit == 0; } @Override public boolean isTerm() { return true; } // no way to make this decision here so always term for lidvid @@ -216,7 +231,7 @@ public void setResponse(HitIterator hits, int real_total) summary.setStart(this.getStart()); summary.setLimit(this.getLimit()); summary.setSort(this.getSort()); - summary.setHits(this.formatters.get(this.format).setResponse(hits, summary, this.fields, this.summaryOnly)); + summary.setHits(this.formatters.get(this.format).setResponse(hits, summary, this.fields, this.isSummaryOnly())); summary.setProperties(new ArrayList()); if (0 < real_total) summary.setHits(real_total); @@ -242,7 +257,7 @@ public void setResponse(SearchHits hits, List uniqueProperties, int tota summary.setHits(total_hits); if (uniqueProperties != null) summary.setProperties(uniqueProperties); - this.formatters.get(this.format).setResponse(hits, summary, this.fields, this.summaryOnly); + this.formatters.get(this.format).setResponse(hits, summary, this.fields, this.isSummaryOnly()); summary.setTook((int)(System.currentTimeMillis() - this.begin_processing)); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/SearchUtil.java b/service/src/main/java/gov/nasa/pds/api/registry/model/SearchUtil.java similarity index 99% rename from service/src/main/java/gov/nasa/pds/api/registry/business/SearchUtil.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/SearchUtil.java index 6678ccbf..8072f801 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/SearchUtil.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/SearchUtil.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.net.URI; import java.net.URISyntaxException; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/model/Unlimited.java b/service/src/main/java/gov/nasa/pds/api/registry/model/Unlimited.java new file mode 100644 index 00000000..8ee2f01c --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/Unlimited.java @@ -0,0 +1,19 @@ +package gov.nasa.pds.api.registry.model; + +import gov.nasa.pds.api.registry.LidvidsContext; + +class Unlimited implements LidvidsContext +{ + final private String lidvid; + Unlimited (String id) { this.lidvid = id; } + + @Override + public String getLidVid() { return this.lidvid; } + + @Override + public Integer getLimit() { return Integer.MAX_VALUE; } + + @Override + public Integer getStart() { return 0; } + +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/business/WyriwygBusinessObject.java b/service/src/main/java/gov/nasa/pds/api/registry/model/WyriwygBusinessObject.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/business/WyriwygBusinessObject.java rename to service/src/main/java/gov/nasa/pds/api/registry/model/WyriwygBusinessObject.java index 9b38e5f6..ed951f6e 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/business/WyriwygBusinessObject.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/model/WyriwygBusinessObject.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.business; +package gov.nasa.pds.api.registry.model; import java.net.URL; import java.util.ArrayList; @@ -16,7 +16,7 @@ import com.fasterxml.jackson.databind.ObjectMapper; import gov.nasa.pds.api.registry.exceptions.UnsupportedSearchProperty; -import gov.nasa.pds.api.registry.opensearch.HitIterator; +import gov.nasa.pds.api.registry.search.HitIterator; import gov.nasa.pds.model.Summary; import gov.nasa.pds.model.WyriwygProduct; import gov.nasa.pds.model.WyriwygProductKeyValuePairs; @@ -126,6 +126,7 @@ public int setResponse(SearchHits hits, Summary summary, List fields, bo products.addDataItem(product); } } + summary.setProperties(new ArrayList(uniqueProperties)); products.setSummary(summary); this.products = products; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestBuildContextFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestBuildContextFactory.java deleted file mode 100644 index 2ce0b9f1..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestBuildContextFactory.java +++ /dev/null @@ -1,23 +0,0 @@ -package gov.nasa.pds.api.registry.opensearch; - -import java.util.ArrayList; -import java.util.Arrays; -import java.util.List; -import java.util.Map; - -import gov.nasa.pds.api.registry.RequestBuildContext; - -public class RequestBuildContextFactory -{ - public static RequestBuildContext given (String field) - { return new SimpleRequestBuildContext(new ArrayList(Arrays.asList(field))); } - - public static RequestBuildContext given (List fields) - { return new SimpleRequestBuildContext(fields); } - - public static RequestBuildContext given (String field, Map preset) - { return new SimpleRequestBuildContext(new ArrayList(Arrays.asList(field)), preset); } - - public static RequestBuildContext given (List fields, Map preset) - { return new SimpleRequestBuildContext(fields, preset); } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestBuildContext.java b/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestBuildContext.java deleted file mode 100644 index db62ab29..00000000 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestBuildContext.java +++ /dev/null @@ -1,31 +0,0 @@ -package gov.nasa.pds.api.registry.opensearch; - -import java.util.HashMap; -import java.util.List; -import java.util.Map; - -import gov.nasa.pds.api.registry.RequestBuildContext; - -class SimpleRequestBuildContext implements RequestBuildContext -{ - final private List fields; - final private Map preset; - - SimpleRequestBuildContext (List fields) - { - this.fields = fields; - this.preset = new HashMap(); - } - - SimpleRequestBuildContext (List fields, Map preset) - { - this.fields = fields; - this.preset = preset; - } - - @Override - public List getFields() { return fields; } - - @Override - public Map getPresetCriteria() { return preset; } -} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/HitIterator.java b/service/src/main/java/gov/nasa/pds/api/registry/search/HitIterator.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/HitIterator.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/HitIterator.java index 9a028732..1b35f9d4 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/HitIterator.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/HitIterator.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.io.IOException; import java.util.Iterator; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchConfig.java b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchConfig.java similarity index 98% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchConfig.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchConfig.java index c3159871..8a03cc24 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchConfig.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchConfig.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.util.List; import org.slf4j.Logger; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImpl.java b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImpl.java similarity index 99% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImpl.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImpl.java index 488080d0..36349c0c 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImpl.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImpl.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.util.List; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImplBuilder.java b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImplBuilder.java similarity index 98% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImplBuilder.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImplBuilder.java index 2210a5a4..e59de8a4 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/OpenSearchRegistryConnectionImplBuilder.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/OpenSearchRegistryConnectionImplBuilder.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.util.Arrays; import java.util.List; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/search/QuickSearch.java b/service/src/main/java/gov/nasa/pds/api/registry/search/QuickSearch.java new file mode 100644 index 00000000..f53df0e3 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/QuickSearch.java @@ -0,0 +1,37 @@ +package gov.nasa.pds.api.registry.search; + +import java.io.IOException; +import java.util.List; + +import org.opensearch.action.search.SearchRequest; +import org.opensearch.client.RequestOptions; + +import gov.nasa.pds.api.registry.ConnectionContext; + +public class QuickSearch +{ + final private static Object get(ConnectionContext connection, String index, String lidvid, String name) throws IOException + { + SearchRequest request = new SearchRequestFactory(RequestConstructionContextFactory.given(lidvid), connection) + .build(RequestBuildContextFactory.given(name), index); + return connection.getRestHighLevelClient().search(request, RequestOptions.DEFAULT) + .getHits() + .getAt(0) + .getSourceAsMap() + .get(name); + } + + final public static String getValue (ConnectionContext connection, String lidvid, String name) throws IOException + { return (String)QuickSearch.get(connection, connection.getRegistryIndex(), lidvid, name); } + + final public static String getValue (ConnectionContext connection, String index, String lidvid, String name) throws IOException + { return (String)QuickSearch.get(connection, index, lidvid, name); } + + @SuppressWarnings("unchecked") + final public static List getValues (ConnectionContext connection, String lidvid, String name) throws IOException + { return (List)QuickSearch.get(connection, connection.getRegistryIndex(), lidvid, name); } + + @SuppressWarnings("unchecked") + final public static List getValues (ConnectionContext connection, String index, String lidvid, String name) throws IOException + { return (List)QuickSearch.get(connection, index, lidvid, name); } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/search/RequestBuildContextFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/search/RequestBuildContextFactory.java new file mode 100644 index 00000000..7e6b0c98 --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/RequestBuildContextFactory.java @@ -0,0 +1,24 @@ +package gov.nasa.pds.api.registry.search; + +import java.util.Arrays; +import java.util.List; + +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.RequestBuildContext; + +public class RequestBuildContextFactory +{ + public static RequestBuildContext empty() { return new SimpleRequestBuildContext(); } + + public static RequestBuildContext given (String field) + { return new SimpleRequestBuildContext(Arrays.asList(field)); } + + public static RequestBuildContext given (List fields) + { return new SimpleRequestBuildContext(fields); } + + public static RequestBuildContext given (String field, GroupConstraint preset) + { return new SimpleRequestBuildContext(Arrays.asList(field), preset); } + + public static RequestBuildContext given (List fields, GroupConstraint preset) + { return new SimpleRequestBuildContext(fields, preset); } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestConstructionContextFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/search/RequestConstructionContextFactory.java similarity index 88% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestConstructionContextFactory.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/RequestConstructionContextFactory.java index ad5ad050..270ca86b 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/RequestConstructionContextFactory.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/RequestConstructionContextFactory.java @@ -1,6 +1,5 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; -import java.util.ArrayList; import java.util.Arrays; import java.util.HashMap; import java.util.List; @@ -22,7 +21,7 @@ public static RequestConstructionContext given (List lidvids) public static RequestConstructionContext given (String key, String value, boolean asTerm) { - List values = new ArrayList(Arrays.asList(value)); + List values = Arrays.asList(value); Map> kvps = new HashMap>(); kvps.put(key, values); return new SimpleRequestConstructionContext(kvps, asTerm); diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SearchRequestFactory.java b/service/src/main/java/gov/nasa/pds/api/registry/search/SearchRequestFactory.java similarity index 72% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/SearchRequestFactory.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/SearchRequestFactory.java index cb8b915b..5e28a9d6 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SearchRequestFactory.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/SearchRequestFactory.java @@ -1,6 +1,7 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.util.List; +import java.util.Map; import org.opensearch.action.search.SearchRequest; import org.opensearch.index.query.BoolQueryBuilder; @@ -10,9 +11,9 @@ import gov.nasa.pds.api.registry.ConnectionContext; import gov.nasa.pds.api.registry.RequestBuildContext; import gov.nasa.pds.api.registry.RequestConstructionContext; -import gov.nasa.pds.api.registry.business.BlobUtil; -import gov.nasa.pds.api.registry.business.LidVidUtils; -import gov.nasa.pds.api.registry.business.ProductQueryBuilderUtil; +import gov.nasa.pds.api.registry.model.BlobUtil; +import gov.nasa.pds.api.registry.model.LidVidUtils; +import gov.nasa.pds.api.registry.model.ProductQueryBuilderUtil; public class SearchRequestFactory { @@ -25,25 +26,24 @@ public SearchRequestFactory(RequestConstructionContext context, ConnectionContex if (!context.getKeyValuePairs().isEmpty()) { - for (String key: context.getKeyValuePairs().keySet()) - if (context.isTerm()) - { - if (context.getKeyValuePairs().get(key).size() == 1) - { this.base.must(QueryBuilders.termsQuery(key, context.getKeyValuePairs().get(key))); } - else - { - if ("lidvid".equals (key)) this.base.filter(QueryBuilders.termsQuery(key, context.getKeyValuePairs().get(key))); - else this.base.should(QueryBuilders.termsQuery(key, context.getKeyValuePairs().get(key))); - } - } - else + for (Map.Entry> entry: context.getKeyValuePairs().entrySet()) { - if (context.getKeyValuePairs().get(key).size() == 1) - { this.base.must(QueryBuilders.matchQuery(key, context.getKeyValuePairs().get(key).get(0))); } + if (context.isTerm()) + { + if (entry.getValue().size() == 1) + { this.base.must(QueryBuilders.termQuery(entry.getKey(), entry.getValue().get(0))); } + else + { this.base.filter(QueryBuilders.termsQuery(entry.getKey(), entry.getValue())); } + } else { - for (String value : context.getKeyValuePairs().get(key)) - { this.base.should(QueryBuilders.matchQuery(key, value)); } + if (entry.getValue().size() == 1) + { this.base.must(QueryBuilders.matchQuery(entry.getKey(), entry.getValue().get(0))); } + else + { + for (String value : entry.getValue()) + { this.base.filter(QueryBuilders.matchQuery(entry.getKey(), value)); } + } } } } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestBuildContext.java b/service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestBuildContext.java new file mode 100644 index 00000000..8a93cebd --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestBuildContext.java @@ -0,0 +1,38 @@ +package gov.nasa.pds.api.registry.search; + +import java.util.ArrayList; +import java.util.List; + +import gov.nasa.pds.api.registry.GroupConstraint; +import gov.nasa.pds.api.registry.RequestBuildContext; +import gov.nasa.pds.api.registry.util.GroupConstraintImpl; + +class SimpleRequestBuildContext implements RequestBuildContext +{ + final private List fields; + final private GroupConstraint preset; + + SimpleRequestBuildContext () + { + this.fields = new ArrayList(); + this.preset = GroupConstraintImpl.empty(); + } + + SimpleRequestBuildContext (List fields) + { + this.fields = fields; + this.preset = GroupConstraintImpl.empty(); + } + + SimpleRequestBuildContext (List fields, GroupConstraint preset) + { + this.fields = fields; + this.preset = preset; + } + + @Override + public List getFields() { return fields; } + + @Override + public GroupConstraint getPresetCriteria() { return preset; } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestConstructionContext.java b/service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestConstructionContext.java similarity index 96% rename from service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestConstructionContext.java rename to service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestConstructionContext.java index ebed8d5c..f3969649 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/opensearch/SimpleRequestConstructionContext.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/search/SimpleRequestConstructionContext.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.opensearch; +package gov.nasa.pds.api.registry.search; import java.util.ArrayList; import java.util.HashMap; @@ -40,6 +40,7 @@ class SimpleRequestConstructionContext implements RequestConstructionContext this.kvps = new HashMap>(); this.lidvid = lidvid; } + @Override public List getKeywords() { return new ArrayList(); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/util/GroupConstraintImpl.java b/service/src/main/java/gov/nasa/pds/api/registry/util/GroupConstraintImpl.java new file mode 100644 index 00000000..c4ceff2c --- /dev/null +++ b/service/src/main/java/gov/nasa/pds/api/registry/util/GroupConstraintImpl.java @@ -0,0 +1,45 @@ +package gov.nasa.pds.api.registry.util; + +import java.util.HashMap; +import java.util.List; +import java.util.Map; + +import com.google.errorprone.annotations.Immutable; + +import gov.nasa.pds.api.registry.GroupConstraint; + +@Immutable +public class GroupConstraintImpl implements GroupConstraint +{ + final private Map> all; + final private Map> any; + final private Map> not; + + private GroupConstraintImpl(Map> all, Map> any, Map>not) + { + this.all = Map.copyOf(all); + this.any = Map.copyOf(any); + this.not = Map.copyOf(not); + } + + @Override + public Map> all() { return all;} + @Override + public Map> any() { return any; } + @Override + public Map> not() { return not; } + + final private static Map> EMPTY = new HashMap>(); + public static GroupConstraint empty() { return new GroupConstraintImpl(EMPTY, EMPTY, EMPTY); } + public static GroupConstraint buildAll(Map> map) { return new GroupConstraintImpl(map, EMPTY, EMPTY); } + public static GroupConstraint buildAny(Map> map) { return new GroupConstraintImpl(EMPTY, map, EMPTY); } + public static GroupConstraint buildNot(Map> map) { return new GroupConstraintImpl(EMPTY, EMPTY, map); } + public static GroupConstraint buildAllAny(Map> allmap, Map> anymap) + { return new GroupConstraintImpl(allmap, anymap, EMPTY); } + public static GroupConstraint buildAllNot(Map> allmap, Map> notmap) + { return new GroupConstraintImpl(allmap, EMPTY, notmap); } + public static GroupConstraint buildAnyNot(Map> anymap, Map> notmap) + { return new GroupConstraintImpl(EMPTY, anymap, notmap); } + public static GroupConstraint build (Map> allmap, Map> anymap, Map> notmap) + { return new GroupConstraintImpl(allmap, anymap, notmap); } +} diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvErrorMessageSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvErrorMessageSerializer.java similarity index 96% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvErrorMessageSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/CsvErrorMessageSerializer.java index 8eb4469b..60bc6648 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvErrorMessageSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvErrorMessageSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStreamWriter; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvPluralSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvPluralSerializer.java similarity index 95% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvPluralSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/CsvPluralSerializer.java index c1e46dbd..0efb279e 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvPluralSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvPluralSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; @@ -39,6 +39,7 @@ protected void writeInternal(WyriwygProducts t, HttpOutputMessage outputMessage) OutputStream os = outputMessage.getBody(); OutputStreamWriter wr = new OutputStreamWriter(os); + Utilities.fix (t.getSummary()); WyriwygSerializer.writeCSV(t, wr, mapper); wr.close(); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvSingularSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvSingularSerializer.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvSingularSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/CsvSingularSerializer.java index c2d6cc58..b650fcda 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/CsvSingularSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/CsvSingularSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/HtmlErrorMessageSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/HtmlErrorMessageSerializer.java similarity index 96% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/HtmlErrorMessageSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/HtmlErrorMessageSerializer.java index e95e33d9..1c403827 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/HtmlErrorMessageSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/HtmlErrorMessageSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStreamWriter; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonErrorMessageSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonErrorMessageSerializer.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonErrorMessageSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/JsonErrorMessageSerializer.java index 7c8e3c3a..c8b64bd6 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonErrorMessageSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonErrorMessageSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStreamWriter; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonPluralSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonPluralSerializer.java similarity index 95% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonPluralSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/JsonPluralSerializer.java index 1392fdb8..d07c3685 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonPluralSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonPluralSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; @@ -39,6 +39,7 @@ protected void writeInternal(WyriwygProducts t, HttpOutputMessage outputMessage) OutputStream os = outputMessage.getBody(); OutputStreamWriter wr = new OutputStreamWriter(os); + Utilities.fix (t.getSummary()); WyriwygSerializer.writeJSON(t, wr, mapper); wr.close(); } diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonProductSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonProductSerializer.java similarity index 94% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonProductSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/JsonProductSerializer.java index 138b9bcf..59db436b 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonProductSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonProductSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.util.ArrayList; import java.util.List; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonSingularSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonSingularSerializer.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonSingularSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/JsonSingularSerializer.java index ee9fad4c..6644a9a9 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/JsonSingularSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/JsonSingularSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductSerializer.java similarity index 98% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductSerializer.java index be20921d..f2b0cd2c 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductsSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductsSerializer.java similarity index 96% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductsSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductsSerializer.java index 5214b46a..fc302a36 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4JsonProductsSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4JsonProductsSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; @@ -60,6 +60,7 @@ public void writeInternal(Pds4Products products, HttpOutputMessage msg) OutputStream os = msg.getBody(); OutputStreamWriter wr = new OutputStreamWriter(os); + Utilities.fix (products.getSummary()); wr.write("{\n"); // Summary diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductSerializer.java similarity index 98% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductSerializer.java index 7da1b46c..4c0432cb 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import gov.nasa.pds.model.Pds4Product; diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductsSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductsSerializer.java similarity index 97% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductsSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductsSerializer.java index f56a4f92..366ab590 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/Pds4XmlProductsSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/Pds4XmlProductsSerializer.java @@ -1,4 +1,4 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; @@ -56,6 +56,7 @@ protected void writeInternal(Pds4Products products, HttpOutputMessage outputMess XMLOutputFactory outputFactory = XMLOutputFactory.newFactory(); outputFactory.setProperty("javax.xml.stream.isRepairingNamespaces", true); XMLStreamWriter writer = outputFactory.createXMLStreamWriter(outputStream); + Utilities.fix (products.getSummary()); writer.setPrefix(Pds4XmlProductSerializer.NAMESPACE_PREFIX, Pds4XmlProductSerializer.NAMESPACE_URL); writer.writeStartElement(Pds4XmlProductSerializer.NAMESPACE_URL, "products"); diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductTextHtmlSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductTextHtmlSerializer.java similarity index 91% rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductTextHtmlSerializer.java rename to service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductTextHtmlSerializer.java index 4da7e0a4..51d77dbc 100644 --- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductTextHtmlSerializer.java +++ b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductTextHtmlSerializer.java @@ -1,8 +1,9 @@ -package gov.nasa.pds.api.registry.serializer; +package gov.nasa.pds.api.registry.view; import java.io.IOException; import java.io.OutputStream; import java.io.OutputStreamWriter; +import java.nio.charset.Charset; import org.springframework.http.HttpInputMessage; import org.springframework.http.HttpOutputMessage; @@ -36,7 +37,7 @@ protected void writeInternal(PdsProduct t, HttpOutputMessage outputMessage) { ObjectMapper mapper = new ObjectMapper(); OutputStream os = outputMessage.getBody(); - OutputStreamWriter wr = new OutputStreamWriter(os); + OutputStreamWriter wr = new OutputStreamWriter(os, Charset.defaultCharset()); mapper.setSerializationInclusion(Include.NON_NULL); wr.write("

JSON as text

");
         wr.write(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(t));
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductXMLSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductXMLSerializer.java
similarity index 97%
rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductXMLSerializer.java
rename to service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductXMLSerializer.java
index b27e024d..3a871401 100644
--- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductXMLSerializer.java
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductXMLSerializer.java
@@ -1,4 +1,4 @@
-package gov.nasa.pds.api.registry.serializer;
+package gov.nasa.pds.api.registry.view;
 
 import gov.nasa.pds.model.PdsProduct;
 
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsTextHtmlSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsTextHtmlSerializer.java
similarity index 85%
rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsTextHtmlSerializer.java
rename to service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsTextHtmlSerializer.java
index 816a089f..c41a80c6 100644
--- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsTextHtmlSerializer.java
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsTextHtmlSerializer.java
@@ -1,8 +1,9 @@
-package gov.nasa.pds.api.registry.serializer;
+package gov.nasa.pds.api.registry.view;
 
 import java.io.IOException;
 import java.io.OutputStream;
 import java.io.OutputStreamWriter;
+import java.nio.charset.Charset;
 
 import org.springframework.http.HttpInputMessage;
 import org.springframework.http.HttpOutputMessage;
@@ -36,9 +37,10 @@ protected void writeInternal(PdsProducts t, HttpOutputMessage outputMessage)
 	{
         ObjectMapper mapper = new ObjectMapper();
         OutputStream os = outputMessage.getBody();
-        OutputStreamWriter wr = new OutputStreamWriter(os);
+        OutputStreamWriter wr = new OutputStreamWriter(os, Charset.defaultCharset());
         mapper.setSerializationInclusion(Include.NON_NULL);
-        wr.write("
JSON as text
");
+        Utilities.fix (t.getSummary());
+       wr.write("
JSON as text
");
         wr.write(mapper.writerWithDefaultPrettyPrinter().writeValueAsString(t));
         wr.write("
");
         wr.close();
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsXMLSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsXMLSerializer.java
similarity index 96%
rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsXMLSerializer.java
rename to service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsXMLSerializer.java
index e48b2d10..58c12021 100644
--- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/PdsProductsXMLSerializer.java
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/PdsProductsXMLSerializer.java
@@ -1,4 +1,4 @@
-package gov.nasa.pds.api.registry.serializer;
+package gov.nasa.pds.api.registry.view;
 
 import gov.nasa.pds.model.PdsProduct;
 import gov.nasa.pds.model.PdsProducts;
@@ -42,6 +42,7 @@ protected void writeInternal(PdsProducts products, HttpOutputMessage outputMessa
 			OutputStream outputStream = outputMessage.getBody();
 			XmlMapper mapper = new XmlMapper();
 			XMLOutputFactory outputFactory = XMLOutputFactory.newFactory();
+	        Utilities.fix (products.getSummary());
 			outputFactory.setProperty("javax.xml.stream.isRepairingNamespaces", true);
 			outputFactory.setProperty(WstxInputProperties.P_RETURN_NULL_FOR_DEFAULT_NAMESPACE, true);
 			XMLStreamWriter writer = outputFactory.createXMLStreamWriter(outputStream);
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/view/Utilities.java b/service/src/main/java/gov/nasa/pds/api/registry/view/Utilities.java
new file mode 100644
index 00000000..419b75df
--- /dev/null
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/Utilities.java
@@ -0,0 +1,22 @@
+package gov.nasa.pds.api.registry.view;
+
+import java.util.ArrayList;
+import java.util.List;
+
+import gov.nasa.pds.api.registry.exceptions.UnsupportedSearchProperty;
+import gov.nasa.pds.api.registry.model.SearchUtil;
+import gov.nasa.pds.model.Summary;
+
+class Utilities
+{
+	static void fix (Summary summary)
+	{
+		List fixed = new ArrayList(summary.getProperties().size());
+		for (String prop : summary.getProperties())
+		{
+			try { fixed.add(SearchUtil.openPropertyToJsonProperty(prop)); }
+			catch (UnsupportedSearchProperty e) { fixed.add(prop); }
+		}
+		summary.setProperties(fixed);
+	}
+}
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/WyriwygSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/WyriwygSerializer.java
similarity index 98%
rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/WyriwygSerializer.java
rename to service/src/main/java/gov/nasa/pds/api/registry/view/WyriwygSerializer.java
index 96aa5118..78623d41 100644
--- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/WyriwygSerializer.java
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/WyriwygSerializer.java
@@ -1,4 +1,4 @@
-package gov.nasa.pds.api.registry.serializer;
+package gov.nasa.pds.api.registry.view;
 
 import java.io.IOException;
 import java.io.Writer;
diff --git a/service/src/main/java/gov/nasa/pds/api/registry/serializer/XmlErrorMessageSerializer.java b/service/src/main/java/gov/nasa/pds/api/registry/view/XmlErrorMessageSerializer.java
similarity index 96%
rename from service/src/main/java/gov/nasa/pds/api/registry/serializer/XmlErrorMessageSerializer.java
rename to service/src/main/java/gov/nasa/pds/api/registry/view/XmlErrorMessageSerializer.java
index f4541fa1..9f306688 100644
--- a/service/src/main/java/gov/nasa/pds/api/registry/serializer/XmlErrorMessageSerializer.java
+++ b/service/src/main/java/gov/nasa/pds/api/registry/view/XmlErrorMessageSerializer.java
@@ -1,4 +1,4 @@
-package gov.nasa.pds.api.registry.serializer;
+package gov.nasa.pds.api.registry.view;
 
 import java.io.IOException;
 import java.io.OutputStreamWriter;
diff --git a/service/src/main/resources/application.properties b/service/src/main/resources/application.properties
index faef02ce..f966e6ee 100644
--- a/service/src/main/resources/application.properties
+++ b/service/src/main/resources/application.properties
@@ -1,6 +1,6 @@
 springfox.documentation.swagger.v2.path=/api-docs
 server.contextPath=/
-server.port=8081
+server.port=8080
 server.use-forward-headers=true
 
 #spring.jackson.date-format=io.swagger.RFC3339DateFormat
@@ -28,4 +28,7 @@ openSearch.ssl=true
 openSearch.sslCertificateCNVerification=false
 
 # Only show products with following archive statuses
-filter.archiveStatus=archived,certified
\ No newline at end of file
+filter.archiveStatus=archived,certified
+
+# source version from maven
+registry.service.version=${project.version}
diff --git a/service/src/main/resources/application.properties.all b/service/src/main/resources/application.properties.all
index 1ffc694e..e84eb4c7 100644
--- a/service/src/main/resources/application.properties.all
+++ b/service/src/main/resources/application.properties.all
@@ -17,7 +17,7 @@ server.ssl.key-store=classpath:keystore.p12
 server.ssl.key-store-type=PKCS12
 
 # note the port is mandatory even when it is default :80 or :443
-openSearch.host=es:9200
+openSearch.host=localhost:9200
 openSearch.registryIndex=registry
 openSearch.registryRefIndex=registry-refs
 openSearch.timeOutSeconds=60
@@ -28,4 +28,7 @@ openSearch.ssl=true
 openSearch.sslCertificateCNVerification=false
 
 # Only show products with following archive statuses
-filter.archiveStatus=archived,certified,staged
\ No newline at end of file
+filter.archiveStatus=archived,certified
+
+# source version from maven
+registry.service.version=${project.version}
\ No newline at end of file
diff --git a/service/src/main/resources/application.properties.aws b/service/src/main/resources/application.properties.aws
index 60d3945e..12cdc312 100644
--- a/service/src/main/resources/application.properties.aws
+++ b/service/src/main/resources/application.properties.aws
@@ -24,3 +24,6 @@ openSearch.timeOutSeconds=60
 openSearch.username=
 openSearch.password=
 openSearch.ssl=true
+
+# source version from maven
+registry.service.version=${project.version}
diff --git a/service/src/main/resources/application.properties.docker b/service/src/main/resources/application.properties.docker
index c6462000..3480353a 100644
--- a/service/src/main/resources/application.properties.docker
+++ b/service/src/main/resources/application.properties.docker
@@ -27,3 +27,6 @@ openSearch.ssl=false
 
 # Only show products with following archive statuses
 filter.archiveStatus=archived,certified
+
+# source version from maven
+registry.service.version=${project.version}
diff --git a/service/src/main/resources/application.properties.local b/service/src/main/resources/application.properties.local
index 231bb819..47e82db3 100644
--- a/service/src/main/resources/application.properties.local
+++ b/service/src/main/resources/application.properties.local
@@ -14,4 +14,7 @@ openSearch.registryRefIndex=registry-refs
 openSearch.timeOutSeconds=60
 openSearch.username=
 openSearch.password=
-openSearch.ssl=false
\ No newline at end of file
+openSearch.ssl=false
+
+# source version from maven
+registry.service.version=${project.version}
diff --git a/service/src/test/java/gov/nasa/pds/api/registry/opensearch/Antlr4SearchListenerTest.java b/service/src/test/java/gov/nasa/pds/api/registry/opensearch/Antlr4SearchListenerTest.java
index e3e2ed75..798923ca 100644
--- a/service/src/test/java/gov/nasa/pds/api/registry/opensearch/Antlr4SearchListenerTest.java
+++ b/service/src/test/java/gov/nasa/pds/api/registry/opensearch/Antlr4SearchListenerTest.java
@@ -15,9 +15,9 @@
 import org.junit.jupiter.api.Test;
 import org.junit.jupiter.api.function.Executable;
 
-import gov.nasa.pds.api.registry.business.Antlr4SearchListener;
 import gov.nasa.pds.api.registry.lexer.SearchLexer;
 import gov.nasa.pds.api.registry.lexer.SearchParser;
+import gov.nasa.pds.api.registry.model.Antlr4SearchListener;
 
 public class Antlr4SearchListenerTest
 {
diff --git a/service/src/test/java/gov/nasa/pds/api/registry/opensearch/RegistrySearchRequestBuilderTest.java b/service/src/test/java/gov/nasa/pds/api/registry/opensearch/RegistrySearchRequestBuilderTest.java
index 7912dc16..2884f7f3 100644
--- a/service/src/test/java/gov/nasa/pds/api/registry/opensearch/RegistrySearchRequestBuilderTest.java
+++ b/service/src/test/java/gov/nasa/pds/api/registry/opensearch/RegistrySearchRequestBuilderTest.java
@@ -102,7 +102,7 @@ void testGetSearchRequest() {
 			for (Entry queryEntry : RegistrySearchRequestBuilderTest.queryMap.entrySet()) {
 				
 				queryString = queryEntry.getKey();
-				List fields = new ArrayList(Arrays.asList("title","ops:Label_File_Info.ops:md5_checksum"));
+				List fields = Arrays.asList("title","ops:Label_File_Info.ops:md5_checksum");
 				searchRequest = null; /* FIXME: later
 				        new SearchRequestBuilder().build();
 						this.requestBuilder.getSearchCollectionRequest(
diff --git a/service/ut/base_ut.py b/service/ut/base_ut.py
index 86aa2f21..03abe78c 100644
--- a/service/ut/base_ut.py
+++ b/service/ut/base_ut.py
@@ -2,8 +2,18 @@
 import helpers
 import unittest
 
+def test_bad_group():
+    ep = '/gid/notreal'
+    status,data = helpers.fetch_kvp_json (helpers.make_url (ep))
+    assert 406 == status
+    assert 'message' in data
+    assert 'request' in data
+    assert data['message'].startswith ("Unknown group 'notreal'. All known groups:")
+    assert data['request'] == ep
+    return
+
 def test_bad_lidvid():
-    ep = '/bundles/notreal'
+    ep = '/uid/notreal'
     status,data = helpers.fetch_kvp_json (helpers.make_url (ep))
     assert 404 == status
     assert 'message' in data
@@ -12,9 +22,80 @@ def test_bad_lidvid():
     assert data['request'] == ep
     return
 
+class TestAny(unittest.TestCase):
+    def test_products(self):
+        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/gid/any'))
+        self.assertEqual (200, status)
+        self.assertIn ('summary', resp)
+        self.assertIn ('hits', resp['summary'])
+        self.assertEqual (25, resp['summary']['hits'])
+        self.assertIn ('data', resp)
+        self.assertEqual (resp['summary']['hits'], len(resp['data']))
+        self.assertIn ('lidvid', resp['data'][0])
+        return resp['data'][-1]['lidvid']
+
+    def test_lidvid(self):
+        lidvid = self.test_products()
+        status,resp = helpers.fetch_kvp_json (helpers.make_url
+                                              (f'/uid/{lidvid}'))
+        self.assertEqual (200, status)
+        self.assertIn ('lidvid', resp)
+        self.assertEqual (lidvid, resp['lidvid'])
+        return
+
+    def test_lidvid_latest(self):
+        lidvid = self.test_products()
+        status,resp = helpers.fetch_kvp_json(helpers.make_url
+                                             (f'/uid/{lidvid}/latest'))
+        self.assertEqual (200, status)
+        self.assertIn ('lidvid', resp)
+        self.assertEqual (lidvid, resp['lidvid'])
+        return
+
+    def test_lidvid_all(self):
+        lidvid = self.test_products()
+        status,resp = helpers.fetch_kvp_json (helpers.make_url
+                                              (f'/uid/{lidvid}/all'))
+        self.assertEqual (200, status)
+        self.assertIn ('summary', resp)
+        self.assertIn ('hits', resp['summary'])
+        self.assertEqual (1, resp['summary']['hits'])
+        self.assertIn ('data', resp)
+        self.assertEqual (resp['summary']['hits'], len(resp['data']))
+        self.assertIn ('lidvid', resp['data'][0])
+        self.assertEqual (lidvid, resp['data'][0]['lidvid'])
+        return
+
+    def test_collections(self):
+        lidvid = self.test_products()
+        status,resp = helpers.fetch_kvp_json (helpers.make_url
+                                              (f'/gid/collection/referencing/{lidvid}'))
+        self.assertEqual (200, status)
+        self.assertIn ('summary', resp)
+        self.assertIn ('hits', resp['summary'])
+        self.assertEqual (1, resp['summary']['hits'])
+        self.assertIn ('data', resp)
+        self.assertEqual (resp['summary']['hits'], len(resp['data']))
+        self.assertIn ('lidvid', resp['data'][0])
+        return
+
+    def test_bundles(self):
+        lidvid = self.test_products()
+        status,resp = helpers.fetch_kvp_json (helpers.make_url
+                                              (f'/gid/bundle/referencing/{lidvid}'))
+        self.assertEqual (200, status)
+        self.assertIn ('summary', resp)
+        self.assertIn ('hits', resp['summary'])
+        self.assertEqual (1, resp['summary']['hits'])
+        self.assertIn ('data', resp)
+        self.assertEqual (resp['summary']['hits'], len(resp['data']))
+        self.assertIn ('lidvid', resp['data'][0])
+        return
+    pass
+
 class TestBundles(unittest.TestCase):
     def test_bundles(self):
-        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/bundles'))
+        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/gid/bundle'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -27,7 +108,7 @@ def test_bundles(self):
     def test_lidvid(self):
         lidvid = self.test_bundles()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid))
+                                              (f'/uid/{lidvid}'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -36,7 +117,7 @@ def test_lidvid(self):
     def test_lidvid_latest(self):
         lidvid = self.test_bundles()
         status,resp = helpers.fetch_kvp_json(helpers.make_url
-                                             ('/bundles/' + lidvid + '/latest'))
+                                             (f'/uid/{lidvid}/latest'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -45,7 +126,7 @@ def test_lidvid_latest(self):
     def test_lidvid_all(self):
         lidvid = self.test_bundles()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid + '/all'))
+                                              (f'/uid/{lidvid}/all'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -58,8 +139,8 @@ def test_lidvid_all(self):
 
     def test_collections(self):
         lidvid = self.test_bundles()
-        status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid + '/collections'))
+        status,resp = helpers.fetch_kvp_json \
+                   (helpers.make_url (f'/uid/{lidvid}/referencing/collection'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -71,8 +152,8 @@ def test_collections(self):
 
     def test_collections_latest(self):
         lidvid = self.test_bundles()
-        status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid + '/collections/latest'))
+        status,resp = helpers.fetch_kvp_json \
+            (helpers.make_url (f'/uid/{lidvid}/referencing/collection/latest'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -84,8 +165,8 @@ def test_collections_latest(self):
 
     def test_collections_all(self):
         lidvid = self.test_bundles()
-        status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid + '/collections/all'))
+        status,resp = helpers.fetch_kvp_json \
+               (helpers.make_url (f'/uid/{lidvid}/referencing/collection/all'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -97,8 +178,8 @@ def test_collections_all(self):
 
     def test_products(self):
         lidvid = self.test_bundles()
-        status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/bundles/' + lidvid + '/products'))
+        status,resp = helpers.fetch_kvp_json\
+                      (helpers.make_url (f'/uid/{lidvid}/referencing/product'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -112,8 +193,8 @@ def test_products(self):
 class TestCollections(unittest.TestCase):
     def test_bundles(self):
         lidvid = self.test_collections()
-        status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid + '/bundles'))
+        status,resp = helpers.fetch_kvp_json \
+                      (helpers.make_url (f'/uid/{lidvid}/referencing/bundle'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -124,7 +205,8 @@ def test_bundles(self):
         return
 
     def test_collections(self):
-        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/collections'))
+        status,resp = helpers.fetch_kvp_json \
+                      (helpers.make_url ('/gid/collection'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -137,7 +219,7 @@ def test_collections(self):
     def test_lidvid(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid))
+                                              (f'/uid/{lidvid}'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -146,7 +228,7 @@ def test_lidvid(self):
     def test_lidvid_latest(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json(helpers.make_url
-                                             ('/collections/' + lidvid + '/latest'))
+                                             (f'/uid/{lidvid}/latest'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -155,7 +237,7 @@ def test_lidvid_latest(self):
     def test_lidvid_all(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid + '/all'))
+                                              (f'/uid/{lidvid}/all'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -169,7 +251,7 @@ def test_lidvid_all(self):
     def test_products(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid + '/products'))
+                                              (f'/uid/{lidvid}/referencing/product'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -182,7 +264,7 @@ def test_products(self):
     def test_products_latest(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid + '/products/latest'))
+                                              (f'/uid/{lidvid}/referencing/product/latest'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -195,7 +277,7 @@ def test_products_latest(self):
     def test_products_all(self):
         lidvid = self.test_collections()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/collections/' + lidvid + '/products/all'))
+                                              (f'/uid/{lidvid}/referencing/product/all'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -208,11 +290,11 @@ def test_products_all(self):
 
 class TestProducts(unittest.TestCase):
     def test_products(self):
-        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/products'))
+        status,resp = helpers.fetch_kvp_json (helpers.make_url ('/gid/product'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
-        self.assertEqual (25, resp['summary']['hits'])
+        self.assertEqual (21, resp['summary']['hits'])
         self.assertIn ('data', resp)
         self.assertEqual (resp['summary']['hits'], len(resp['data']))
         self.assertIn ('lidvid', resp['data'][0])
@@ -221,7 +303,7 @@ def test_products(self):
     def test_lidvid(self):
         lidvid = self.test_products()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/products/' + lidvid))
+                                              (f'/uid/{lidvid}'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -230,7 +312,7 @@ def test_lidvid(self):
     def test_lidvid_latest(self):
         lidvid = self.test_products()
         status,resp = helpers.fetch_kvp_json(helpers.make_url
-                                             ('/products/' + lidvid + '/latest'))
+                                             (f'/uid/{lidvid}/latest'))
         self.assertEqual (200, status)
         self.assertIn ('lidvid', resp)
         self.assertEqual (lidvid, resp['lidvid'])
@@ -239,7 +321,7 @@ def test_lidvid_latest(self):
     def test_lidvid_all(self):
         lidvid = self.test_products()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/products/' + lidvid + '/all'))
+                                              (f'/uid/{lidvid}/all'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -253,7 +335,7 @@ def test_lidvid_all(self):
     def test_collections(self):
         lidvid = self.test_products()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/products/' + lidvid + '/collections'))
+                                              (f'/gid/collection/referencing/{lidvid}'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
@@ -266,7 +348,7 @@ def test_collections(self):
     def test_bundles(self):
         lidvid = self.test_products()
         status,resp = helpers.fetch_kvp_json (helpers.make_url
-                                              ('/products/' + lidvid + '/bundles'))
+                                              (f'/gid/bundle/referencing/{lidvid}'))
         self.assertEqual (200, status)
         self.assertIn ('summary', resp)
         self.assertIn ('hits', resp['summary'])
diff --git a/service/ut/helpers/__init__.py b/service/ut/helpers/__init__.py
index 2cc7f63b..aa0d04cc 100644
--- a/service/ut/helpers/__init__.py
+++ b/service/ut/helpers/__init__.py
@@ -3,8 +3,9 @@
 import requests
 
 def fetch_kvp_json (url:str):
-    result = requests.get(url + '?fields=lidvid',
-                          headers={'Accept':'application/kvp+json'})
+    url += '?fields=lidvid'
+    print ('url:', url)
+    result = requests.get(url, headers={'Accept':'application/kvp+json'})
     return result.status_code,result.json()
 
 def make_url (endpoint:str)->str: