From 9159189804b5737ed810766f7498a5c065867586 Mon Sep 17 00:00:00 2001 From: =?UTF-8?q?Jack=20=28=EC=A0=95=ED=99=98=29?= Date: Tue, 4 Jun 2019 13:09:48 -0700 Subject: [PATCH 01/16] Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) [maven-release-plugin] prepare release 5.0.0-pr1 [maven-release-plugin] prepare for next development iteration Renamed graphQL file to match test (#1002) [maven-release-plugin] prepare release 5.0.0-pr2 [maven-release-plugin] prepare for next development iteration Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment Manager transacton manually (#1021) * Manager transacton manually * Add readonly Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause function name fixed to enableISO8601Dates (#1052) Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView Fixed issues with rebase Add auto configuration for aggregation store (#1087) * Added autoconfiguration for QueryEngineFactory * Unified class scanning. Started cleaning up datastores so they only register the entities they manage * Full build passes * Minor cleanup * Minor refactoring * Added EntityManagerFactory bean configuratino * Refactored class scanning for Elide standalone * Updated spring boot starter pom * Removed @Entity from all metadata models. Started cleaning up entity dictionary entity registration * Broken implementation. Just checking in so I can revert if needed. * All tests pass * Added unit tests * Minor cleanup * One more fix * Fixed broken tests * Added package include support back * Class scanning for annotations ignores inherited * Added a test based on inspection comments * Inspection comment fix * Changed initalization of MetadataStore * More inspection rework * Turned back on OWASP scanning * More rework remove @Inherited (#1092) Support Non JPA Entity in AggregationDataStore (#1051) * Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) * [maven-release-plugin] prepare release 5.0.0-pr1 * [maven-release-plugin] prepare for next development iteration * Renamed graphQL file to match test (#1002) * [maven-release-plugin] prepare release 5.0.0-pr2 * [maven-release-plugin] prepare for next development iteration * Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework * Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment * Manager transacton manually * Add readonly * Manager transacton manually (#1021) * Manager transacton manually * Add readonly * Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set * AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments * some rework * use getTimeDimension() * change exception * Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments * ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause * View Design * Add tests and cleanup * rename annotation * function name fixed to enableISO8601Dates (#1052) * fix bugs * Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved * merge annotations * don't group by view relationship * Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix * Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute * MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check * SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments * Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView * integrate view with aggregation and metadata * remove includeField * remove @view * Use NonEntityDictionary * remove id * revert access changes * fix JPA entity check * remove @Entity from analyticViews * use table name as relationship type id * revert NonEntitydictinoary * tiny rework * Integration tests * Add jsonapi ittest * aggregation data store doesn't manage jpa entities * address comments fix integration dependencies (#1093) [maven-release-plugin] prepare release 5.0.0-pr3 [maven-release-plugin] prepare for next development iteration Fixed elide standalone pom from rebase Fixed minor bug in rebase Fixed rebase Improving class scanning performance for MetadataStore (#1117) Enable elide5 travis builds (#1129) * Move repeated @Sql annotations to class level (#1119) * Turning on travis builds with code coverage for Elide 5.x * Fixing security issue in spring-boot-web Co-authored-by: Brutus5000 Fix sorting and ambiguous join issue (#1127) * Added sorting on aggregated metric based on latest elide-5.x * Fix ambiguity problem * update comments * fix codacy * refactor generateColumnReference * update comment * address comments * test cleanup * update unittest * fix elide core alias * QueryValidatorTest * EntityProjectionTranslatorTest * go joinFragment approach * delete jointrienode Support no metric query (#1137) [maven-release-plugin] prepare release 5.0.0-pr4 [maven-release-plugin] prepare for next development iteration Check dependency injection (#1138) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Adding support for dependency injection of Checks. Added test injection classes * Unit tests pass * Tests pass * Removed Initializer Concept Co-authored-by: Brutus5000 Fix travis log length (#1140) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Removed unnecessary request/response logging (to shorten travis logs) * Address inspection comments * Address inspection comments * Address inspection comments * Removed logging of graphQL model building to shorten length * Fixed compilation error Co-authored-by: Brutus5000 [maven-release-plugin] prepare release 5.0.0-pr5 [maven-release-plugin] prepare for next development iteration --- .travis.yml | 1 + checkstyle-suppressions.xml | 2 +- elide-annotations/pom.xml | 3 +- .../com/yahoo/elide/annotation/Exclude.java | 2 - .../com/yahoo/elide/annotation/Include.java | 2 - elide-contrib/elide-swagger/pom.xml | 6 +- elide-contrib/elide-test-helpers/pom.xml | 2 +- .../testhelpers/graphql/GraphQLDSL.java | 4 + .../testhelpers/graphql/elements/Field.java | 2 +- elide-contrib/pom.xml | 4 +- elide-core/pom.xml | 10 +- .../src/main/java/com/yahoo/elide/Elide.java | 9 + .../com/yahoo/elide/core/ArgumentType.java | 23 + .../yahoo/elide/core/CheckInstantiator.java | 9 +- .../elide/core/DataStoreTransaction.java | 78 +- .../com/yahoo/elide/core/EntityBinding.java | 40 +- .../yahoo/elide/core/EntityDictionary.java | 171 +++- .../com/yahoo/elide/core/Initializer.java | 22 - .../yahoo/elide/core/PersistentResource.java | 341 ++++--- .../com/yahoo/elide/core/RequestScope.java | 28 +- .../com/yahoo/elide/core/TimedFunction.java | 40 + .../datastore/inmemory/HashMapDataStore.java | 4 +- .../inmemory/HashMapStoreTransaction.java | 24 +- .../inmemory/InMemoryStoreTransaction.java | 159 +-- .../datastore/wrapped/TransactionWrapper.java | 33 +- .../elide/core/filter/FilterPredicate.java | 33 +- .../filter/dialect/RSQLFilterDialect.java | 1 - .../expression/AndFilterExpression.java | 26 + .../filter/expression/OrFilterExpression.java | 26 + .../elide/core/pagination/Pagination.java | 19 +- .../com/yahoo/elide/core/sort/Sorting.java | 2 + .../elide/extensions/PatchRequestScope.java | 2 + .../elide/jsonapi/EntityProjectionMaker.java | 383 +++++++ .../processors/IncludedProcessor.java | 20 +- .../yahoo/elide/jsonapi/models/Resource.java | 13 +- .../jsonapi/models/ResourceIdentifier.java | 5 +- .../state/CollectionTerminalState.java | 30 +- .../elide/parsers/state/RecordState.java | 116 +-- .../state/RelationshipTerminalState.java | 11 +- .../yahoo/elide/parsers/state/StartState.java | 36 +- .../com/yahoo/elide/request/Argument.java | 32 + .../com/yahoo/elide/request/Attribute.java | 43 + .../yahoo/elide/request/EntityProjection.java | 261 +++++ .../com/yahoo/elide/request/Relationship.java | 48 + .../elide/security/FilterExpressionCheck.java | 10 +- .../com/yahoo/elide/utils/ClassScanner.java | 15 +- .../elide/core/EntityDictionaryTest.java | 72 +- .../com/yahoo/elide/core/LifeCycleTest.java | 70 +- .../elide/core/PermissionAnnotationTest.java | 3 +- .../core/PersistenceResourceTestSetup.java | 4 +- .../elide/core/PersistentResourceTest.java | 774 ++++++++++---- .../com/yahoo/elide/core/TestDictionary.java | 70 ++ .../com/yahoo/elide/core/TestInjector.java | 33 + .../yahoo/elide/core/TestRequestScope.java | 53 + .../yahoo/elide/core/UpdateOnCreateTest.java | 365 +++++-- .../InMemoryStoreTransactionTest.java | 308 +++--- .../wrapped/TransactionWrapperTest.java | 35 +- .../elide/core/utils/ClassScannerTest.java | 11 + .../jsonapi/EntityProjectionMakerTest.java | 745 ++++++++++++++ .../com/yahoo/elide/jsonapi/JsonApiTest.java | 65 +- .../processors/IncludedProcessorTest.java | 59 +- .../expression/CanPaginateVisitorTest.java | 27 +- .../PermissionExpressionVisitorTest.java | 3 +- ...rmissionToFilterExpressionVisitorTest.java | 3 +- .../security/PermissionExecutorTest.java | 11 +- .../PermissionExpressionBuilderTest.java | 3 +- .../elide-datastore-aggregation/.gitignore | 1 + .../elide-datastore-aggregation/pom.xml | 203 ++++ .../aggregation/AggregationDataStore.java | 72 ++ .../AggregationDataStoreTransaction.java | 71 ++ .../EntityProjectionTranslator.java | 206 ++++ .../datastores/aggregation/QueryEngine.java | 72 ++ .../aggregation/QueryEngineFactory.java | 15 + .../aggregation/QueryValidator.java | 155 +++ .../aggregation/annotation/Cardinality.java | 44 + .../annotation/CardinalitySize.java | 34 + .../aggregation/annotation/FriendlyName.java | 23 + .../aggregation/annotation/Meta.java | 25 + .../annotation/MetricAggregation.java | 24 + .../annotation/MetricComputation.java | 62 ++ .../aggregation/annotation/Temporal.java | 47 + .../annotation/TimeGrainDefinition.java | 29 + .../filter/visitor/FilterConstraints.java | 111 ++ .../visitor/SplitFilterExpressionVisitor.java | 238 +++++ .../aggregation/metadata/MetaDataStore.java | 183 ++++ .../metadata/enums/Aggregation.java | 15 + .../aggregation/metadata/enums/Format.java | 14 + .../aggregation/metadata/enums/Tag.java | 13 + .../aggregation/metadata/enums/ValueType.java | 19 + .../metric/MetricFunctionInvocation.java | 69 ++ .../metadata/models/AnalyticView.java | 48 + .../aggregation/metadata/models/Column.java | 87 ++ .../aggregation/metadata/models/DataType.java | 58 ++ .../metadata/models/Dimension.java | 24 + .../metadata/models/FunctionArgument.java | 39 + .../aggregation/metadata/models/Metric.java | 50 + .../metadata/models/MetricFunction.java | 114 +++ .../metadata/models/RelationshipType.java | 22 + .../aggregation/metadata/models/Table.java | 148 +++ .../metadata/models/TimeDimension.java | 44 + .../metadata/models/TimeDimensionGrain.java | 35 + .../aggregation/query/ColumnProjection.java | 91 ++ .../datastores/aggregation/query/Query.java | 57 ++ .../query/TimeDimensionProjection.java | 86 ++ .../queryengines/AbstractEntityHydrator.java | 203 ++++ .../aggregation/queryengines/StitchList.java | 162 +++ .../queryengines/sql/SQLEntityHydrator.java | 83 ++ .../queryengines/sql/SQLQuery.java | 53 + .../queryengines/sql/SQLQueryConstructor.java | 546 ++++++++++ .../queryengines/sql/SQLQueryEngine.java | 299 ++++++ .../sql/SQLQueryEngineFactory.java | 30 + .../sql/annotation/FromSubquery.java | 28 + .../sql/annotation/FromTable.java | 28 + .../queryengines/sql/annotation/JoinTo.java | 37 + .../sql/metadata/SQLAnalyticView.java | 34 + .../queryengines/sql/metadata/SQLColumn.java | 42 + .../queryengines/sql/metadata/SQLTable.java | 51 + .../sql/metric/SQLMetricFunction.java | 62 ++ .../sql/metric/functions/SqlAvg.java | 19 + .../sql/metric/functions/SqlMax.java | 19 + .../sql/metric/functions/SqlMin.java | 19 + .../sql/metric/functions/SqlSum.java | 19 + .../sql/query/SQLQueryTemplate.java | 111 ++ .../aggregation/time/TimeGrain.java | 26 + .../EntityProjectionTranslatorTest.java | 164 +++ .../aggregation/QueryValidatorTest.java | 204 ++++ .../aggregation/example/Continent.java | 28 + .../aggregation/example/Country.java | 73 ++ .../aggregation/example/CountryView.java | 80 ++ .../example/CountryViewNested.java | 54 + .../aggregation/example/Player.java | 34 + .../aggregation/example/PlayerStats.java | 235 +++++ .../aggregation/example/PlayerStatsView.java | 46 + .../example/PlayerStatsWithView.java | 223 +++++ .../aggregation/example/SubCountry.java | 61 ++ .../aggregation/example/VideoGame.java | 79 ++ .../filter/visitor/FilterConstraintsTest.java | 70 ++ .../SplitFilterExpressionVisitorTest.java | 176 ++++ .../AggregationDataStoreTestHarness.java | 42 + .../aggregation/framework/SQLUnitTest.java | 99 ++ .../AggregationDataStoreIntegrationTest.java | 947 ++++++++++++++++++ .../metadata/MetaDataStoreTest.java | 47 + .../queryengines/sql/QueryEngineTest.java | 681 +++++++++++++ .../queryengines/sql/SubselectTest.java | 267 +++++ .../queryengines/sql/ViewTest.java | 261 +++++ .../test/resources/META-INF/persistence.xml | 28 + .../src/test/resources/continent.csv | 3 + .../src/test/resources/country.csv | 3 + .../src/test/resources/create_tables.sql | 37 + .../src/test/resources/player.csv | 4 + .../src/test/resources/player_stats.csv | 4 + .../src/test/resources/video_games.csv | 5 + .../elide-datastore-hibernate/pom.xml | 4 +- .../elide/core/filter/FilterTranslator.java | 57 +- .../hql/AbstractHQLQueryBuilder.java | 21 +- .../hql/RootCollectionFetchQueryBuilder.java | 2 +- .../RootCollectionPageTotalsQueryBuilder.java | 4 +- .../hql/SubCollectionFetchQueryBuilder.java | 2 +- .../SubCollectionPageTotalsQueryBuilder.java | 2 +- .../core/filter/FilterTranslatorTest.java | 3 +- .../hql/AbstractHQLQueryBuilderTest.java | 4 +- .../RootCollectionFetchQueryBuilderTest.java | 24 +- ...tCollectionPageTotalsQueryBuilderTest.java | 19 +- .../SubCollectionFetchQueryBuilderTest.java | 6 +- ...bCollectionPageTotalsQueryBuilderTest.java | 18 +- .../elide-datastore-hibernate3/pom.xml | 8 +- .../hibernate3/HibernateTransaction.java | 76 +- .../elide-datastore-hibernate5/pom.xml | 8 +- .../hibernate5/HibernateTransaction.java | 78 +- .../elide-datastore-inmemorydb/pom.xml | 4 +- .../inmemory/HashMapDataStoreTest.java | 22 +- elide-datastore/elide-datastore-jpa/pom.xml | 8 +- .../transaction/AbstractJpaTransaction.java | 76 +- .../elide-datastore-multiplex/pom.xml | 8 +- .../multiplex/MultiplexManager.java | 15 +- .../multiplex/MultiplexTransaction.java | 61 +- .../multiplex/MultiplexWriteTransaction.java | 30 +- .../multiplex/MultiplexManagerTest.java | 33 +- .../datastores/multiplex/TestDataStore.java | 17 +- .../bridgeable/BridgeableRedisStore.java | 64 +- elide-datastore/elide-datastore-noop/pom.xml | 2 +- .../datastores/noop/NoopTransaction.java | 28 +- .../java/com/yahoo/elide/beans/NoopBean.java | 2 +- .../datastores/noop/NoopTransactionTest.java | 21 +- .../elide-datastore-search/pom.xml | 6 +- .../search/SearchDataTransaction.java | 22 +- .../datastores/search/DataStoreLoadTest.java | 114 ++- elide-datastore/pom.xml | 5 +- elide-example-models/pom.xml | 2 +- .../elide-blog-example-resteasy/pom.xml | 6 +- elide-example/elide-blog-example/pom.xml | 8 +- .../elide-hibernate3-mysql-example/pom.xml | 6 +- elide-example/pom.xml | 4 +- elide-graphql/pom.xml | 6 +- .../java/com/yahoo/elide/graphql/Entity.java | 67 +- .../com/yahoo/elide/graphql/Environment.java | 2 +- .../elide/graphql/GraphQLConversionUtils.java | 40 + .../yahoo/elide/graphql/GraphQLEndpoint.java | 2 - .../elide/graphql/GraphQLRequestScope.java | 19 +- .../java/com/yahoo/elide/graphql/KeyWord.java | 47 + .../com/yahoo/elide/graphql/ModelBuilder.java | 13 +- .../graphql/PersistentResourceFetcher.java | 208 ++-- .../com/yahoo/elide/graphql/QueryRunner.java | 25 +- .../containers/ConnectionContainer.java | 10 +- .../graphql/containers/EdgesContainer.java | 6 +- .../graphql/containers/NodeContainer.java | 24 +- .../graphql/containers/PageInfoContainer.java | 19 +- .../graphql/containers/RootContainer.java | 22 +- .../graphql/parser/FragmentResolver.java | 139 +++ .../parser/GraphQLEntityProjectionMaker.java | 557 ++++++++++ .../graphql/parser/GraphQLProjectionInfo.java | 37 + .../graphql/parser/VariableResolver.java | 122 +++ .../elide/graphql/FetcherDeleteTest.java | 11 + .../yahoo/elide/graphql/FetcherFetchTest.java | 106 +- .../elide/graphql/FetcherRemoveTest.java | 12 + .../elide/graphql/FetcherReplaceTest.java | 7 +- .../elide/graphql/GraphQLEndpointTest.java | 241 ++++- .../com/yahoo/elide/graphql/GraphQLTest.java | 2 +- .../yahoo/elide/graphql/ModelBuilderTest.java | 22 + .../PersistentResourceFetcherTest.java | 34 +- .../graphqlEndpointTestModels/Author.java | 2 +- .../requests/fetch/aliasAmbiguous.graphql | 10 + .../requests/fetch/aliasAttribute.graphql | 9 + .../fetch/aliasPartialQueryAmbiguous.graphql | 23 + .../requests/fetch/aliasRelationship.graphql | 18 + .../fetch/aliasSameRelationship.graphql | 26 + ...agment.graphql => fragmentCorrect.graphql} | 0 .../requests/fetch/fragmentInline.graphql | 26 + .../requests/fetch/fragmentLoop.graphql | 31 + .../requests/fetch/fragmentUnknown.graphql | 31 + .../fetch/nestedCollectionFilter.graphql | 4 +- .../fetch/rootCollectionInvalidSort.graphql | 10 + .../requests/fetch/rootMultiple.graphql | 16 + .../requests/fetch/rootUnknownField.graphql | 18 + .../requests/fetch/variableDefinition.graphql | 18 + .../fetch/variableInvalidNonNull.graphql | 18 + .../fetch/variableUnknownReference.graphql | 16 + ...ls.graphql => replaceWithIdsFails.graphql} | 0 .../responses/fetch/aliasAttribute.json | 11 + .../responses/fetch/aliasRelationship.json | 22 + .../fetch/aliasSameRelationship.json | 32 + ...WithFragment.json => fragmentCorrect.json} | 0 .../responses/fetch/variableDefinition.json | 22 + elide-integration-tests/pom.xml | 4 +- .../EncodedErrorObjectsIT.java | 2 +- .../EncodedErrorResponsesIT.java | 2 +- .../VerboseEncodedErrorResponsesIT.java | 14 - .../AbstractApiResourceInitializer.java | 7 +- .../EncodedErrorResponsesTestBinder.java | 5 +- ...egrationTestApplicationResourceConfig.java | 9 +- .../ErrorObjectsTestBinder.java | 10 +- .../elide/initialization/IntegrationTest.java | 2 +- .../initialization/StandardTestBinder.java | 7 +- .../java/com/yahoo/elide/tests/GraphQLIT.java | 145 --- .../com/yahoo/elide/tests/ResourceIT.java | 11 +- .../graphQLFetchError.json | 10 - .../graphQLFetchErrorObjectEncoded.json | 7 + .../graphQLFetchErrorResponseEncoded.json | 5 + .../elide-spring-boot-autoconfigure/pom.xml | 23 +- .../spring/config/ElideAutoConfiguration.java | 28 +- .../spring/models/aggregation/Stats.java | 42 + .../models/{ => jpa}/ArtifactGroup.java | 2 +- .../models/{ => jpa}/ArtifactProduct.java | 2 +- .../models/{ => jpa}/ArtifactVersion.java | 2 +- .../spring/tests/AggregationStoreTest.java | 54 + .../elide/spring/tests/ControllerTest.java | 3 +- .../src/test/resources/application.yaml | 2 +- .../elide-spring-boot-starter/pom.xml | 40 +- elide-spring/pom.xml | 2 +- elide-standalone/pom.xml | 12 +- .../elide/standalone/ElideStandalone.java | 4 +- .../config/ElideStandaloneSettings.java | 10 +- pom.xml | 6 +- 273 files changed, 14861 insertions(+), 2013 deletions(-) create mode 100644 elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java delete mode 100644 elide-core/src/main/java/com/yahoo/elide/core/Initializer.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Argument.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Attribute.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Relationship.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java create mode 100644 elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/.gitignore create mode 100644 elide-datastore/elide-datastore-aggregation/pom.xml create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java create mode 100644 elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql rename elide-graphql/src/test/resources/graphql/requests/fetch/{fetchWithFragment.graphql => fragmentCorrect.graphql} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql create mode 100644 elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql rename elide-graphql/src/test/resources/graphql/requests/replace/{replaceWithidsFails.graphql => replaceWithIdsFails.graphql} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json rename elide-graphql/src/test/resources/graphql/responses/fetch/{fetchWithFragment.json => fragmentCorrect.json} (100%) create mode 100644 elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json delete mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json create mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json create mode 100644 elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json create mode 100644 elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactGroup.java (95%) rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactProduct.java (94%) rename elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/{ => jpa}/ArtifactVersion.java (92%) create mode 100644 elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java diff --git a/.travis.yml b/.travis.yml index 2daa1906d7..deb9b6fbfb 100644 --- a/.travis.yml +++ b/.travis.yml @@ -17,6 +17,7 @@ branches: only: - master + - elide-5.x - "/^[0-9]+\\.[0-9]+(\\.[0-9]+|-(alpha|beta)-[0-9]+)/" install: true diff --git a/checkstyle-suppressions.xml b/checkstyle-suppressions.xml index b79828e727..d69b31805c 100644 --- a/checkstyle-suppressions.xml +++ b/checkstyle-suppressions.xml @@ -10,7 +10,7 @@ "http://www.puppycrawl.com/dtds/suppressions_1_0.dtd"> - + diff --git a/elide-annotations/pom.xml b/elide-annotations/pom.xml index cc1af1cc97..177db5d79a 100644 --- a/elide-annotations/pom.xml +++ b/elide-annotations/pom.xml @@ -9,7 +9,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -53,5 +53,4 @@ - diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java index 043fa15b70..60dcce6e1e 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Exclude.java @@ -11,7 +11,6 @@ import static java.lang.annotation.ElementType.TYPE; import static java.lang.annotation.RetentionPolicy.RUNTIME; -import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.Target; @@ -20,6 +19,5 @@ */ @Target({METHOD, FIELD, TYPE, PACKAGE}) @Retention(RUNTIME) -@Inherited public @interface Exclude { } diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java index af1faadf13..4b90e31683 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Include.java @@ -9,7 +9,6 @@ import static java.lang.annotation.ElementType.TYPE; import static java.lang.annotation.RetentionPolicy.RUNTIME; -import java.lang.annotation.Inherited; import java.lang.annotation.Retention; import java.lang.annotation.Target; @@ -18,7 +17,6 @@ */ @Target({TYPE, PACKAGE}) @Retention(RUNTIME) -@Inherited public @interface Include { /** diff --git a/elide-contrib/elide-swagger/pom.xml b/elide-contrib/elide-swagger/pom.xml index 16b2c806fd..f845698b84 100644 --- a/elide-contrib/elide-swagger/pom.xml +++ b/elide-contrib/elide-swagger/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -42,7 +42,7 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -54,7 +54,7 @@ com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-contrib/elide-test-helpers/pom.xml b/elide-contrib/elide-test-helpers/pom.xml index c276f1cc1a..4663f250cc 100644 --- a/elide-contrib/elide-test-helpers/pom.xml +++ b/elide-contrib/elide-test-helpers/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java index 4b53d99e1d..ea101ddf7f 100644 --- a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java +++ b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java @@ -416,6 +416,10 @@ public static Selection field(String name, Arguments arguments, SelectionSet... return new Field(null, name, arguments, relayWrap(Arrays.asList(selectionSet))); } + public static Selection field(String alias, String name, Arguments arguments, SelectionSet... selectionSet) { + return new Field(alias, name, arguments, relayWrap(Arrays.asList(selectionSet))); + } + /** * Creates an attribute(scalar field) selection. * diff --git a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java index 902d575af3..09e18b329c 100644 --- a/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java +++ b/elide-contrib/elide-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java @@ -83,7 +83,7 @@ public String toGraphQLSpec() { @Override public String toResponse() { - if (selectionSet instanceof String) { + if (selectionSet instanceof String || selectionSet instanceof Number) { // scalar response field return String.format( "\"%s\":%s", diff --git a/elide-contrib/pom.xml b/elide-contrib/pom.xml index 8d60d09d90..e87677d9c9 100644 --- a/elide-contrib/pom.xml +++ b/elide-contrib/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -53,7 +53,7 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-core/pom.xml b/elide-core/pom.xml index b3f8cc04ff..0b166b78ff 100644 --- a/elide-core/pom.xml +++ b/elide-core/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -183,6 +183,13 @@ test + + com.google.inject + guice + 4.2.2 + test + + ch.qos.logback logback-classic @@ -201,6 +208,7 @@ org.eclipse.jetty jetty-webapp + ${version.jetty} test diff --git a/elide-core/src/main/java/com/yahoo/elide/Elide.java b/elide-core/src/main/java/com/yahoo/elide/Elide.java index f5d3fef134..df5fd02f2e 100644 --- a/elide-core/src/main/java/com/yahoo/elide/Elide.java +++ b/elide-core/src/main/java/com/yahoo/elide/Elide.java @@ -23,6 +23,7 @@ import com.yahoo.elide.core.exceptions.UnableToAddSerdeException; import com.yahoo.elide.extensions.JsonApiPatch; import com.yahoo.elide.extensions.PatchRequestScope; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.parsers.BaseVisitor; @@ -161,6 +162,8 @@ public ElideResponse get(String path, MultivaluedMap queryParams return handleRequest(true, opaqueUser, dataStore::beginReadTransaction, (tx, user) -> { JsonApiDocument jsonApiDoc = new JsonApiDocument(); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, queryParams, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new GetVisitor(requestScope); return visit(path, requestScope, visitor); }); @@ -179,6 +182,8 @@ public ElideResponse post(String path, String jsonApiDocument, Object opaqueUser return handleRequest(false, opaqueUser, dataStore::beginTransaction, (tx, user) -> { JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new PostVisitor(requestScope); return visit(path, requestScope, visitor); }); @@ -213,6 +218,8 @@ public ElideResponse patch(String contentType, String accept, handler = (tx, user) -> { JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new PatchVisitor(requestScope); return visit(path, requestScope, visitor); }; @@ -235,6 +242,8 @@ public ElideResponse delete(String path, String jsonApiDocument, Object opaqueUs ? new JsonApiDocument() : mapper.readJsonApiDocument(jsonApiDocument); RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings); + requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(), + requestScope).parsePath(path)); BaseVisitor visitor = new DeleteVisitor(requestScope); return visit(path, requestScope, visitor); }); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java b/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java new file mode 100644 index 0000000000..ffe0773280 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java @@ -0,0 +1,23 @@ +/* + * Copyright 2015, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.core; + +import lombok.Getter; + +/** + * Argument Type wraps an argument to the type of value it accepts. + */ +public class ArgumentType { + @Getter + private String name; + @Getter + private Class type; + + public ArgumentType(String name, Class type) { + this.name = name; + this.type = type; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java b/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java index 43773fe074..c1b25701ea 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/CheckInstantiator.java @@ -6,6 +6,7 @@ package com.yahoo.elide.core; +import com.yahoo.elide.Injector; import com.yahoo.elide.security.checks.Check; import java.util.Objects; @@ -25,7 +26,7 @@ public interface CheckInstantiator { */ default Check getCheck(EntityDictionary dictionary, String checkName) { Class checkCls = dictionary.getCheck(checkName); - return instantiateCheck(checkCls); + return instantiateCheck(checkCls, dictionary.getInjector()); } /** @@ -34,9 +35,11 @@ default Check getCheck(EntityDictionary dictionary, String checkName) { * @return the instance of the check * @throws IllegalArgumentException if the check class cannot be instantiated with a zero argument constructor */ - default Check instantiateCheck(Class checkCls) { + default Check instantiateCheck(Class checkCls, Injector injector) { try { - return Objects.requireNonNull(checkCls).newInstance(); + Check check = Objects.requireNonNull(checkCls).newInstance(); + injector.inject(check); + return check; } catch (InstantiationException | IllegalAccessException | NullPointerException e) { String checkName = (checkCls != null) ? checkCls.getName() : "null"; throw new IllegalArgumentException("Could not instantiate specified check '" + checkName + "'.", e); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java index 9286603661..8344da8fc8 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java @@ -8,14 +8,15 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import java.io.Closeable; import java.io.Serializable; import java.util.Iterator; -import java.util.Optional; import java.util.Set; /** @@ -112,19 +113,21 @@ default T createNewObject(Class entityClass) { } /** - * Loads an object by ID. + * Loads an object by ID. The reason we support both load by ID and load by filter is that + * some legacy stores are optimized to load by ID. * - * @param entityClass the type of class to load + * @param entityProjection the collection to load. * @param id - the ID of the object to load. - * @param filterExpression - security filters that can be evaluated in the data store. * @param scope - the current request scope * It is optional for the data store to attempt evaluation. * @return the loaded object if it exists AND any provided security filters pass. */ - default Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { + default Object loadObject(EntityProjection entityProjection, + Serializable id, + RequestScope scope) { + Class entityClass = entityProjection.getType(); + FilterExpression filterExpression = entityProjection.getFilterExpression(); + EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); String idField = dictionary.getIdFieldName(entityClass); @@ -132,14 +135,15 @@ default Object loadObject(Class entityClass, new Path.PathElement(entityClass, idType, idField), id ); - FilterExpression joinedFilterExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(idFilter, fe)) - .orElse(idFilter); - Iterable results = loadObjects(entityClass, - Optional.of(joinedFilterExpression), - Optional.empty(), - Optional.empty(), + FilterExpression joinedFilterExpression = (filterExpression != null) + ? new AndFilterExpression(idFilter, filterExpression) + : idFilter; + + Iterable results = loadObjects(entityProjection.copyOf() + .filterExpression(joinedFilterExpression) + .build(), scope); + Iterator it = results == null ? null : results.iterator(); if (it != null && it.hasNext()) { return it.next(); @@ -150,19 +154,12 @@ default Object loadObject(Class entityClass, /** * Loads a collection of objects. * - * @param entityClass - the class to load - * @param filterExpression - filters that can be evaluated in the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param entityProjection - the class to load * @param scope - contains request level metadata. * @return a collection of the loaded objects */ Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection entityProjection, RequestScope scope); /** @@ -170,25 +167,18 @@ Iterable loadObjects( * * @param relationTx - The datastore that governs objects of the relationhip's type. * @param entity - The object which owns the relationship. - * @param relationName - name of the relationship. - * @param filterExpression - filtering which can be pushed down to the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param relationship - the relationship to fetch. * @param scope - contains request level metadata. * @return the object in the relation */ default Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { - return PersistentResource.getValue(entity, relationName, scope); - } + return PersistentResource.getValue(entity, relationship.getName(), scope); + } /** * Elide core will update the in memory representation of the objects to the requested state. @@ -230,14 +220,14 @@ default void updateToOneRelation(DataStoreTransaction relationTx, * Get an attribute from an object. * * @param entity - The object which owns the attribute. - * @param attributeName - name of the attribute. + * @param attribute - The attribute to fetch * @param scope - contains request level metadata. * @return the value of the attribute */ default Object getAttribute(Object entity, - String attributeName, + Attribute attribute, RequestScope scope) { - return PersistentResource.getValue(entity, attributeName, scope); + return PersistentResource.getValue(entity, attribute.getName(), scope); } @@ -248,13 +238,11 @@ default Object getAttribute(Object entity, * This function allow a data store to optionally persist the attribute if needed. * * @param entity - The object which owns the attribute. - * @param attributeName - name of the attribute. - * @param attributeValue - the desired attribute value. + * @param attribute - the attribute to set. * @param scope - contains request level metadata. */ default void setAttribute(Object entity, - String attributeName, - Object attributeValue, + Attribute attribute, RequestScope scope) { } @@ -270,7 +258,7 @@ default FeatureSupport supportsFiltering(Class entityClass, FilterExpression /** * Whether or not the transaction can sort the provided class. - * @param entityClass + * @param entityClass The entity class that is being sorted. * @return true if sorting is possible */ default boolean supportsSorting(Class entityClass, Sorting sorting) { @@ -279,7 +267,7 @@ default boolean supportsSorting(Class entityClass, Sorting sorting) { /** * Whether or not the transaction can paginate the provided class. - * @param entityClass + * @param entityClass The entity class that is being paged. * @return true if pagination is possible */ default boolean supportsPagination(Class entityClass) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java index c2c904602c..79ae2a9f43 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java @@ -36,7 +36,6 @@ import org.apache.commons.lang3.tuple.Pair; import lombok.Getter; -import lombok.Setter; import java.lang.annotation.Annotation; import java.lang.reflect.AccessibleObject; @@ -51,14 +50,15 @@ import java.util.Collection; import java.util.Collections; import java.util.Deque; +import java.util.HashSet; import java.util.List; import java.util.Optional; +import java.util.Set; import java.util.concurrent.ConcurrentHashMap; import java.util.concurrent.ConcurrentLinkedDeque; import java.util.function.Predicate; import java.util.stream.Collectors; import java.util.stream.Stream; - import javax.persistence.AccessType; import javax.persistence.CascadeType; import javax.persistence.Column; @@ -95,9 +95,6 @@ public class EntityBinding { @Getter private Class idType; @Getter - @Setter - private Initializer initializer; - @Getter private AccessType accessType; public final EntityPermissions entityPermissions; @@ -116,10 +113,12 @@ public class EntityBinding { public final ConcurrentHashMap> fieldsToTypes = new ConcurrentHashMap<>(); public final ConcurrentHashMap aliasesToFields = new ConcurrentHashMap<>(); public final ConcurrentHashMap requestScopeableMethods = new ConcurrentHashMap<>(); + public final ConcurrentHashMap> attributeArguments = new ConcurrentHashMap<>(); public final ConcurrentHashMap, Annotation> annotations = new ConcurrentHashMap<>(); public static final EntityBinding EMPTY_BINDING = new EntityBinding(); + public static final Set EMPTY_ATTRIBUTES_ARGS = Collections.unmodifiableSet(new HashSet<>()); private static final String ALL_FIELDS = "*"; /* empty binding constructor */ @@ -586,4 +585,35 @@ private List> getInheritedTypes(Class entityClass) { return results; } + + + /** + * Add a collection of arguments to the attributes of this Entity. + * @param attribute attribute name to which argument has to be added + * @param arguments Set of Argument Type for the attribute + */ + public void addArgumentsToAttribute(String attribute, Set arguments) { + AccessibleObject fieldObject = fieldsToValues.get(attribute); + if (fieldObject != null && arguments != null) { + Set existingArgs = attributeArguments.get(fieldObject); + if (existingArgs != null) { + //Replace any argument names with new value + existingArgs.addAll(arguments); + } else { + attributeArguments.put(fieldObject, new HashSet<>(arguments)); + } + } + } + + /** + * Returns the Collection of all attributes of an argument. + * @param attribute Name of the argument for ehich arguments are to be retrieved. + * @return A Set of ArgumentType for the given attribute. + */ + public Set getAttributeArguments(String attribute) { + AccessibleObject fieldObject = fieldsToValues.get(attribute); + return (fieldObject != null) + ? attributeArguments.getOrDefault(fieldObject, EMPTY_ATTRIBUTES_ARGS) + : EMPTY_ATTRIBUTES_ARGS; + } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index 2423be38b3..78f21b2b42 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -20,6 +20,7 @@ import com.yahoo.elide.core.exceptions.InternalServerErrorException; import com.yahoo.elide.core.exceptions.InvalidAttributeException; import com.yahoo.elide.functions.LifeCycleHook; +import com.yahoo.elide.security.FilterExpressionCheck; import com.yahoo.elide.security.checks.Check; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; @@ -31,10 +32,12 @@ import com.google.common.collect.BiMap; import com.google.common.collect.HashBiMap; import com.google.common.collect.Maps; +import com.google.common.collect.Sets; import org.antlr.v4.runtime.tree.ParseTree; import org.apache.commons.lang3.StringUtils; +import lombok.Getter; import lombok.extern.slf4j.Slf4j; import java.lang.annotation.Annotation; @@ -63,7 +66,9 @@ import javax.persistence.AccessType; import javax.persistence.CascadeType; +import javax.persistence.Column; import javax.persistence.Entity; +import javax.persistence.JoinColumn; import javax.persistence.Transient; import javax.ws.rs.WebApplicationException; @@ -81,6 +86,8 @@ public class EntityDictionary { protected final CopyOnWriteArrayList> bindEntityRoots = new CopyOnWriteArrayList<>(); protected final ConcurrentHashMap, List>> subclassingEntities = new ConcurrentHashMap<>(); protected final BiMap> checkNames; + + @Getter protected final Injector injector; public final static String REGULAR_ID_NAME = "id"; @@ -95,7 +102,24 @@ public class EntityDictionary { * to their implementing classes */ public EntityDictionary(Map> checks) { - this(checks, null); + this.checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + initializeChecks(); + + //Default injector only injects Elide internals. + this.injector = new Injector() { + @Override + public void inject(Object entity) { + if (entity instanceof FilterExpressionCheck) { + try { + Field field = FilterExpressionCheck.class.getDeclaredField("dictionary"); + field.setAccessible(true); + field.set(entity, EntityDictionary.this); + } catch (NoSuchFieldException | IllegalAccessException e) { + throw new IllegalStateException(e); + } + } + } + }; } /** @@ -109,17 +133,20 @@ public EntityDictionary(Map> checks) { * initialize Elide models. */ public EntityDictionary(Map> checks, Injector injector) { - checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + this.checkNames = Maps.synchronizedBiMap(HashBiMap.create(checks)); + initializeChecks(); + this.injector = injector; + } + private void initializeChecks() { addPrefabCheck("Prefab.Role.All", Role.ALL.class); addPrefabCheck("Prefab.Role.None", Role.NONE.class); addPrefabCheck("Prefab.Collections.AppendOnly", AppendOnly.class); addPrefabCheck("Prefab.Collections.RemoveOnly", RemoveOnly.class); addPrefabCheck("Prefab.Common.UpdateOnCreate", Common.UpdateOnCreate.class); - - this.injector = injector; } + private void addPrefabCheck(String alias, Class checkClass) { if (checkNames.containsKey(alias) || checkNames.inverse().containsKey(checkClass)) { return; @@ -272,8 +299,8 @@ public ParseTree getPermissionsForClass(Class resourceClass, Class resourceClass, - String field, - Class annotationClass) { + String field, + Class annotationClass) { EntityBinding binding = getEntityBinding(resourceClass); return binding.entityPermissions.getFieldChecksForPermission(field, annotationClass); } @@ -795,28 +822,10 @@ public String getNameFromAlias(Object entity, String alias) { */ public void initializeEntity(T entity) { if (entity != null) { - @SuppressWarnings("unchecked") - Initializer initializer = getEntityBinding(entity.getClass()).getInitializer(); - if (initializer != null) { - initializer.initialize(entity); - } else if (injector != null) { - injector.inject(entity); - } + injector.inject(entity); } } - /** - * Bind a particular initializer to a class. - * - * @param the type parameter - * @param initializer Initializer to use for class - * @param cls Class to bind initialization - */ - public void bindInitializer(Initializer initializer, Class cls) { - bindIfUnbound(cls); - getEntityBinding(cls).setInitializer(initializer); - } - /** * Returns whether or not an entity is shareable. * @@ -1044,7 +1053,7 @@ public Collection getIdAnnotations(Object value) { } /** - * Follow for this class or super-class for Entity annotation. + * Follow for this class or super-class for JPA {@link Entity} annotation. * * @param objClass provided class * @return class with Entity annotation @@ -1066,6 +1075,12 @@ public Class lookupEntityClass(Class objClass) { public Class lookupIncludeClass(Class objClass) { Annotation first = getFirstAnnotation(objClass, Arrays.asList(Exclude.class, Include.class)); if (first instanceof Include) { + Class declaringClass = lookupAnnotationDeclarationClass(objClass, Include.class); + if (declaringClass != null) { + return declaringClass; + } + + //If we didn't find Include declared on a class, it must be declared at the package level. return objClass; } return null; @@ -1086,6 +1101,7 @@ public Class lookupAnnotationDeclarationClass(Class objClass, Class objClass) { } + /** + * Check whether a class is a JPA entity + * + * @param objClass class + * @return True if it is a JPA entity + */ + public final boolean isJPAEntity(Class objClass) { + try { + lookupEntityClass(objClass); + return true; + } catch (IllegalArgumentException e) { + return false; + } + } + /** * Retrieve the accessible object for a field from a target object. * @@ -1317,7 +1348,7 @@ public List walkEntityGraph(Set> entities, Function, T /** * Returns whether or not a class is already bound. - * @param cls + * @param cls The class to verify. * @return true if the class is bound. False otherwise. */ public boolean hasBinding(Class cls) { @@ -1468,6 +1499,36 @@ private Map coerceMap(Object target, Map values, String fieldName) { return result; } + /** + * Returns whether or not a specified annotation is present on an entity field or its corresponding method. + * + * @param fieldName The entity field + * @param annotationClass The provided annotation class + * + * @param The type of the {@code annotationClass} + * + * @return {@code true} if the field is annotated by the {@code annotationClass} + */ + public boolean attributeOrRelationAnnotationExists( + Class cls, + String fieldName, + Class annotationClass + ) { + return getAttributeOrRelationAnnotation(cls, annotationClass, fieldName) != null; + } + + /** + * Returns whether or not a specified field exists in an entity. + * + * @param cls The entity + * @param fieldName The provided field to check + * + * @return {@code true} if the field exists in the entity + */ + public boolean isValidField(Class cls, String fieldName) { + return getAllFields(cls).contains(fieldName); + } + private boolean isValidParameterizedMap(Map values, Class keyType, Class valueType) { for (Map.Entry entry : values.entrySet()) { Object key = entry.getKey(); @@ -1485,8 +1546,64 @@ private boolean isValidParameterizedMap(Map values, Class keyType, Clas * @param entityClass the class to bind. */ private void bindIfUnbound(Class entityClass) { + /* This is safe to call with non-proxy objects. Not safe to call with ORM proxy objects. */ + if (! isClassBound(entityClass)) { bindEntity(entityClass); } } + + /** + * Add a collection of argument to the attributes + * @param cls The entity + * @param attributeName attribute name to which argument has to be added + * @param arguments Set of Argument type containing name and type of each argument. + */ + public void addArgumentsToAttribute(Class cls, String attributeName, Set arguments) { + getEntityBinding(cls).addArgumentsToAttribute(attributeName, arguments); + } + + /** + * Add a single argument to the attribute + * @param cls The entity + * @param attributeName attribute name to which argument has to be added + * @param argument A single argument + */ + public void addArgumentToAttribute(Class cls, String attributeName, ArgumentType argument) { + this.addArgumentsToAttribute(cls, attributeName, Sets.newHashSet(argument)); + } + + /** + * Returns the Collection of all attributes of an argument. + * @param cls The entity + * @param attributeName Name of the argument for ehich arguments are to be retrieved. + * @return A Set of ArgumentType for the given attribute. + */ + public Set getAttributeArguments(Class cls, String attributeName) { + return entityBindings.getOrDefault(cls, EMPTY_BINDING).getAttributeArguments(attributeName); + } + + /** + * Get column name using JPA. + * + * @param cls The entity class. + * @param fieldName The entity attribute. + * @return The jpa column name. + */ + public String getAnnotatedColumnName(Class cls, String fieldName) { + Column[] column = getAttributeOrRelationAnnotations(cls, Column.class, fieldName); + + // this would only be valid for dimension columns + JoinColumn[] joinColumn = getAttributeOrRelationAnnotations(cls, JoinColumn.class, fieldName); + + if (column == null || column.length == 0) { + if (joinColumn == null || joinColumn.length == 0) { + return fieldName; + } else { + return joinColumn[0].name(); + } + } else { + return column[0].name(); + } + } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java b/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java deleted file mode 100644 index 79aa554139..0000000000 --- a/elide-core/src/main/java/com/yahoo/elide/core/Initializer.java +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2017, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.core; - -/** - * Used to perform any additional initialization required on entity beans which is not - * possible at time of construction. - * @param bean type - */ -@FunctionalInterface -public interface Initializer { - - /** - * Initialize an entity bean. - * - * @param entity Entity bean to initialize - */ - void initialize(T entity); -} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java index e93727a57d..e1aa4b7419 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java @@ -31,6 +31,9 @@ import com.yahoo.elide.jsonapi.models.ResourceIdentifier; import com.yahoo.elide.jsonapi.models.SingleElementSet; import com.yahoo.elide.parsers.expression.CanPaginateVisitor; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.permissions.ExpressionResult; import com.yahoo.elide.utils.coerce.CoerceUtil; @@ -38,13 +41,11 @@ import com.fasterxml.jackson.annotation.JsonIgnore; import com.google.common.base.Preconditions; import com.google.common.collect.Sets; - import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; import org.apache.commons.lang3.StringUtils; import lombok.NonNull; - import java.io.Serializable; import java.lang.annotation.Annotation; import java.util.ArrayList; @@ -77,6 +78,7 @@ public class PersistentResource implements com.yahoo.elide.security.Persisten private final DataStoreTransaction transaction; private final RequestScope requestScope; private int hashCode = 0; + static final String CLASS_NO_FIELD = ""; /** @@ -96,6 +98,21 @@ public String toString() { } /** + * Create a resource in the database. + * @param entityClass the entity class + * @param requestScope the request scope + * @param uuid the (optional) uuid + * @param object type + * @return persistent resource + */ + public static PersistentResource createObject( + Class entityClass, + RequestScope requestScope, + Optional uuid) { + return createObject(null, entityClass, requestScope, uuid); + } + + /** * Create a resource in the database. * @param parent - The immediate ancestor in the lineage or null if this is a root. * @param entityClass the entity class @@ -110,7 +127,7 @@ public static PersistentResource createObject( RequestScope requestScope, Optional uuid) { - //instead of calling transcation.createObject, create the new object here. + //instead of calling transaction.createObject, create the new object here. T obj = requestScope.getTransaction().createNewObject(entityClass); String id = uuid.orElse(null); @@ -150,7 +167,12 @@ public static PersistentResource createObject( * @param id the id * @param scope the request scope */ - public PersistentResource(@NonNull T obj, PersistentResource parent, String id, @NonNull RequestScope scope) { + public PersistentResource( + @NonNull T obj, + PersistentResource parent, + String id, + @NonNull RequestScope scope + ) { this.obj = obj; this.uuid = Optional.ofNullable(id); this.lineage = parent != null ? new ResourceLineage(parent.lineage, parent) : new ResourceLineage(); @@ -161,7 +183,7 @@ public PersistentResource(@NonNull T obj, PersistentResource parent, String id, dictionary.initializeEntity(obj); } - /** + /** * Check whether an id matches for this persistent resource. * * @param checkId the check id @@ -183,7 +205,7 @@ public boolean matchesId(String checkId) { /** * Load an single entity from the DB. * - * @param loadClass resource type + * @param projection What to load from the DB. * @param id the id * @param requestScope the request scope * @param type of resource @@ -192,14 +214,17 @@ public boolean matchesId(String checkId) { */ @SuppressWarnings("resource") @NonNull public static PersistentResource loadRecord( - Class loadClass, String id, RequestScope requestScope) - throws InvalidObjectIdentifierException { - Preconditions.checkNotNull(loadClass); + EntityProjection projection, + String id, + RequestScope requestScope + ) throws InvalidObjectIdentifierException { + Preconditions.checkNotNull(projection); Preconditions.checkNotNull(id); Preconditions.checkNotNull(requestScope); DataStoreTransaction tx = requestScope.getTransaction(); EntityDictionary dictionary = requestScope.getDictionary(); + Class loadClass = projection.getType(); // Check the resource cache if exists Object obj = requestScope.getObjectById(dictionary.getJsonAliasFor(loadClass), id); @@ -208,15 +233,24 @@ public boolean matchesId(String checkId) { Optional permissionFilter = getPermissionFilterExpression(loadClass, requestScope); Class idType = dictionary.getIdType(loadClass); - obj = tx.loadObject(loadClass, (Serializable) CoerceUtil.coerce(id, idType), - permissionFilter, requestScope); + + projection = projection + .copyOf() + .filterExpression(permissionFilter.orElse(null)) + .build(); + + obj = tx.loadObject(projection, (Serializable) CoerceUtil.coerce(id, idType), requestScope); if (obj == null) { throw new InvalidObjectIdentifierException(id, dictionary.getJsonAliasFor(loadClass)); } } PersistentResource resource = new PersistentResource<>( - loadClass.cast(obj), null, requestScope.getUUIDFor(obj), requestScope); + (T) obj, + null, + requestScope.getUUIDFor(obj), + requestScope); + // No need to have read access for a newly created object if (!requestScope.getNewResources().contains(resource)) { resource.checkFieldAwarePermissions(ReadPermission.class); @@ -245,25 +279,26 @@ private static Optional getPermissionFilterExpression(Clas /** * Load a collection from the datastore. * - * @param loadClass the load class + * @param projection the projection to load * @param requestScope the request scope * @param ids a list of object identifiers to optionally load. Can be empty. * @return a filtered collection of resources loaded from the datastore. */ public static Set loadRecords( - Class loadClass, + EntityProjection projection, List ids, - Optional filter, - Optional sorting, - Optional pagination, RequestScope requestScope) { + Class loadClass = projection.getType(); + Pagination pagination = projection.getPagination(); + Sorting sorting = projection.getSorting(); + + FilterExpression filterExpression = projection.getFilterExpression(); + EntityDictionary dictionary = requestScope.getDictionary(); - FilterExpression filterExpression; DataStoreTransaction tx = requestScope.getTransaction(); - if (shouldSkipCollection(loadClass, ReadPermission.class, requestScope)) { if (ids.isEmpty()) { return Collections.emptySet(); @@ -271,8 +306,7 @@ public static Set loadRecords( throw new InvalidObjectIdentifierException(ids.toString(), dictionary.getJsonAliasFor(loadClass)); } - - if (pagination.isPresent() && !pagination.get().isDefaultInstance() + if (pagination != null && !pagination.isDefaultInstance() && !CanPaginateVisitor.canPaginate(loadClass, dictionary, requestScope)) { throw new InvalidPredicateException(String.format("Cannot paginate %s", dictionary.getJsonAliasFor(loadClass))); @@ -289,11 +323,9 @@ public static Set loadRecords( FilterExpression idExpression = buildIdFilterExpression(ids, loadClass, dictionary, requestScope); // Combine filters if necessary - filterExpression = filter + filterExpression = Optional.ofNullable(filterExpression) .map(fe -> (FilterExpression) new AndFilterExpression(idExpression, fe)) .orElse(idExpression); - } else { - filterExpression = filter.orElse(null); } Optional permissionFilter = getPermissionFilterExpression(loadClass, requestScope); @@ -305,9 +337,15 @@ public static Set loadRecords( } } + EntityProjection modifiedProjection = projection + .copyOf() + .filterExpression(filterExpression) + .sorting(sorting) + .pagination(Optional.ofNullable(pagination).map(p -> p.evaluate(loadClass)).orElse(null)) + .build(); + Set existingResources = filter(ReadPermission.class, - new PersistentResourceSet(tx.loadObjects(loadClass, Optional.ofNullable(filterExpression), sorting, - pagination.map(p -> p.evaluate(loadClass)), requestScope), requestScope)); + new PersistentResourceSet(tx.loadObjects(modifiedProjection, requestScope), requestScope)); Set allResources = Sets.union(newResources, existingResources); @@ -340,7 +378,13 @@ public boolean updateAttribute(String fieldName, Object newVal) { this.markDirty(); //Hooks for customize logic for setAttribute/Relation if (dictionary.isAttribute(obj.getClass(), fieldName)) { - transaction.setAttribute(obj, fieldName, newVal, requestScope); + transaction.setAttribute(obj, Attribute.builder() + .name(fieldName) + .type(fieldClass) + .argument(Argument.builder() + .name("_UNUSED_") + .value(newVal).build()) + .build(), requestScope); } return true; } @@ -788,49 +832,44 @@ public Optional getUUID() { * * NOTE: Filter expressions for this type are _not_ applied at this level. * - * @param relation relation name + * @param relationship The relationship * @param id single id to lookup * @return The PersistentResource of the sought id or null if does not exist. */ - public PersistentResource getRelation(String relation, String id) { - Set resources = getRelation(relation, Collections.singletonList(id), - Optional.empty(), Optional.empty(), Optional.empty()); - if (resources.isEmpty()) { + public PersistentResource getRelation(com.yahoo.elide.request.Relationship relationship, String id) { + Set resources = getRelation(Collections.singletonList(id), relationship); + + if (resources.isEmpty()) { return null; - } - // If this is an in-memory object (i.e. UUID being created within tx), datastore may not be able to filter. - // If we get multiple results back, make sure we find the right id first. - for (PersistentResource resource : resources) { - if (resource.matchesId(id)) { - return resource; - } - } - return null; + } + // If this is an in-memory object (i.e. UUID being created within tx), datastore may not be able to filter. + // If we get multiple results back, make sure we find the right id first. + for (PersistentResource resource : resources) { + if (resource.matchesId(id)) { + return resource; + } + } + return null; } /** - * Load a single entity relation from the PersistentResource. Example: GET /book/2 + * Load a relation from the PersistentResource. * - * @param relation the relation + * @param relationship the relation * @param ids a list of object identifiers to optionally load. Can be empty. * @return PersistentResource relation */ - public Set getRelation(String relation, - List ids, - Optional filter, - Optional sorting, - Optional pagination) { + public Set getRelation(List ids, com.yahoo.elide.request.Relationship relationship) { - FilterExpression filterExpression; + FilterExpression filterExpression = Optional.ofNullable(relationship.getProjection().getFilterExpression()) + .orElse(null); - Class entityType = dictionary.getParameterizedType(getResourceClass(), relation); + assertRelationshipExists(relationship.getName()); + Class entityType = dictionary.getParameterizedType(getResourceClass(), relationship.getName()); Set newResources = new LinkedHashSet<>(); /* If this is a bulk edit request and the ID we are fetching for is newly created... */ - if (entityType == null) { - throw new InvalidAttributeException(relation, type); - } if (!ids.isEmpty()) { // Fetch our set of new resources that we know about since we can't find them in the datastore newResources = requestScope.getNewPersistentResources().stream() @@ -841,18 +880,21 @@ public Set getRelation(String relation, FilterExpression idExpression = buildIdFilterExpression(ids, entityType, dictionary, requestScope); // Combine filters if necessary - filterExpression = filter + filterExpression = Optional.ofNullable(relationship.getProjection().getFilterExpression()) .map(fe -> (FilterExpression) new AndFilterExpression(idExpression, fe)) .orElse(idExpression); - } else { - filterExpression = filter.orElse(null); } // TODO: Filter on new resources? // TODO: Update pagination to subtract the number of new resources created? Set existingResources = filter(ReadPermission.class, - getRelation(relation, Optional.ofNullable(filterExpression), sorting, pagination, true)); + + getRelation(relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(filterExpression) + .build()) + .build(), true)); // TODO: Sort again in memory now that two sets are glommed together? @@ -863,7 +905,7 @@ public Set getRelation(String relation, Set missedIds = Sets.difference(new HashSet<>(ids), allExpectedIds); if (!missedIds.isEmpty()) { - throw new InvalidObjectIdentifierException(missedIds.toString(), relation); + throw new InvalidObjectIdentifierException(missedIds.toString(), relationship.getName()); } return allResources; @@ -903,60 +945,73 @@ private static FilterExpression buildIdFilterExpression(List ids, /** * Get collection of resources from relation field. * - * @param relationName field + * @param relationship relationship * @return collection relation */ - public Set getRelationCheckedFiltered(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { - + public Set getRelationCheckedFiltered(com.yahoo.elide.request.Relationship relationship) { return filter(ReadPermission.class, - getRelation(relationName, filterExpression, sorting, pagination, true)); + getRelation(relationship, true)); } private Set getRelationUncheckedUnfiltered(String relationName) { - return getRelation(relationName, Optional.empty(), Optional.empty(), Optional.empty(), false); + assertRelationshipExists(relationName); + return getRelation(com.yahoo.elide.request.Relationship.builder() + .name(relationName) + .alias(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .build()) + .build(), false); } private Set getRelationCheckedUnfiltered(String relationName) { - return getRelation(relationName, Optional.empty(), Optional.empty(), Optional.empty(), true); + assertRelationshipExists(relationName); + return getRelation(com.yahoo.elide.request.Relationship.builder() + .name(relationName) + .alias(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .build()) + .build(), true); + } + + private void assertRelationshipExists(String relationName) { + if (relationName == null || dictionary.getParameterizedType(obj, relationName) == null) { + throw new InvalidAttributeException(relationName, this.getType()); + } } - private Set getRelation(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + private Set getRelation(com.yahoo.elide.request.Relationship relationship, boolean checked) { - - if (checked && !checkRelation(relationName)) { + if (checked && !checkRelation(relationship)) { return Collections.emptySet(); } - final Class relationClass = dictionary.getParameterizedType(obj, relationName); + final Class relationClass = dictionary.getParameterizedType(obj, relationship.getName()); + + Optional pagination = Optional.ofNullable(relationship.getProjection().getPagination()); + if (pagination.isPresent() && !pagination.get().isDefaultInstance() && !CanPaginateVisitor.canPaginate(relationClass, dictionary, requestScope)) { throw new InvalidPredicateException(String.format("Cannot paginate %s", dictionary.getJsonAliasFor(relationClass))); } - return getRelationUnchecked(relationName, filterExpression, sorting, pagination); + return getRelationUnchecked(relationship); } /** * Check the permissions of the relationship, and return true or false. - * @param relationName The relationship to the entity + * @param relationship The relationship to the entity * @return True if the relationship to the entity has valid permissions for the user */ - protected boolean checkRelation(String relationName) { - List relations = dictionary.getRelationships(obj); + protected boolean checkRelation(com.yahoo.elide.request.Relationship relationship) { + String relationName = relationship.getName(); String realName = dictionary.getNameFromAlias(obj, relationName); relationName = (realName == null) ? relationName : realName; - if (relationName == null || relations == null || !relations.contains(relationName)) { - throw new InvalidAttributeException(relationName, type); - } + assertRelationshipExists(relationName); checkFieldAwareDeferPermissions(ReadPermission.class, relationName, null, null); @@ -969,70 +1024,73 @@ protected boolean checkRelation(String relationName) { /** * Get collection of resources from relation field. * - * @param relationName field - * @param filterExpression An optional filter expression + * @param relationship the relationship to fetch * @return collection relation */ - protected Set getRelationChecked(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { - if (!checkRelation(relationName)) { + protected Set getRelationChecked(com.yahoo.elide.request.Relationship relationship) { + if (!checkRelation(relationship)) { return Collections.emptySet(); } - return getRelationUnchecked(relationName, filterExpression, sorting, pagination); + return getRelationUnchecked(relationship); } /** - * Retrieve an uncheck set of relations. - * - * @param relationName field - * @param filterExpression An optional filter expression - * @param sorting the sorting clause - * @param pagination the pagination params - * @return the resources in the relationship - */ - private Set getRelationUnchecked(String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination) { + * Retrieve an unchecked set of relations. + */ + private Set getRelationUnchecked(com.yahoo.elide.request.Relationship relationship) { + String relationName = relationship.getName(); + FilterExpression filterExpression = relationship.getProjection().getFilterExpression(); + Pagination pagination = relationship.getProjection().getPagination(); + Sorting sorting = relationship.getProjection().getSorting(); + RelationshipType type = getRelationshipType(relationName); final Class relationClass = dictionary.getParameterizedType(obj, relationName); if (relationClass == null) { throw new InvalidAttributeException(relationName, this.getType()); } - Optional computedPagination = pagination.map(p -> p.evaluate(relationClass)); + Optional computedPagination = Optional.ofNullable(pagination) + .map(p -> p.evaluate(relationClass)); //Invoke filterExpressionCheck and then merge with filterExpression. Optional permissionFilter = getPermissionFilterExpression(relationClass, requestScope); - Optional computedFilters = filterExpression; + Optional computedFilters = Optional.ofNullable(filterExpression); - if (permissionFilter.isPresent() && filterExpression.isPresent()) { + if (permissionFilter.isPresent() && filterExpression != null) { FilterExpression mergedExpression = - new AndFilterExpression(filterExpression.get(), permissionFilter.get()); + new AndFilterExpression(filterExpression, permissionFilter.get()); computedFilters = Optional.of(mergedExpression); } else if (permissionFilter.isPresent()) { computedFilters = permissionFilter; } - Object val = transaction.getRelation(transaction, obj, relationName, - computedFilters, sorting, computedPagination, requestScope); + com.yahoo.elide.request.Relationship modifiedRelationship = relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(computedFilters.orElse(null)) + .sorting(sorting) + .pagination(computedPagination.orElse(null)) + .build() + ).build(); + + Object val = transaction.getRelation(transaction, obj, modifiedRelationship, requestScope); if (val == null) { return Collections.emptySet(); } Set resources = Sets.newLinkedHashSet(); + if (val instanceof Iterable) { Iterable filteredVal = (Iterable) val; resources = new PersistentResourceSet(this, filteredVal, requestScope); } else if (type.isToOne()) { - resources = new SingleElementSet<>( - new PersistentResource<>(val, this, requestScope.getUUIDFor(val), requestScope)); + resources = new SingleElementSet( + new PersistentResource(val, this, + requestScope.getUUIDFor(val), requestScope)); } else { - resources.add(new PersistentResource<>(val, this, requestScope.getUUIDFor(val), requestScope)); + resources.add(new PersistentResource(val, this, + requestScope.getUUIDFor(val), requestScope)); } return resources; @@ -1071,10 +1129,20 @@ public RelationshipType getRelationshipType(String relation) { * @param attr Attribute name * @return Object value for attribute */ + @Deprecated public Object getAttribute(String attr) { return this.getValueChecked(attr); } + /** + * Get the value for a particular attribute (i.e. non-relational field) + * @param attr the Attribute + * @return Object value for attribute + */ + public Object getAttribute(Attribute attr) { + return this.getValueChecked(attr); + } + /** * Wrapped Entity bean. * @@ -1188,8 +1256,8 @@ public Resource toResource() { * Fetch a resource with support for lambda function for getting relationships and attributes. * @return The Resource */ - public Resource toResourceWithSortingAndPagination() { - return toResource(this::getRelationshipsWithSortingAndPagination, this::getAttributes); + public Resource toResource(EntityProjection projection) { + return toResource(() -> { return getRelationships(projection); }, this::getAttributes); } /** @@ -1198,7 +1266,7 @@ public Resource toResourceWithSortingAndPagination() { * @param attributeSupplier The attribute supplier * @return The Resource */ - public Resource toResource(final Supplier> relationshipSupplier, + private Resource toResource(final Supplier> relationshipSupplier, final Supplier> attributeSupplier) { final Resource resource = new Resource(type, (obj == null) ? uuid.orElseThrow( @@ -1216,8 +1284,17 @@ public Resource toResource(final Supplier> relationshi */ protected Map getRelationships() { return getRelationshipsWithRelationshipFunction((relationName) -> { - Optional filterExpression = requestScope.getExpressionForRelation(this, relationName); - return getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); + Optional filterExpression = requestScope.getExpressionForRelation(getResourceClass(), + relationName); + + return getRelationCheckedFiltered(com.yahoo.elide.request.Relationship.builder() + .alias(relationName) + .name(relationName) + .projection(EntityProjection.builder() + .type(dictionary.getParameterizedType(getResourceClass(), relationName)) + .filterExpression(filterExpression.orElse(null)) + .build()) + .build()); }); } @@ -1226,14 +1303,11 @@ protected Map getRelationships() { * * @return Relationship mapping */ - protected Map getRelationshipsWithSortingAndPagination() { - return getRelationshipsWithRelationshipFunction((relationName) -> { - Optional filterExpression = requestScope.getExpressionForRelation(this, relationName); - Optional sorting = Optional.ofNullable(requestScope.getSorting()); - Optional pagination = Optional.ofNullable(requestScope.getPagination()); - return getRelationCheckedFiltered(relationName, - filterExpression, sorting, pagination); - }); + private Map getRelationships(EntityProjection projection) { + return getRelationshipsWithRelationshipFunction( + (relationName) -> getRelationCheckedFiltered(projection.getRelationship(relationName) + .orElseThrow(IllegalStateException::new) + )); } /** @@ -1324,6 +1398,7 @@ protected void nullValue(String fieldName, PersistentResource oldValue) { * @param fieldName the field name * @return value value */ + @Deprecated protected Object getValueChecked(String fieldName) { requestScope.publishLifecycleEvent(this, CRUDEvent.CRUDAction.READ); requestScope.publishLifecycleEvent(this, fieldName, CRUDEvent.CRUDAction.READ, Optional.empty()); @@ -1331,6 +1406,18 @@ protected Object getValueChecked(String fieldName) { return getValue(getObject(), fieldName, requestScope); } + /** + * Gets a value from an entity and checks read permissions. + * @param attribute the attribute to fetch. + * @return value value + */ + protected Object getValueChecked(Attribute attribute) { + requestScope.publishLifecycleEvent(this, CRUDEvent.CRUDAction.READ); + requestScope.publishLifecycleEvent(this, attribute.getName(), CRUDEvent.CRUDAction.READ, Optional.empty()); + checkFieldAwareDeferPermissions(ReadPermission.class, attribute.getName(), (Object) null, (Object) null); + return transaction.getAttribute(getObject(), attribute, requestScope); + } + /** * Retrieve an object without checking read permissions (i.e. value is used internally and not sent to others) * @@ -1438,7 +1525,9 @@ protected void delFromCollection( */ protected void setValue(String fieldName, Object value) { final Object original = getValueUnchecked(fieldName); + dictionary.setValue(obj, fieldName, value); + triggerUpdate(fieldName, original, value); } @@ -1450,8 +1539,7 @@ protected void setValue(String fieldName, Object value) { * @return the value */ public static Object getValue(Object target, String fieldName, RequestScope requestScope) { - EntityDictionary dictionary = requestScope.getDictionary(); - return dictionary.getValue(target, fieldName, requestScope); + return requestScope.getDictionary().getValue(target, fieldName, requestScope); } /** @@ -1467,7 +1555,8 @@ protected void deleteInverseRelation(String relationName, Object inverseEntity) Class inverseType = dictionary.getType(inverseEntity.getClass(), inverseField); String uuid = requestScope.getUUIDFor(inverseEntity); - PersistentResource inverseResource = new PersistentResource(inverseEntity, this, uuid, requestScope); + PersistentResource inverseResource = new PersistentResource(inverseEntity, + this, uuid, requestScope); Object inverseRelation = inverseResource.getValueUnchecked(inverseField); if (inverseRelation == null) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java index 08fe19b20f..f036d0ea16 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java @@ -30,6 +30,7 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.PermissionExecutor; import com.yahoo.elide.security.User; @@ -39,6 +40,7 @@ import io.reactivex.subjects.PublishSubject; import io.reactivex.subjects.ReplaySubject; import lombok.Getter; +import lombok.Setter; import java.util.Collections; import java.util.HashMap; @@ -60,7 +62,7 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final JsonApiDocument jsonApiDocument; @Getter private final DataStoreTransaction transaction; @Getter private final User user; - @Getter private final EntityDictionary dictionary; + @Getter protected final EntityDictionary dictionary; @Getter private final JsonApiMapper mapper; @Getter private final AuditLogger auditLogger; @Getter private final Optional> queryParams; @@ -76,8 +78,11 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final ElideSettings elideSettings; @Getter private final boolean useFilterExpressions; @Getter private final int updateStatusCode; - @Getter private final MultipleFilterDialect filterDialect; + + //TODO - this ought to be read only and set in the constructor. + @Getter @Setter private EntityProjection entityProjection; + private final Map expressionsByType; private PublishSubject lifecycleEvents; @@ -233,7 +238,7 @@ public boolean isNewResource(Object entity) { * @param queryParams The request query parameters * @return Parsed sparseFields map */ - private static Map> parseSparseFields(MultivaluedMap queryParams) { + public static Map> parseSparseFields(MultivaluedMap queryParams) { Map> result = new HashMap<>(); for (Map.Entry> kv : queryParams.entrySet()) { @@ -264,6 +269,15 @@ public Optional getFilterExpressionByType(String type) { return Optional.ofNullable(expressionsByType.get(type)); } + /** + * Get filter expression for a specific collection type. + * @param entityClass The class to lookup + * @return The filter expression for the given type + */ + public Optional getFilterExpressionByType(Class entityClass) { + return Optional.ofNullable(expressionsByType.get(dictionary.getJsonAliasFor(entityClass))); + } + /** * Get the global/cross-type filter expression. * @param loadClass Entity class @@ -295,14 +309,14 @@ public Optional getLoadFilterExpression(Class loadClass) { /** * Get the filter expression for a particular relationship - * @param parent The object which has the relationship + * @param parentType The parent type which has the relationship * @param relationName The relationship name * @return A type specific filter expression for the given relationship */ - public Optional getExpressionForRelation(PersistentResource parent, String relationName) { - final Class entityClass = dictionary.getParameterizedType(parent.getObject(), relationName); + public Optional getExpressionForRelation(Class parentType, String relationName) { + final Class entityClass = dictionary.getParameterizedType(parentType, relationName); if (entityClass == null) { - throw new InvalidAttributeException(relationName, parent.getType()); + throw new InvalidAttributeException(relationName, dictionary.getJsonAliasFor(parentType)); } if (dictionary.isMappedInterface(entityClass) && interfaceHasFilterExpression(entityClass)) { throw new InvalidOperationException( diff --git a/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java b/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java new file mode 100644 index 0000000000..df5b187b31 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/core/TimedFunction.java @@ -0,0 +1,40 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import lombok.Data; +import lombok.extern.slf4j.Slf4j; + +import java.util.function.Supplier; + +/** + * Wraps a function and logs how long it took to run (in millis). + * @param The function return type. + */ +@Slf4j +@Data +public class TimedFunction implements Supplier { + + public TimedFunction(Supplier toRun, String logMessage) { + this.toRun = toRun; + this.logMessage = logMessage; + } + + private Supplier toRun; + private String logMessage; + + @Override + public R get() { + long start = System.currentTimeMillis(); + R ret = toRun.get(); + long end = System.currentTimeMillis(); + + log.debug(logMessage + "\tTime spent: {}", end - start); + + return ret; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java index b20183c4d6..8d2f72fd82 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapDataStore.java @@ -27,8 +27,8 @@ * Simple in-memory only database. */ public class HashMapDataStore implements DataStore, DataStoreTestHarness { - private final Map, Map> dataStore = Collections.synchronizedMap(new HashMap<>()); - @Getter private EntityDictionary dictionary; + protected final Map, Map> dataStore = Collections.synchronizedMap(new HashMap<>()); + @Getter protected EntityDictionary dictionary; @Getter private final Set beanPackages; @Getter private final ConcurrentHashMap, AtomicLong> typeIds = new ConcurrentHashMap<>(); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java index 943ca2d8bd..a1ba45304d 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java @@ -10,15 +10,17 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.TransactionException; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; +import lombok.extern.slf4j.Slf4j; + import java.io.IOException; import java.io.Serializable; import java.util.ArrayList; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.concurrent.atomic.AtomicLong; import javax.persistence.GeneratedValue; @@ -131,31 +133,25 @@ public void setId(Object value, String id) { @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { - return dictionary.getValue(entity, relationName, scope); + return dictionary.getValue(entity, relationship.getName(), scope); } @Override - public Iterable loadObjects(Class entityClass, Optional filterExpression, - Optional sorting, Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { synchronized (dataStore) { - Map data = dataStore.get(entityClass); + Map data = dataStore.get(projection.getType()); return data.values(); } } @Override - public Object loadObject(Class entityClass, Serializable id, - Optional filterExpression, - RequestScope scope) { + public Object loadObject(EntityProjection projection, Serializable id, RequestScope scope) { synchronized (dataStore) { - Map data = dataStore.get(entityClass); + Map data = dataStore.get(projection.getType()); if (data == null) { return null; } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java index 0c4475a3c2..3eba0004b9 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java @@ -17,6 +17,9 @@ import com.yahoo.elide.core.filter.expression.InMemoryFilterExecutor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.apache.commons.lang3.tuple.Pair; @@ -39,6 +42,8 @@ */ public class InMemoryStoreTransaction implements DataStoreTransaction { + private final DataStoreTransaction tx; + private static final Comparator NULL_SAFE_COMPARE = (a, b) -> { if (a == null && b == null) { return 0; @@ -53,8 +58,6 @@ public class InMemoryStoreTransaction implements DataStoreTransaction { } }; - private DataStoreTransaction tx; - /** * Fetches data from the store. */ @@ -71,6 +74,80 @@ public InMemoryStoreTransaction(DataStoreTransaction tx) { this.tx = tx; } + @Override + public Object getRelation(DataStoreTransaction relationTx, + Object entity, + Relationship relationship, + RequestScope scope) { + DataFetcher fetcher = new DataFetcher() { + @Override + public Object fetch(Optional filterExpression, + Optional sorting, + Optional pagination, + RequestScope scope) { + + return tx.getRelation(relationTx, entity, relationship.copyOf() + .projection(relationship.getProjection().copyOf() + .filterExpression(filterExpression.orElse(null)) + .sorting(sorting.orElse(null)) + .pagination(pagination.orElse(null)) + .build() + ).build(), scope); + } + }; + + + /* + * If we are mutating multiple entities, the data store transaction cannot perform filter & pagination directly. + * It must be done in memory by Elide as some newly created entities have not yet been persisted. + */ + boolean filterInMemory = scope.getNewPersistentResources().size() > 0; + return fetchData(fetcher, relationship.getProjection().getType(), + Optional.ofNullable(relationship.getProjection().getFilterExpression()), + Optional.ofNullable(relationship.getProjection().getSorting()), + Optional.ofNullable(relationship.getProjection().getPagination()), + filterInMemory, scope); + } + + @Override + public Object loadObject(EntityProjection projection, + Serializable id, + RequestScope scope) { + + if (projection.getFilterExpression() == null + || tx.supportsFiltering(projection.getType(), + projection.getFilterExpression()) == FeatureSupport.FULL) { + return tx.loadObject(projection, id, scope); + } else { + return DataStoreTransaction.super.loadObject(projection, id, scope); + } + } + + @Override + public Iterable loadObjects(EntityProjection projection, + RequestScope scope) { + + DataFetcher fetcher = new DataFetcher() { + @Override + public Iterable fetch(Optional filterExpression, + Optional sorting, + Optional pagination, + RequestScope scope) { + + return tx.loadObjects(projection.copyOf() + .filterExpression(filterExpression.orElse(null)) + .pagination(pagination.orElse(null)) + .sorting(sorting.orElse(null)) + .build(), scope); + } + }; + + return (Iterable) fetchData(fetcher, projection.getType(), + Optional.ofNullable(projection.getFilterExpression()), + Optional.ofNullable(projection.getSorting()), + Optional.ofNullable(projection.getPagination()), + false, scope); + } @Override public void save(Object entity, RequestScope scope) { @@ -98,34 +175,8 @@ public T createNewObject(Class entityClass) { } @Override - public Object getRelation(DataStoreTransaction relationTx, - Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - Class relationClass = scope.getDictionary().getParameterizedType(entity, relationName); - - DataFetcher fetcher = new DataFetcher() { - @Override - public Object fetch(Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - return tx.getRelation(relationTx, entity, relationName, filterExpression, sorting, pagination, scope); - } - }; - - - /* - * If we are mutating multiple entities, the data store transaction cannot perform filter & pagination directly. - * It must be done in memory by Elide as some newly created entities have not yet been persisted. - */ - boolean filterInMemory = scope.getNewPersistentResources().size() > 0; - return fetchData(fetcher, relationClass, filterExpression, sorting, pagination, filterInMemory, scope); + public void close() throws IOException { + tx.close(); } @Override @@ -148,14 +199,13 @@ public void updateToOneRelation(DataStoreTransaction relationTx, } @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return tx.getAttribute(entity, attributeName, scope); + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { + return tx.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - tx.setAttribute(entity, attributeName, attributeValue, scope); - + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { + tx.setAttribute(entity, attribute, scope); } @Override @@ -173,45 +223,6 @@ public void createObject(Object entity, RequestScope scope) { tx.createObject(entity, scope); } - @Override - public Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { - - if (! filterExpression.isPresent() - || tx.supportsFiltering(entityClass, filterExpression.get()) == FeatureSupport.FULL) { - return tx.loadObject(entityClass, id, filterExpression, scope); - } - return DataStoreTransaction.super.loadObject(entityClass, id, filterExpression, scope); - } - - @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - - DataFetcher fetcher = new DataFetcher() { - @Override - public Iterable fetch(Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope scope) { - return tx.loadObjects(entityClass, filterExpression, sorting, pagination, scope); - } - }; - - return (Iterable) fetchData(fetcher, entityClass, - filterExpression, sorting, pagination, false, scope); - } - - @Override - public void close() throws IOException { - tx.close(); - } - private Iterable filterLoadedData(Iterable loadedRecords, Optional filterExpression, RequestScope scope) { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java index f27e4e0219..3171047efc 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java @@ -9,15 +9,16 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; import lombok.Data; import java.io.IOException; import java.io.Serializable; -import java.util.Optional; import java.util.Set; /** @@ -44,16 +45,15 @@ public T createNewObject(Class entityClass) { } @Override - public Object loadObject(Class entityClass, Serializable id, Optional filterExpression, + public Object loadObject(EntityProjection projection, Serializable id, RequestScope scope) { - return tx.loadObject(entityClass, id, filterExpression, scope); + return tx.loadObject(projection, id, scope); } @Override - public Object getRelation(DataStoreTransaction relationTx, Object entity, String relationName, - Optional filterExpression, Optional sorting, - Optional pagination, RequestScope scope) { - return tx.getRelation(relationTx, entity, relationName, filterExpression, sorting, pagination, scope); + public Object getRelation(DataStoreTransaction relationTx, Object entity, + Relationship relationship, RequestScope scope) { + return tx.getRelation(relationTx, entity, relationship, scope); } @Override @@ -71,13 +71,13 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return tx.getAttribute(entity, attributeName, scope); + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { + return tx.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - tx.setAttribute(entity, attributeName, attributeValue, scope); + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { + tx.setAttribute(entity, attribute, scope); } @Override @@ -119,16 +119,11 @@ public void commit(RequestScope requestScope) { @Override public void createObject(Object o, RequestScope requestScope) { tx.createObject(o, requestScope); - } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, - RequestScope requestScope) { - return tx.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { + return tx.loadObjects(projection, scope); } @Override diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java index 3b7be39375..70446b5f19 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java @@ -105,21 +105,34 @@ public FilterPredicate scopedBy(PathElement scope) { } /** - * Returns an alias that uniquely identifies the last collection of entities in the path. - * @return An alias for the path. + * Generate alias for representing a relationship path which dose not include the last field name. + * The path would start with the class alias of the first element, and then each field would append "_fieldName" to + * the result. + * The last field would not be included as that's not a part of the relationship path. + * + * @param path path that represents a relationship chain + * @return relationship path alias, i.e. foo.bar.baz would be foo_bar */ - public String getAlias() { - List elements = path.getPathElements(); + public static String getPathAlias(Path path) { + List elements = path.getPathElements(); + String alias = getTypeAlias(elements.get(0).getType()); - PathElement last = elements.get(elements.size() - 1); - - if (elements.size() == 1) { - return getTypeAlias(last.getType()); + for (int i = 0; i < elements.size() - 1; i++) { + alias = appendAlias(alias, elements.get(i).getFieldName()); } - PathElement previous = elements.get(elements.size() - 2); + return alias; + } - return getTypeAlias(previous.getType()) + UNDERSCORE + previous.getFieldName(); + /** + * Append a new field to a parent alias to get new alias. + * + * @param parentAlias parent path alias + * @param fieldName field name + * @return alias for the field + */ + public static String appendAlias(String parentAlias, String fieldName) { + return parentAlias + "_" + fieldName; } /** diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java index 68e5a55037..3832a1d3ff 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/dialect/RSQLFilterDialect.java @@ -168,7 +168,6 @@ public Map parseTypedExpression(String path, Multivalu return expressionByType; } - /** * Parses a RSQL string into an Elide FilterExpression. * @param expressionText the RSQL string diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java index 9571351ba3..4c87ef7a42 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/AndFilterExpression.java @@ -17,6 +17,32 @@ public class AndFilterExpression implements FilterExpression { @Getter private FilterExpression left; @Getter private FilterExpression right; + /** + * Returns a new {@link AndFilterExpression} instance with the specified null-able left and right operands. + *

+ * The publication rules are + *

    + *
  1. If both left and right are not {@code null}, this method produces the same instance as + * {@link #AndFilterExpression(FilterExpression, FilterExpression)} does, + *
  2. If one of them is {@code null}, the other non-null is returned with no modification, + *
  3. If both left and right are {@code null}, this method returns + * {@code null}. + *
+ * + * @param left The provided left {@link FilterExpression} + * @param right The provided right {@link FilterExpression} + * + * @return a new {@link AndFilterExpression} instance or {@code null} + */ + public static FilterExpression fromPair(FilterExpression left, FilterExpression right) { + if (left != null && right != null) { + return new AndFilterExpression(left, right); + } else if (left == null) { + return right; + } + return left; + } + public AndFilterExpression(FilterExpression left, FilterExpression right) { this.left = left; this.right = right; diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java index 02cfb62ed8..90f5e628ef 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/expression/OrFilterExpression.java @@ -18,6 +18,32 @@ public class OrFilterExpression implements FilterExpression { @Getter private FilterExpression left; @Getter private FilterExpression right; + /** + * Returns a new {@link OrFilterExpression} instance with the specified null-able left and right operands. + *

+ * The publication rules are + *

    + *
  1. If both left and right are not {@code null}, this method produces the same instance as + * {@link #OrFilterExpression(FilterExpression, FilterExpression)} does, + *
  2. If one of them is {@code null}, the other non-null is returned with no modification, + *
  3. If both left and right are {@code null}, this method returns + * {@code null}. + *
+ * + * @param left The provided left {@link FilterExpression} + * @param right The provided right {@link FilterExpression} + * + * @return a new {@link OrFilterExpression} instance or {@code null} + */ + public static FilterExpression fromPair(FilterExpression left, FilterExpression right) { + if (left != null && right != null) { + return new OrFilterExpression(left, right); + } else if (left == null) { + return right; + } + return left; + } + public OrFilterExpression(FilterExpression left, FilterExpression right) { this.left = left; this.right = right; diff --git a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java b/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java index 61ae0b6241..860146762e 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java @@ -85,6 +85,21 @@ private Pagination(Map pageData, int defaultMaxPageSize, this.defaultPageSize = defaultPageSize; } + /** + * Set limit. + * + * @param perPage page size. + */ + public void setLimit(Integer perPage) { + this.limit = perPage; + pageData.put(PaginationKey.limit, perPage); + } + + public void setOffset(Integer offset) { + this.offset = offset; + pageData.put(PaginationKey.offset, offset); + } + /** * TODO - Refactor Pagination. * IMPORTANT - This method should only be used for testing until Pagination is refactored. The @@ -99,8 +114,8 @@ private Pagination(Map pageData, int defaultMaxPageSize, public static Pagination fromOffsetAndLimit(int limit, int offset, boolean generatePageTotals) { ImmutableMap.Builder pageData = ImmutableMap.builder() - .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) - .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), limit); + .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) + .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), limit); if (generatePageTotals) { pageData.put(PAGE_KEYS.get(PAGE_TOTALS_KEY), 1); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java b/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java index 042f80e5d7..ec1d091417 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.Path; import com.yahoo.elide.core.exceptions.InvalidValueException; +import lombok.EqualsAndHashCode; import lombok.ToString; import java.util.Arrays; @@ -23,6 +24,7 @@ * Generates a simple wrapper around the sort fields from the JSON-API GET Query. */ @ToString +@EqualsAndHashCode public class Sorting { /** diff --git a/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java b/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java index 4e1e993cd2..1333e10162 100644 --- a/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/extensions/PatchRequestScope.java @@ -8,6 +8,7 @@ import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.security.User; @@ -50,5 +51,6 @@ public PatchRequestScope( */ public PatchRequestScope(String path, JsonApiDocument jsonApiDocument, PatchRequestScope scope) { super(path, jsonApiDocument, scope); + this.setEntityProjection(new EntityProjectionMaker(dictionary, this).parsePath(path)); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java new file mode 100644 index 0000000000..b3e4eee9c5 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java @@ -0,0 +1,383 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.jsonapi; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.exceptions.InvalidCollectionException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.generated.parsers.CoreBaseVisitor; +import com.yahoo.elide.generated.parsers.CoreParser; +import com.yahoo.elide.parsers.JsonApiParser; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import com.google.common.collect.Sets; +import org.apache.commons.lang3.tuple.Pair; + +import lombok.Builder; +import lombok.Data; + +import java.util.Arrays; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +import javax.ws.rs.core.MultivaluedHashMap; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Converts a JSON-API request (URL and query parameters) into an EntityProjection. + */ +public class EntityProjectionMaker + extends CoreBaseVisitor, EntityProjectionMaker.NamedEntityProjection>> { + + /** + * An entity projection labeled with the class name or relationship name it is associated with. + */ + @Data + @Builder + public static class NamedEntityProjection { + private String name; + private EntityProjection projection; + } + + private static final String INCLUDE = "include"; + + private EntityDictionary dictionary; + private MultivaluedMap queryParams; + private Map> sparseFields; + private RequestScope scope; + + public EntityProjectionMaker(EntityDictionary dictionary, RequestScope scope) { + this.dictionary = dictionary; + this.queryParams = scope.getQueryParams().orElse(new MultivaluedHashMap<>()); + sparseFields = RequestScope.parseSparseFields(queryParams); + this.scope = scope; + } + + public EntityProjection parsePath(String path) { + return visit(JsonApiParser.parse(path)).apply(null).projection; + } + + public EntityProjection parseInclude(Class entityClass) { + return EntityProjection.builder() + .type(entityClass) + .relationships(toRelationshipSet(getIncludedRelationships(entityClass))) + .build(); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionLoadEntities( + CoreParser.RootCollectionLoadEntitiesContext ctx) { + return visitTerminalCollection(ctx.term()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionReadCollection( + CoreParser.SubCollectionReadCollectionContext ctx) { + return visitTerminalCollection(ctx.term()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionSubCollection( + CoreParser.RootCollectionSubCollectionContext ctx) { + return visitEntityWithSubCollection(ctx.entity(), ctx.subCollection()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionSubCollection( + CoreParser.SubCollectionSubCollectionContext ctx) { + return visitEntityWithSubCollection(ctx.entity(), ctx.subCollection()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionRelationship( + CoreParser.RootCollectionRelationshipContext ctx) { + return visitEntityWithRelationship(ctx.entity(), ctx.relationship()); + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionRelationship( + CoreParser.SubCollectionRelationshipContext ctx) { + return visitEntityWithRelationship(ctx.entity(), ctx.relationship()); + } + + @Override + public Function, NamedEntityProjection> visitRootCollectionLoadEntity( + CoreParser.RootCollectionLoadEntityContext ctx) { + return (unused) -> { + return ctx.entity().accept(this).apply(null); + }; + } + + @Override + public Function, NamedEntityProjection> visitSubCollectionReadEntity( + CoreParser.SubCollectionReadEntityContext ctx) { + return (parentClass) -> { + return ctx.entity().accept(this).apply(parentClass); + }; + } + + @Override + public Function, NamedEntityProjection> visitRelationship(CoreParser.RelationshipContext ctx) { + return (parentClass) -> { + String entityName = ctx.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + FilterExpression filter = scope.getExpressionForRelation(parentClass, entityName).orElse(null); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .filterExpression(filter) + .sorting(scope.getSorting()) + .pagination(scope.getPagination()) + .type(entityClass) + .build() + ).build(); + }; + } + + @Override + public Function, NamedEntityProjection> visitEntity(CoreParser.EntityContext ctx) { + return (parentClass) -> { + String entityName = ctx.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .attributes(getSparseAttributes(entityClass)) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .build() + ).build(); + }; + } + + @Override + protected Function, NamedEntityProjection> aggregateResult( + Function, NamedEntityProjection> aggregate, + Function, NamedEntityProjection> nextResult) { + + if (aggregate == null) { + return nextResult; + } else { + return aggregate; + } + } + + public EntityProjection visitIncludePath(Path path) { + Path.PathElement pathElement = path.getPathElements().get(0); + int size = path.getPathElements().size(); + + Class entityClass = pathElement.getFieldType(); + + if (size > 1) { + Path nextPath = new Path(path.getPathElements().subList(1, size)); + EntityProjection relationshipProjection = visitIncludePath(nextPath); + + return EntityProjection.builder() + .relationships(toRelationshipSet(getSparseRelationships(entityClass))) + .relationship(nextPath.getPathElements().get(0).getFieldName(), relationshipProjection) + .attributes(getSparseAttributes(entityClass)) + .filterExpression(scope.getFilterExpressionByType(entityClass).orElse(null)) + .type(entityClass) + .build(); + } + + return EntityProjection.builder() + .relationships(toRelationshipSet(getSparseRelationships(entityClass))) + .attributes(getSparseAttributes(entityClass)) + .type(entityClass) + .filterExpression(scope.getFilterExpressionByType(entityClass).orElse(null)) + .build(); + } + + private Function, NamedEntityProjection> visitEntityWithSubCollection(CoreParser.EntityContext entity, + CoreParser.SubCollectionContext subCollection) { + return (parentClass) -> { + String entityName = entity.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + NamedEntityProjection projection = subCollection.accept(this).apply(entityClass); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .relationship(projection.name, projection.projection) + .build() + ).build(); + }; + } + + private Function, NamedEntityProjection> visitEntityWithRelationship(CoreParser.EntityContext entity, + CoreParser.RelationshipContext relationship) { + return (parentClass) -> { + String entityName = entity.term().getText(); + + Class entityClass = getEntityClass(parentClass, entityName); + + String relationshipName = relationship.term().getText(); + NamedEntityProjection relationshipProjection = relationship.accept(this).apply(entityClass); + + FilterExpression filter = scope.getFilterExpressionByType(entityClass).orElse(null); + + return NamedEntityProjection.builder() + .name(entityName) + .projection(EntityProjection.builder() + .type(entityClass) + .filterExpression(filter) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .relationship(relationshipName, relationshipProjection.projection) + .build() + ).build(); + }; + } + + private Function, NamedEntityProjection> visitTerminalCollection(CoreParser.TermContext collectionName) { + return (parentClass) -> { + String collectionNameText = collectionName.getText(); + + Class entityClass = getEntityClass(parentClass, collectionNameText); + + FilterExpression filter; + if (parentClass == null) { + filter = scope.getLoadFilterExpression(entityClass).orElse(null); + } else { + filter = scope.getExpressionForRelation(parentClass, collectionNameText).orElse(null); + } + + return NamedEntityProjection.builder() + .name(collectionNameText) + .projection(EntityProjection.builder() + .filterExpression(filter) + .sorting(scope.getSorting()) + .pagination(scope.getPagination()) + .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) + .attributes(getSparseAttributes(entityClass)) + .type(entityClass) + .build() + ).build(); + }; + } + + private Class getEntityClass(Class parentClass, String entityLabel) { + + //entityLabel represents a root collection. + if (parentClass == null) { + Class entityClass = dictionary.getEntityClass(entityLabel); + + if (entityClass != null) { + return entityClass; + } + + + //entityLabel represents a relationship. + } else if (dictionary.isRelation(parentClass, entityLabel)) { + return dictionary.getParameterizedType(parentClass, entityLabel); + } + + throw new InvalidCollectionException(entityLabel); + } + + private Map getIncludedRelationships(Class entityClass) { + Set includePaths = getIncludePaths(entityClass); + + Map relationships = includePaths.stream() + .map((path) -> Pair.of(path.getPathElements().get(0).getFieldName(), visitIncludePath(path))) + .collect(Collectors.toMap( + Pair::getKey, + Pair::getValue, + EntityProjection::merge + )); + + return relationships; + } + + private Set getSparseAttributes(Class entityClass) { + Set allAttributes = new HashSet<>(dictionary.getAttributes(entityClass)); + + Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); + if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { + sparseFieldsForEntity = allAttributes; + } + + return Sets.intersection(allAttributes, sparseFieldsForEntity).stream() + .map(attributeName -> Attribute.builder() + .name(attributeName) + .type(dictionary.getParameterizedType(entityClass, attributeName)) + .build()) + .collect(Collectors.toSet()); + } + + private Map getSparseRelationships(Class entityClass) { + Set allRelationships = new HashSet<>(dictionary.getRelationships(entityClass)); + Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); + + if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { + sparseFieldsForEntity = allRelationships; + } + + sparseFieldsForEntity = Sets.intersection(allRelationships, sparseFieldsForEntity); + + return sparseFieldsForEntity.stream() + .collect(Collectors.toMap( + Function.identity(), + (relationshipName) -> { + FilterExpression filter = scope.getExpressionForRelation(entityClass, relationshipName) + .orElse(null); + + return EntityProjection.builder() + .type(dictionary.getParameterizedType(entityClass, relationshipName)) + .filterExpression(filter) + .build(); + } + )); + } + + private Map getRequiredRelationships(Class entityClass) { + return Stream.concat( + getIncludedRelationships(entityClass).entrySet().stream(), + getSparseRelationships(entityClass).entrySet().stream() + ).collect(Collectors.toMap( + Map.Entry::getKey, + Map.Entry::getValue, + EntityProjection::merge + )); + } + + private Set getIncludePaths(Class entityClass) { + if (queryParams.get(INCLUDE) != null) { + return queryParams.get(INCLUDE).stream() + .flatMap(param -> Arrays.stream(param.split(","))) + .map(pathString -> new Path(entityClass, dictionary, pathString)) + .collect(Collectors.toSet()); + } + + return new HashSet<>(); + } + + private Set toRelationshipSet(Map relationships) { + return relationships.entrySet().stream() + .map(entry -> Relationship.builder() + .name(entry.getKey()) + .alias(entry.getKey()) + .projection(entry.getValue()) + .build()) + .collect(Collectors.toSet()); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java index 40056b0c11..a7805da4bb 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessor.java @@ -7,8 +7,10 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; -import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.jsonapi.EntityProjectionMaker; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Lists; @@ -60,13 +62,16 @@ public void execute(JsonApiDocument jsonApiDocument, Set res */ private void addIncludedResources(JsonApiDocument jsonApiDocument, PersistentResource rec, List requestedRelationPaths) { + + EntityProjectionMaker maker = new EntityProjectionMaker(rec.getDictionary(), rec.getRequestScope()); + EntityProjection projection = maker.parseInclude(rec.getResourceClass()); // Process each include relation path requestedRelationPaths.forEach(pathParam -> { List pathList = Arrays.asList(pathParam.split(RELATION_PATH_SEPARATOR)); pathList.forEach(requestedRelationPath -> { List relationPath = Lists.newArrayList(requestedRelationPath.split(RELATION_PATH_DELIMITER)); - addResourcesForPath(jsonApiDocument, rec, relationPath); + addResourcesForPath(jsonApiDocument, rec, relationPath, projection); }); }); } @@ -76,15 +81,17 @@ private void addIncludedResources(JsonApiDocument jsonApiDocument, PersistentRes * JsonApiDocument. */ private void addResourcesForPath(JsonApiDocument jsonApiDocument, PersistentResource rec, - List relationPath) { + List relationPath, + EntityProjection projection) { //Pop off a relation of relation path String relation = relationPath.remove(0); - Optional filterExpression = rec.getRequestScope().getExpressionForRelation(rec, relation); Set collection; + Relationship relationship = projection.getRelationship(relation).orElseThrow(IllegalStateException::new); try { - collection = rec.getRelationCheckedFiltered(relation, filterExpression, Optional.empty(), Optional.empty()); + collection = rec.getRelationCheckedFiltered(relationship); + } catch (ForbiddenAccessException e) { return; } @@ -95,7 +102,8 @@ private void addResourcesForPath(JsonApiDocument jsonApiDocument, PersistentReso //If more relations left in the path, process a level deeper if (!relationPath.isEmpty()) { //Use a copy of the relationPath to preserve the path for remaining branches of the relationship tree - addResourcesForPath(jsonApiDocument, resource, new ArrayList<>(relationPath)); + addResourcesForPath(jsonApiDocument, resource, new ArrayList<>(relationPath), + relationship.getProjection()); } }); } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java index 9b71f59c6d..ac6cd3388a 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/Resource.java @@ -5,12 +5,14 @@ */ package com.yahoo.elide.jsonapi.models; +import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; import com.yahoo.elide.core.exceptions.UnknownEntityException; import com.yahoo.elide.jsonapi.serialization.KeySerializer; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.annotation.JsonInclude; import com.fasterxml.jackson.annotation.JsonProperty; @@ -147,13 +149,20 @@ public boolean equals(Object obj) { public PersistentResource toPersistentResource(RequestScope requestScope) throws ForbiddenAccessException, InvalidObjectIdentifierException { - Class cls = requestScope.getDictionary().getEntityClass(type); + EntityDictionary dictionary = requestScope.getDictionary(); + Class cls = dictionary.getEntityClass(type); + if (cls == null) { throw new UnknownEntityException(type); } if (id == null) { throw new InvalidObjectIdentifierException(id, type); } - return PersistentResource.loadRecord(cls, id, requestScope); + + EntityProjection projection = EntityProjection.builder() + .type(cls) + .build(); + + return PersistentResource.loadRecord(projection, id, requestScope); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java index 5cf97420b5..726178e883 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/models/ResourceIdentifier.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.annotation.JsonProperty; @@ -38,7 +39,9 @@ public String getId() { public PersistentResource toPersistentResource(RequestScope requestScope) throws ForbiddenAccessException, InvalidObjectIdentifierException { Class cls = requestScope.getDictionary().getEntityClass(type); - return PersistentResource.loadRecord(cls, id, requestScope); + return PersistentResource.loadRecord(EntityProjection.builder() + .type(cls) + .build(), id, requestScope); } public Resource castToResource() { diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java index 7673900c80..7ca47bdd33 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java @@ -14,9 +14,7 @@ import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; import com.yahoo.elide.core.exceptions.InvalidValueException; import com.yahoo.elide.core.exceptions.UnknownEntityException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.document.processors.DocumentProcessor; import com.yahoo.elide.jsonapi.document.processors.IncludedProcessor; @@ -25,11 +23,11 @@ import com.yahoo.elide.jsonapi.models.Meta; import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.JsonNode; import com.google.common.base.Preconditions; - import org.apache.commons.collections4.IterableUtils; import org.apache.commons.lang3.tuple.Pair; @@ -56,9 +54,11 @@ public class CollectionTerminalState extends BaseState { private final Optional relationName; private final Class entityClass; private PersistentResource newObject; + private final EntityProjection parentProjection; public CollectionTerminalState(Class entityClass, Optional parent, - Optional relationName) { + Optional relationName, EntityProjection projection) { + this.parentProjection = projection; this.parent = parent; this.relationName = relationName; this.entityClass = entityClass; @@ -126,27 +126,13 @@ private Set getResourceCollection(RequestScope requestScope) // TODO: In case of join filters, apply pagination after getting records // instead of passing it to the datastore - Optional pagination = Optional.ofNullable(requestScope.getPagination()); - Optional sorting = Optional.ofNullable(requestScope.getSorting()); - if (parent.isPresent()) { - Optional filterExpression = - requestScope.getExpressionForRelation(parent.get(), relationName.get()); - collection = parent.get().getRelationCheckedFiltered( - relationName.get(), - filterExpression, - sorting, - pagination); + parentProjection.getRelationship(relationName.get()).orElseThrow(IllegalStateException::new)); } else { - Optional filterExpression = requestScope.getLoadFilterExpression(entityClass); - collection = PersistentResource.loadRecords( - entityClass, + parentProjection, new ArrayList<>(), //Empty list of IDs - filterExpression, - sorting, - pagination, requestScope); } @@ -187,8 +173,8 @@ private PersistentResource createObject(RequestScope requestScope) + " to type: " + entityClass); } - PersistentResource pResource = PersistentResource.createObject( - parent.orElse(null), newObjectClass, requestScope, Optional.ofNullable(id)); + PersistentResource pResource = PersistentResource.createObject(parent.orElse(null), newObjectClass, + requestScope, Optional.ofNullable(id)); Map attributes = resource.getAttributes(); if (attributes != null) { diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java index f1c759fc22..a49ffcbb23 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RecordState.java @@ -8,14 +8,13 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RelationshipType; -import com.yahoo.elide.core.exceptions.InvalidAttributeException; -import com.yahoo.elide.core.exceptions.InvalidCollectionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionReadCollectionContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionReadEntityContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionRelationshipContext; import com.yahoo.elide.generated.parsers.CoreParser.SubCollectionSubCollectionContext; import com.yahoo.elide.jsonapi.models.SingleElementSet; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.base.Preconditions; @@ -28,54 +27,52 @@ public class RecordState extends BaseState { private final PersistentResource resource; - public RecordState(PersistentResource resource) { + /* The projection which loaded this record */ + private final EntityProjection projection; + + public RecordState(PersistentResource resource, EntityProjection projection) { Preconditions.checkNotNull(resource); this.resource = resource; + this.projection = projection; } @Override public void handle(StateContext state, SubCollectionReadCollectionContext ctx) { String subCollection = ctx.term().getText(); EntityDictionary dictionary = state.getRequestScope().getDictionary(); + Class entityClass; String entityName; - try { - RelationshipType type = dictionary.getRelationshipType(resource.getObject(), subCollection); - if (type == RelationshipType.NONE) { - throw new InvalidCollectionException(subCollection); - } - Class paramType = dictionary.getParameterizedType(resource.getObject(), subCollection); - if (dictionary.isMappedInterface(paramType)) { - entityName = EntityDictionary.getSimpleName(paramType); - entityClass = paramType; - } else { - entityName = dictionary.getJsonAliasFor(paramType); - entityClass = dictionary.getEntityClass(entityName); - - } - if (entityClass == null) { - throw new IllegalArgumentException("Unknown type " + entityName); - } - final BaseState nextState; - final CollectionTerminalState collectionTerminalState = - new CollectionTerminalState(entityClass, Optional.of(resource), Optional.of(subCollection)); - Set collection = null; - if (type.isToOne()) { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(resource, subCollection); - collection = resource.getRelationCheckedFiltered(subCollection, - filterExpression, Optional.empty(), Optional.empty()); - } - if (collection instanceof SingleElementSet) { - PersistentResource record = ((SingleElementSet) collection).getValue(); - nextState = new RecordTerminalState(record, collectionTerminalState); - } else { - nextState = collectionTerminalState; - } - state.setState(nextState); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); + + RelationshipType type = dictionary.getRelationshipType(resource.getObject(), subCollection); + + Class paramType = dictionary.getParameterizedType(resource.getObject(), subCollection); + if (dictionary.isMappedInterface(paramType)) { + entityName = EntityDictionary.getSimpleName(paramType); + entityClass = paramType; + } else { + entityName = dictionary.getJsonAliasFor(paramType); + entityClass = dictionary.getEntityClass(entityName); } + if (entityClass == null) { + throw new IllegalArgumentException("Unknown type " + entityName); + } + final BaseState nextState; + final CollectionTerminalState collectionTerminalState = + new CollectionTerminalState(entityClass, Optional.of(resource), + Optional.of(subCollection), projection); + Set collection = null; + if (type.isToOne()) { + collection = resource.getRelationCheckedFiltered(projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new)); + } + if (collection instanceof SingleElementSet) { + PersistentResource record = ((SingleElementSet) collection).getValue(); + nextState = new RecordTerminalState(record, collectionTerminalState); + } else { + nextState = collectionTerminalState; + } + state.setState(nextState); } @Override @@ -83,46 +80,35 @@ public void handle(StateContext state, SubCollectionReadEntityContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); - try { - PersistentResource nextRecord = resource.getRelation(subCollection, id); - state.setState(new RecordTerminalState(nextRecord)); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } + PersistentResource nextRecord = resource.getRelation( + projection.getRelationship(subCollection).orElseThrow(IllegalStateException::new), id); + state.setState(new RecordTerminalState(nextRecord)); } @Override public void handle(StateContext state, SubCollectionSubCollectionContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); - try { - state.setState(new RecordState(resource.getRelation(subCollection, id))); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } + + Relationship relationship = projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new); + + state.setState(new RecordState(resource.getRelation(relationship, id), relationship.getProjection())); } @Override public void handle(StateContext state, SubCollectionRelationshipContext ctx) { String id = ctx.entity().id().getText(); String subCollection = ctx.entity().term().getText(); + String relationName = ctx.relationship().term().getText(); PersistentResource childRecord; - try { - childRecord = resource.getRelation(subCollection, id); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(subCollection); - } - String relationName = ctx.relationship().term().getText(); - try { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(resource, subCollection); - childRecord.getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(relationName); - } + Relationship childRelationship = projection.getRelationship(subCollection) + .orElseThrow(IllegalStateException::new); + + childRecord = resource.getRelation(childRelationship , id); - state.setState(new RelationshipTerminalState(childRecord, relationName)); + state.setState(new RelationshipTerminalState(childRecord, relationName, childRelationship.getProjection())); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java index 0c0ed93346..ae096090a0 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java @@ -18,6 +18,7 @@ import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.databind.JsonNode; @@ -41,8 +42,13 @@ public class RelationshipTerminalState extends BaseState { private final RelationshipType relationshipType; private final String relationshipName; - public RelationshipTerminalState(PersistentResource record, String relationshipName) { + /* The projection which loaded the resource which owns the relationship */ + private final EntityProjection parentProjection; + + public RelationshipTerminalState(PersistentResource record, String relationshipName, + EntityProjection parentProjection) { this.record = record; + this.parentProjection = parentProjection; this.relationshipType = record.getRelationshipType(relationshipName); this.relationshipName = relationshipName; @@ -55,7 +61,8 @@ public Supplier> handleGet(StateContext state) { JsonApiMapper mapper = requestScope.getMapper(); Optional> queryParams = requestScope.getQueryParams(); - Map relationships = record.toResourceWithSortingAndPagination().getRelationships(); + Map relationships = record.toResource(parentProjection).getRelationships(); + Relationship relationship = null; if (relationships != null) { Relationship relationship = relationships.get(relationshipName); diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java index dfdba1827c..7b6734a85b 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/StartState.java @@ -7,14 +7,12 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.exceptions.InvalidAttributeException; -import com.yahoo.elide.core.exceptions.InvalidCollectionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.generated.parsers.CoreParser.EntityContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionLoadEntitiesContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionLoadEntityContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionRelationshipContext; import com.yahoo.elide.generated.parsers.CoreParser.RootCollectionSubCollectionContext; +import com.yahoo.elide.request.EntityProjection; import java.util.Optional; @@ -27,10 +25,9 @@ public void handle(StateContext state, RootCollectionLoadEntitiesContext ctx) { String entityName = ctx.term().getText(); EntityDictionary dictionary = state.getRequestScope().getDictionary(); Class entityClass = dictionary.getEntityClass(entityName); - if (entityClass == null || !dictionary.isRoot(entityClass)) { - throw new InvalidCollectionException(entityName); - } - state.setState(new CollectionTerminalState(entityClass, Optional.empty(), Optional.empty())); + + state.setState(new CollectionTerminalState(entityClass, Optional.empty(), Optional.empty(), + state.getRequestScope().getEntityProjection())); } @Override @@ -42,23 +39,21 @@ public void handle(StateContext state, RootCollectionLoadEntityContext ctx) { @Override public void handle(StateContext state, RootCollectionSubCollectionContext ctx) { PersistentResource record = entityRecord(state, ctx.entity()); - state.setState(new RecordState(record)); + + state.setState(new RecordState(record, state.getRequestScope().getEntityProjection())); } @Override public void handle(StateContext state, RootCollectionRelationshipContext ctx) { PersistentResource record = entityRecord(state, ctx.entity()); + EntityProjection projection = state.getRequestScope().getEntityProjection(); String relationName = ctx.relationship().term().getText(); - try { - Optional filterExpression = - state.getRequestScope().getExpressionForRelation(record, relationName); - record.getRelationCheckedFiltered(relationName, filterExpression, Optional.empty(), Optional.empty()); - } catch (InvalidAttributeException e) { - throw new InvalidCollectionException(relationName); - } - state.setState(new RelationshipTerminalState(record, relationName)); + record.getRelationCheckedFiltered(projection.getRelationship(relationName) + .orElseThrow(IllegalStateException::new)); + + state.setState(new RelationshipTerminalState(record, relationName, projection)); } @Override @@ -67,14 +62,9 @@ public String toString() { } private PersistentResource entityRecord(StateContext state, EntityContext entity) { - String entityName = entity.term().getText(); String id = entity.id().getText(); - EntityDictionary dictionary = state.getRequestScope().getDictionary(); - Class entityClass = dictionary.getEntityClass(entityName); - if (entityClass == null || !dictionary.isRoot(entityClass)) { - throw new InvalidCollectionException(entityName); - } - return PersistentResource.loadRecord(entityClass, id, state.getRequestScope()); + return PersistentResource.loadRecord(state.getRequestScope().getEntityProjection(), + id, state.getRequestScope()); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Argument.java b/elide-core/src/main/java/com/yahoo/elide/request/Argument.java new file mode 100644 index 0000000000..9cc32089ba --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Argument.java @@ -0,0 +1,32 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Represents an argument passed to an attribute. + */ +@Data +@Builder +public class Argument { + + @NonNull + String name; + + Object value; + + /** + * Returns the argument type. + * @return the argument type. + */ + public Class getType() { + return value.getClass(); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java b/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java new file mode 100644 index 0000000000..7a05e71fd4 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Attribute.java @@ -0,0 +1,43 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; +import lombok.Singular; +import lombok.ToString; + +import java.util.Set; + +/** + * Represents an attribute on an Elide entity. Attributes can take arguments. + */ +@Data +@Builder +public class Attribute { + @NonNull + @ToString.Exclude + private Class type; + + @NonNull + private String name; + + @ToString.Exclude + private String alias; + + @Singular + @ToString.Exclude + private Set arguments; + + private Attribute(@NonNull Class type, @NonNull String name, String alias, Set arguments) { + this.type = type; + this.name = name; + this.alias = alias == null ? name : alias; + this.arguments = arguments; + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java new file mode 100644 index 0000000000..da7a5c8caa --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java @@ -0,0 +1,261 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; + +import com.google.common.collect.Sets; +import lombok.AllArgsConstructor; +import lombok.Builder; +import lombok.Data; +import lombok.Getter; +import lombok.NonNull; + +import java.util.LinkedHashSet; +import java.util.Optional; +import java.util.Set; + +import javax.ws.rs.BadRequestException; + +/** + * Represents a client data request against a subgraph of the entity relationship graph. + */ +@Data +@Builder +@AllArgsConstructor +public class EntityProjection { + @NonNull + private Class type; + + private Set attributes; + + private Set relationships; + + private FilterExpression filterExpression; + + private Sorting sorting; + + private Pagination pagination; + + /** + * Creates a builder initialized as a copy of this collection + * @return The new builder + */ + public EntityProjectionBuilder copyOf() { + return EntityProjection.builder() + .type(this.type) + .attributes(new LinkedHashSet<>(attributes)) + .relationships(new LinkedHashSet<>(this.relationships)) + .filterExpression(this.filterExpression) + .sorting(this.sorting) + .pagination(this.pagination); + } + + /** + * Returns a relationship subgraph by name. + * @param name The name of the relationship. + * @return + */ + public Optional getRelationship(String name) { + return relationships.stream() + .filter((relationship) -> relationship.getName().equalsIgnoreCase(name)) + .findFirst(); + } + + /** + * Returns a relationship subgraph by name. + * @param name The name of the relationship. + * @param name The alias of the relationship. + * @return + */ + public Optional getRelationship(String name, String alias) { + return relationships.stream() + .filter((relationship) -> relationship.getName().equalsIgnoreCase(name)) + .filter((relationship) -> relationship.getAlias().equalsIgnoreCase(alias)) + .findFirst(); + } + + /** + * Recursively merges two EntityProjections. + * @param toMerge The projection to merge + * @return A newly created and merged EntityProjection. + */ + public EntityProjection merge(EntityProjection toMerge) { + EntityProjectionBuilder merged = copyOf(); + + for (Relationship relationship: toMerge.getRelationships()) { + EntityProjection theirs = relationship.getProjection(); + + Relationship ourRelationship = getRelationship(relationship.getName(), + relationship.getAlias()).orElse(null); + + if (ourRelationship != null) { + merged.relationships.remove(ourRelationship); + merged.relationships.add((Relationship.builder() + .name(relationship.getName()) + .alias(relationship.getAlias()) + .projection(ourRelationship.getProjection().merge(theirs)) + .build())); + } else { + merged.relationships.add((relationship)); + } + } + if (toMerge.getPagination() != null) { + merged.pagination = toMerge.getPagination(); + } + + if (toMerge.getSorting() != null) { + merged.sorting = toMerge.getSorting(); + } + + if (toMerge.getFilterExpression() != null) { + merged.filterExpression = toMerge.getFilterExpression(); + } + + merged.attributes.addAll(toMerge.attributes); + + return merged.build(); + } + + /** + * Customizes the lombok builder to our needs. + */ + public static class EntityProjectionBuilder { + @Getter + private Class type; + + private Set relationships = new LinkedHashSet<>(); + + private Set attributes = new LinkedHashSet<>(); + + @Getter + private FilterExpression filterExpression; + + @Getter + private Sorting sorting; + + @Getter + private Pagination pagination; + + public EntityProjectionBuilder relationships(Set relationships) { + this.relationships = relationships; + return this; + } + + public EntityProjectionBuilder attributes(Set attributes) { + this.attributes = attributes; + return this; + } + + public EntityProjectionBuilder relationship(String name, EntityProjection projection) { + return relationship(Relationship.builder() + .alias(name) + .name(name) + .projection(projection) + .build()); + } + + /** + * Add a new relationship into this project or merge an existing relationship that has same field name + * and alias as this relationship. If there exists another attribute/relationship of different field that is + * using the same alias, it would throw exception because that's ambiguous. + * + * @param relationship new relationship to add + * @return this builder after adding the relationship + */ + public EntityProjectionBuilder relationship(Relationship relationship) { + String relationshipName = relationship.getName(); + String relationshipAlias = relationship.getAlias(); + + Relationship existing = relationships.stream() + .filter(r -> r.getName().equals(relationshipName) && r.getAlias().equals(relationshipAlias)) + .findFirst().orElse(null); + + if (existing != null) { + relationships.remove(existing); + relationships.add(Relationship.builder() + .name(relationshipName) + .alias(relationshipAlias) + .projection(existing.getProjection().merge(relationship.getProjection())) + .build()); + } else { + if (isAmbiguous(relationshipName, relationshipAlias)) { + throw new BadRequestException( + String.format("Alias {%s}.{%s} is ambiguous.", type, relationshipAlias) + ); + } + relationships.add(relationship); + } + + return this; + } + + /** + * Add a new attribute into this project or merge an existing attribute that has same field name + * and alias as this attribute. If there exists another attribute/relationship of different field that is + * using the same alias, it would throw exception because that's ambiguous. + * + * @param attribute new attribute to add + * @return this builder after adding the attribute + */ + public EntityProjectionBuilder attribute(Attribute attribute) { + String attributeName = attribute.getName(); + String attributeAlias = attribute.getAlias(); + + Attribute existing = attributes.stream() + .filter(a -> a.getName().equals(attributeName) && a.getAlias().equals(attributeAlias)) + .findFirst().orElse(null); + + if (existing != null) { + attributes.remove(existing); + attributes.add(Attribute.builder() + .type(attribute.getType()) + .name(attributeName) + .alias(attributeAlias) + .arguments(Sets.union(attribute.getArguments(), existing.getArguments())) + .build()); + } else { + if (isAmbiguous(attributeName, attributeAlias)) { + throw new BadRequestException( + String.format("Alias {%s}.{%s} is ambiguous.", type, attributeAlias) + ); + } + attributes.add(attribute); + } + + return this; + } + + /** + * Get an attribute by alias. + * + * @param attributeAlias alias to refer to an attribute field + * @return found attribute or null + */ + public Attribute getAttributeByAlias(String attributeAlias) { + return attributes.stream() + .filter(attribute -> attribute.getAlias().equals(attributeAlias)) + .findAny() + .orElse(null); + } + + /** + * Check whether a field alias is ambiguous. + * + * @param fieldName field that the alias is bound to + * @param alias an field alias + * @return whether new alias would cause ambiguous + */ + private boolean isAmbiguous(String fieldName, String alias) { + return attributes.stream().anyMatch(a -> !fieldName.equals(a.getName()) && alias.equals(a.getAlias())) + || relationships.stream().anyMatch( + r -> !fieldName.equals(r.getName()) && alias.equals(r.getAlias())); + } + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java b/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java new file mode 100644 index 0000000000..5cbbad0014 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Relationship.java @@ -0,0 +1,48 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Represents a relationship on an Elide entity. + */ +@Data +@Builder +public class Relationship { + + public RelationshipBuilder copyOf() { + return Relationship.builder() + .alias(alias) + .name(name) + .projection(projection); + } + + @NonNull + private String name; + + private String alias; + + @NonNull + private EntityProjection projection; + + private Relationship(@NonNull String name, String alias, @NonNull EntityProjection projection) { + this.name = name; + this.alias = alias == null ? name : alias; + this.projection = projection; + } + + public Relationship merge(Relationship toMerge) { + return Relationship.builder() + .name(name) + .alias(alias) + .projection(projection.merge(toMerge.projection)) + .build(); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java index 4b650ac189..6a03492201 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java @@ -18,6 +18,7 @@ import java.util.Optional; import java.util.function.Predicate; +import javax.inject.Inject; /** * Check for FilterExpression. This is a super class for user defined FilterExpression check. The subclass should @@ -28,6 +29,9 @@ @Slf4j public abstract class FilterExpressionCheck extends InlineCheck { + @Inject + protected EntityDictionary dictionary; + /** * Returns a FilterExpression from FilterExpressionCheck. * @@ -43,7 +47,6 @@ public final boolean ok(User user) { throw new UnsupportedOperationException(); } - /** * The filter expression is evaluated in memory if it cannot be pushed to the data store by elide for any reason. * @@ -54,7 +57,7 @@ public final boolean ok(User user) { */ @Override public final boolean ok(T object, RequestScope requestScope, Optional changeSpec) { - Class entityClass = coreScope(requestScope).getDictionary().lookupBoundClass(object.getClass()); + Class entityClass = dictionary.lookupBoundClass(object.getClass()); FilterExpression filterExpression = getFilterExpression(entityClass, requestScope); return filterExpression.accept(new FilterExpressionCheckEvaluationVisitor(object, this, requestScope)); } @@ -89,8 +92,7 @@ public boolean applyPredicateToObject(T object, FilterPredicate filterPredicate, * @param defaultPath path to use if no FieldExpressionPath defined * @return Predicates */ - protected static Path getFieldPath(Class type, RequestScope requestScope, String method, String defaultPath) { - EntityDictionary dictionary = coreScope(requestScope).getDictionary(); + protected Path getFieldPath(Class type, RequestScope requestScope, String method, String defaultPath) { try { FilterExpressionPath fep = getFilterExpressionPath(type, method, dictionary); return new Path(type, dictionary, fep == null ? defaultPath : fep.value()); diff --git a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java index f078fc90e7..3311c28377 100644 --- a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java +++ b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java @@ -10,6 +10,7 @@ import io.github.classgraph.ScanResult; import java.lang.annotation.Annotation; +import java.util.HashSet; import java.util.Set; import java.util.stream.Collectors; @@ -46,15 +47,19 @@ static public Set> getAnnotatedClasses(String packageName, Class> getAnnotatedClasses(Class annotation) { + static public Set> getAnnotatedClasses(Class ...annotations) { + Set> result = new HashSet<>(); try (ScanResult scanResult = new ClassGraph().enableAllInfo().scan()) { - return scanResult.getClassesWithAnnotation(annotation.getCanonicalName()).stream() - .map((ClassInfo::loadClass)) - .collect(Collectors.toSet()); + for (Class annotation : annotations) { + result.addAll(scanResult.getClassesWithAnnotation(annotation.getCanonicalName()).stream() + .map((ClassInfo::loadClass)) + .collect(Collectors.toSet())); + } } + return result; } /** diff --git a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java index b767465c88..918a3ecf61 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java @@ -23,9 +23,12 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.annotation.SecurityCheck; import com.yahoo.elide.core.exceptions.InvalidAttributeException; +import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.functions.LifeCycleHook; import com.yahoo.elide.models.generics.Employee; import com.yahoo.elide.models.generics.Manager; +import com.yahoo.elide.security.FilterExpressionCheck; +import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.UserCheck; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; @@ -60,11 +63,12 @@ import java.util.Map; import java.util.Set; import java.util.stream.Collectors; - +import javax.inject.Inject; import javax.persistence.AccessType; import javax.persistence.Entity; import javax.persistence.GeneratedValue; import javax.persistence.Id; +import javax.persistence.OneToOne; import javax.persistence.Transient; public class EntityDictionaryTest extends EntityDictionary { @@ -109,7 +113,7 @@ public void testFindCheckByExpression() { public void testCheckScan() { @SecurityCheck("User is Admin") - class Foo extends UserCheck { + class Bar extends UserCheck { @Override public boolean ok(com.yahoo.elide.security.User user) { @@ -120,7 +124,34 @@ public boolean ok(com.yahoo.elide.security.User user) { EntityDictionary testDictionary = new EntityDictionary(new HashMap<>()); testDictionary.scanForSecurityChecks(); - assertEquals("User is Admin", testDictionary.getCheckIdentifier(Foo.class)); + assertEquals("User is Admin", testDictionary.getCheckIdentifier(Bar.class)); + } + + @Test + public void testCheckInjection() { + + @SecurityCheck("Filter Expression Injection Test") + class Foo extends FilterExpressionCheck { + + @Inject + Long testLong; + + @Override + public FilterExpression getFilterExpression(Class entityClass, RequestScope requestScope) { + assertEquals(testLong, 123L); + return null; + } + } + + EntityDictionary testDictionary = new EntityDictionary(new HashMap<>(), new Injector() { + @Override + public void inject(Object entity) { + ((Foo) entity).testLong = 123L; + } + }); + testDictionary.scanForSecurityChecks(); + + assertEquals("Filter Expression Injection Test", testDictionary.getCheckIdentifier(Foo.class)); } @Test @@ -527,7 +558,7 @@ class SubsubclassBinding extends SubclassBinding { bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupEntityClass(SuperclassBinding.class)); @@ -564,11 +595,10 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SubsubclassBinding.class, getEntityBinding(SubsubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupEntityClass(SuperclassBinding.class)); @@ -605,15 +635,13 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); - bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); - assertEquals(SubsubclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); } @Test @@ -634,15 +662,14 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertEquals(SubsubclassBinding.class, getEntityBinding(SubsubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); assertEquals(SubsubclassBinding.class, lookupIncludeClass(SubsubclassBinding.class)); } @@ -662,17 +689,16 @@ class SubsubclassBinding extends SubclassBinding { } bindEntity(SuperclassBinding.class); - bindEntity(SubclassBinding.class); bindEntity(SubsubclassBinding.class); - assertEquals(SubclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); + assertEquals(SuperclassBinding.class, getEntityBinding(SubclassBinding.class).entityClass); assertEquals(SuperclassBinding.class, getEntityBinding(SuperclassBinding.class).entityClass); assertThrows(IllegalArgumentException.class, () -> { getEntityBinding(SubsubclassBinding.class); }); assertEquals(SuperclassBinding.class, lookupIncludeClass(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, lookupIncludeClass(SubclassBinding.class)); + assertEquals(SuperclassBinding.class, lookupIncludeClass(SubclassBinding.class)); assertEquals(null, lookupIncludeClass(SubsubclassBinding.class)); } @@ -795,4 +821,16 @@ class CoerceBean { bean.map); assertEquals(ImmutableSet.of(3.0, 4.0), bean.set); } + + @Test + public void testAttributeOrRelationAnnotationExists() { + assertTrue(attributeOrRelationAnnotationExists(Job.class, "jobId", Id.class)); + assertFalse(attributeOrRelationAnnotationExists(Job.class, "title", OneToOne.class)); + } + + @Test + public void testIsValidField() { + assertTrue(isValidField(Job.class, "title")); + assertFalse(isValidField(Job.class, "foo")); + } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java index 0e650a1add..a19a6f6f7e 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java @@ -46,6 +46,7 @@ import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; import com.yahoo.elide.core.datastore.inmemory.InMemoryDataStore; import com.yahoo.elide.functions.LifeCycleHook; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.User; import com.yahoo.elide.security.checks.Check; @@ -58,7 +59,6 @@ import example.Book; import example.Editor; import example.Publisher; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; @@ -66,7 +66,6 @@ import java.util.Arrays; import java.util.HashMap; import java.util.HashSet; -import java.util.Map; import java.util.Optional; import javax.persistence.Entity; @@ -89,7 +88,6 @@ public class LifeCycleTest { private MockCallback onUpdatePostCommitCallback; private MockCallback onUpdatePostCommitAuthor; - public class MockCallback implements LifeCycleHook { @Override public void execute(T object, com.yahoo.elide.security.RequestScope scope, Optional changes) { @@ -97,29 +95,13 @@ public void execute(T object, com.yahoo.elide.security.RequestScope scope, Optio } } - public class TestEntityDictionary extends EntityDictionary { - public TestEntityDictionary(Map> checks) { - super(checks); - } - - @Override - public Class lookupBoundClass(Class objClass) { - // Special handling for mocked Book class which has Entity annotation - if (objClass.getName().contains("$MockitoMock$")) { - objClass = objClass.getSuperclass(); - } - return super.lookupBoundClass(objClass); - } - - } - LifeCycleTest() throws Exception { callback = mock(MockCallback.class); onUpdateDeferredCallback = mock(MockCallback.class); onUpdateImmediateCallback = mock(MockCallback.class); onUpdatePostCommitCallback = mock(MockCallback.class); onUpdatePostCommitAuthor = mock(MockCallback.class); - dictionary = new TestEntityDictionary(TestCheckMappings.MAPPINGS); + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Book.class); dictionary.bindEntity(Author.class); dictionary.bindEntity(Publisher.class); @@ -261,7 +243,7 @@ public void testElideGetRelationship() throws Exception { when(store.beginReadTransaction()).thenCallRealMethod(); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); MultivaluedMap headers = new MultivaluedHashMap<>(); ElideResponse response = elide.get("/book/1/relationships/authors", headers, null); @@ -291,7 +273,7 @@ public void testElidePatch() throws Exception { when(book.getId()).thenReturn(1L); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); String bookBody = "{\"data\":{\"type\":\"book\",\"id\":1,\"attributes\": {\"title\":\"Grapes of Wrath\"}}}"; @@ -378,7 +360,7 @@ public void testElideDelete() throws Exception { when(book.getId()).thenReturn(1L); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); ElideResponse response = elide.delete("/book/1", "", null); assertEquals(HttpStatus.SC_NO_CONTENT, response.getResponseCode()); @@ -405,7 +387,7 @@ public void testCreate() { DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.createNewObject(Book.class)).thenReturn(book); RequestScope scope = buildRequestScope(dictionary, tx); - PersistentResource resource = PersistentResource.createObject(null, Book.class, scope, Optional.of("uuid")); + PersistentResource resource = PersistentResource.createObject(Book.class, scope, Optional.of("uuid")); resource.setValueChecked("title", "should not affect calls since this is create!"); resource.setValueChecked("genre", "boring books"); assertNotNull(resource); @@ -642,8 +624,8 @@ public void testUpdateRelationshipWithChangeSpec() { book.setAuthors(Sets.newHashSet(author)); author.setBooks(Sets.newHashSet(book)); DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(author), eq("books"), any(), any(), any(), any())).then((i) -> author.getBooks()); - when(tx.getRelation(any(), eq(book), eq("authors"), any(), any(), any(), any())).then((i) -> book.getAuthors()); + when(tx.getRelation(any(), eq(author), any(), any())).then((i) -> author.getBooks()); + when(tx.getRelation(any(), eq(book), any(), any())).then((i) -> book.getAuthors()); RequestScope scope = buildRequestScope(dictionary, tx); PersistentResource resourceBook = new PersistentResource(book, null, scope.getUUIDFor(book), scope); @@ -766,7 +748,7 @@ public void blowUp(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -790,7 +772,7 @@ public void blowUp(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -844,7 +826,7 @@ public void readPostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -906,7 +888,7 @@ public void updatePostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -968,7 +950,7 @@ public void createPostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -976,6 +958,7 @@ public void createPostCommit(RequestScope scope) { when(tx.createNewObject(Book.class)).thenReturn(book); RequestScope scope = buildRequestScope(dictionary, tx); PersistentResource bookResource = PersistentResource.createObject(null, Book.class, scope, Optional.of("123")); + PersistentResource bookResource = PersistentResource.createObject(Book.class, scope, Optional.of("123")); bookResource.updateAttribute("title", "Foo"); assertEquals(0, book.createPreSecurityInvoked); @@ -1032,7 +1015,7 @@ public void deletePostCommit(RequestScope scope) { } } - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); DataStoreTransaction tx = mock(DataStoreTransaction.class); dictionary.bindEntity(Book.class); @@ -1064,12 +1047,16 @@ public void testAddToCollectionTrigger() { HashMap> checkMappings = new HashMap<>(); checkMappings.put("Book operation check", Book.BookOperationCheck.class); checkMappings.put("Field path editor check", Editor.FieldPathFilterExpression.class); - store.populateEntityDictionary(new EntityDictionary(checkMappings)); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); + + store.populateEntityDictionary(dictionary); DataStoreTransaction tx = store.beginTransaction(); RequestScope scope = buildRequestScope(wrapped.getDictionary(), tx); - PersistentResource publisherResource = PersistentResource.createObject(null, Publisher.class, scope, Optional.of("1")); + + PersistentResource publisherResource = PersistentResource.createObject(Publisher.class, scope, Optional.of("1")); PersistentResource book1Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("1")); + publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book1Resource))); scope.runQueuedPreCommitTriggers(); @@ -1085,7 +1072,10 @@ public void testAddToCollectionTrigger() { scope = buildRequestScope(wrapped.getDictionary(), tx); PersistentResource book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); - publisherResource = PersistentResource.loadRecord(Publisher.class, "1", scope); + publisherResource = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Publisher.class) + .build(), "1", scope); publisherResource.addRelation("books", book2Resource); scope.runQueuedPreCommitTriggers(); @@ -1104,12 +1094,12 @@ public void testRemoveFromCollectionTrigger() { HashMap> checkMappings = new HashMap<>(); checkMappings.put("Book operation check", Book.BookOperationCheck.class); checkMappings.put("Field path editor check", Editor.FieldPathFilterExpression.class); - store.populateEntityDictionary(new EntityDictionary(checkMappings)); + store.populateEntityDictionary(TestDictionary.getTestDictionary(checkMappings)); DataStoreTransaction tx = store.beginTransaction(); RequestScope scope = buildRequestScope(wrapped.getDictionary(), tx); - PersistentResource publisherResource = PersistentResource.createObject(null, Publisher.class, scope, Optional.of("1")); + PersistentResource publisherResource = PersistentResource.createObject(Publisher.class, scope, Optional.of("1")); PersistentResource book1Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("1")); PersistentResource book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book1Resource, book2Resource))); @@ -1127,7 +1117,11 @@ public void testRemoveFromCollectionTrigger() { scope = buildRequestScope(wrapped.getDictionary(), tx); book2Resource = PersistentResource.createObject(publisherResource, Book.class, scope, Optional.of("2")); - publisherResource = PersistentResource.loadRecord(Publisher.class, "1", scope); + + publisherResource = PersistentResource.loadRecord(EntityProjection.builder() + .type(Publisher.class) + .build(), "1", scope); + publisherResource.updateRelation("books", new HashSet<>(Arrays.asList(book2Resource))); scope.runQueuedPreCommitTriggers(); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java index 080eebef53..8434bdea03 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PermissionAnnotationTest.java @@ -21,7 +21,6 @@ import com.yahoo.elide.security.executors.ActivePermissionExecutor; import example.FunWithPermissions; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -35,7 +34,7 @@ public class PermissionAnnotationTest { private static PersistentResource funRecord; private static PersistentResource badRecord; - private static EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + private static EntityDictionary dictionary = TestDictionary.getTestDictionary(); public PermissionAnnotationTest() { } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java index 2692721f95..9558f6e917 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java @@ -42,7 +42,6 @@ import example.Parent; import example.Publisher; import example.Right; -import example.TestCheckMappings; import example.UpdateAndCreate; import example.packageshareable.ContainerWithPackageShare; import example.packageshareable.ShareableWithPackageShare; @@ -71,7 +70,8 @@ public class PersistenceResourceTestSetup extends PersistentResource { protected final ElideSettings elideSettings; protected static EntityDictionary initDictionary() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + EntityDictionary dictionary = TestDictionary.getTestDictionary(); + dictionary.bindEntity(UpdateAndCreate.class); dictionary.bindEntity(Author.class); dictionary.bindEntity(Book.class); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java index 2d90dbcaf8..79c8a3f8fc 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java @@ -38,6 +38,8 @@ import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; import com.yahoo.elide.jsonapi.models.ResourceIdentifier; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.User; @@ -67,12 +69,21 @@ import example.packageshareable.ContainerWithPackageShare; import example.packageshareable.ShareableWithPackageShare; import example.packageshareable.UnshareableWithEntityUnshare; +<<<<<<< HEAD import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; import org.junit.jupiter.api.Test; import org.mockito.Answers; +======= +import org.junit.jupiter.api.BeforeEach; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; +import org.mockito.ArgumentCaptor; +import lombok.AllArgsConstructor; +import lombok.EqualsAndHashCode; +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) import nocreate.NoCreateEntity; import java.util.ArrayList; @@ -92,19 +103,26 @@ import javax.ws.rs.core.MultivaluedHashMap; import javax.ws.rs.core.MultivaluedMap; - /** * Test PersistentResource. */ +@TestInstance(TestInstance.Lifecycle.PER_CLASS) public class PersistentResourceTest extends PersistenceResourceTestSetup { - private final RequestScope goodUserScope; - private final RequestScope badUserScope; + private final User goodUser = new User(1); + private final User badUser = new User(-1); + private DataStoreTransaction tx = mock(DataStoreTransaction.class); +<<<<<<< HEAD public PersistentResourceTest() { goodUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(1)); badUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(-1)); reset(goodUserScope.getTransaction()); +======= + @BeforeEach + public void beforeTest() { + reset(tx); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test @@ -112,11 +130,15 @@ public void testUpdateToOneRelationHookInAddRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation3", childResource); @@ -131,9 +153,6 @@ public void testUpdateToOneRelationHookInUpdateRelation() { Child child1 = newChild(1); Child child2 = newChild(2); fun.setRelation1(Sets.newHashSet(child1)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); @@ -150,9 +169,6 @@ public void testUpdateToOneRelationHookInRemoveRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child = newChild(1); fun.setRelation3(child); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); @@ -168,12 +184,15 @@ public void testUpdateToOneRelationHookInClearRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child1 = newChild(1); fun.setRelation3(child1); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child1); + when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child1); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); funResource.clearRelation("relation3"); @@ -186,11 +205,16 @@ public void testUpdateToManyRelationHookInAddRelationBidirection() { Parent parent = new Parent(); Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); parentResource.addRelation("children", childResource); @@ -206,9 +230,6 @@ public void testUpdateToManyRelationHookInRemoveRelationBidirection() { Child child = newChild(1); parent.setChildren(Sets.newHashSet(child)); child.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -230,10 +251,8 @@ public void testUpdateToManyRelationHookInClearRelationBidirection() { parent.setChildren(children); child1.setParents(Sets.newHashSet(parent)); child2.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(children); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -257,10 +276,8 @@ public void testUpdateToManyRelationHookInUpdateRelationBidirection() { parent.setChildren(children); child1.setParents(Sets.newHashSet(parent)); child2.setParents(Sets.newHashSet(parent)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(children); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); @@ -281,14 +298,16 @@ public void testUpdateToManyRelationHookInUpdateRelationBidirection() { @Test public void testSetAttributeHookInUpdateAttribute() { Parent parent = newParent(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + ArgumentCaptor attributeArgument = ArgumentCaptor.forClass(Attribute.class); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); parentResource.updateAttribute("firstName", "foobar"); - verify(tx, times(1)).setAttribute(parent, "firstName", "foobar", goodScope); + verify(tx, times(1)).setAttribute(eq(parent), attributeArgument.capture(), eq(goodScope)); + + assertEquals(attributeArgument.getValue().getName(), "firstName"); + assertEquals(attributeArgument.getValue().getArguments().iterator().next().getValue(), "foobar"); } @Test @@ -298,7 +317,9 @@ public void testGetRelationships() { fun.setRelation2(Sets.newHashSet()); fun.setRelation3(null); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Map relationships = funResource.getRelationships(); @@ -309,7 +330,9 @@ public void testGetRelationships() { assertTrue(relationships.containsKey("relation4"), "relation4 should be present"); assertTrue(relationships.containsKey("relation5"), "relation5 should be present"); - PersistentResource funResourceWithBadScope = new PersistentResource<>(fun, null, "3", badUserScope); + scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource funResourceWithBadScope = new PersistentResource<>(fun, null, "3", scope); relationships = funResourceWithBadScope.getRelationships(); assertEquals(0, relationships.size(), "All relationships should be filtered out"); @@ -319,8 +342,6 @@ public void testGetRelationships() { public void testNoCreate() { assertNotNull(dictionary); NoCreateEntity noCreate = new NoCreateEntity(); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); when(tx.createNewObject(NoCreateEntity.class)).thenReturn(noCreate); @@ -328,7 +349,7 @@ public void testNoCreate() { assertThrows( ForbiddenAccessException.class, () -> PersistentResource.createObject( - null, NoCreateEntity.class, goodScope, Optional.of("1"))); // should throw here + NoCreateEntity.class, goodScope, Optional.of("1"))); // should throw here } @Test @@ -339,7 +360,8 @@ public void testGetAttributes() { fun.setField2(null); fun.setField4("bar"); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Map attributes = funResource.getAttributes(); @@ -357,6 +379,7 @@ public void testGetAttributes() { assertEquals(attributes.get("field3"), "Foobar", "field3 should be set to original value."); assertEquals(attributes.get("field4"), "bar", "field4 should be set to original value."); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); PersistentResource funResourceBad = new PersistentResource<>(fun, null, "3", badUserScope); attributes = funResourceBad.getAttributes(); @@ -377,10 +400,11 @@ public void testFilter() { Child child4 = newChild(-4); { - PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", goodUserScope); - PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", goodUserScope); - PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", goodUserScope); - PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", scope); + PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", scope); + PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", scope); + PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", scope); Set resources = Sets.newHashSet(child1Resource, child2Resource, child3Resource, child4Resource); @@ -392,10 +416,11 @@ public void testFilter() { } { - PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", badUserScope); - PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", badUserScope); - PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", badUserScope); - PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", badUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource child1Resource = new PersistentResource<>(child1, null, "1", scope); + PersistentResource child2Resource = new PersistentResource<>(child2, null, "-2", scope); + PersistentResource child3Resource = new PersistentResource<>(child3, null, "3", scope); + PersistentResource child4Resource = new PersistentResource<>(child4, null, "-4", scope); Set resources = Sets.newHashSet(child1Resource, child2Resource, child3Resource, child4Resource); @@ -512,7 +537,8 @@ public void testDeleteBidirectionalRelation() { left.setOne2one(right); right.setOne2one(left); - PersistentResource leftResource = new PersistentResource<>(left, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource leftResource = new PersistentResource<>(left, null, "3", scope); leftResource.deleteInverseRelation("one2one", right); @@ -525,7 +551,8 @@ public void testDeleteBidirectionalRelation() { parent.setChildren(Sets.newHashSet(child)); parent.setSpouses(Sets.newHashSet()); - PersistentResource childResource = new PersistentResource<>(child, null, "4", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource childResource = new PersistentResource<>(child, null, "4", scope); childResource.deleteInverseRelation("parents", parent); @@ -538,7 +565,8 @@ public void testAddBidirectionalRelation() { Left left = new Left(); Right right = new Right(); - PersistentResource leftResource = new PersistentResource<>(left, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource leftResource = new PersistentResource<>(left, null, "3", scope); leftResource.addInverseRelation("one2one", right); @@ -550,7 +578,8 @@ public void testAddBidirectionalRelation() { parent.setChildren(Sets.newHashSet()); parent.setSpouses(Sets.newHashSet()); - PersistentResource childResource = new PersistentResource<>(child, null, "4", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource childResource = new PersistentResource<>(child, null, "4", scope); childResource.addInverseRelation("parents", parent); @@ -560,26 +589,28 @@ public void testAddBidirectionalRelation() { @Test public void testSuccessfulOneToOneRelationshipAdd() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); Left left = new Left(); Right right = new Right(); left.setId(2); right.setId(3); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "2", goodScope); Relationship ids = new Relationship(null, new Data<>(new ResourceIdentifier("right", "3").castToResource())); - when(tx.loadObject(eq(Right.class), eq(3L), any(), any())).thenReturn(right); + when(tx.loadObject(any(), eq(3L), any())).thenReturn(right); boolean updated = leftResource.updateRelation("one2one", ids.toPersistentResources(goodScope)); goodScope.saveOrCreateObjects(); verify(tx, times(1)).save(left, goodScope); verify(tx, times(1)).save(right, goodScope); - verify(tx, times(1)).getRelation(tx, left, "one2one", Optional.empty(), Optional.empty(), Optional.empty(), - goodScope); + verify(tx, times(1)).getRelation(tx, left, getRelationship(Right.class, "one2one"), goodScope); + assertTrue(updated, "The one-2-one relationship should be added."); assertEquals(3, left.getOne2one().getId(), "The correct object was set in the one-2-one relationship"); } @@ -601,8 +632,6 @@ public void testSuccessfulOneToOneRelationshipAdd() throws Exception { */ @Test public void testSuccessfulOneToOneRelationshipAddNull() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); Left left = new Left(); left.setId(2); @@ -636,11 +665,12 @@ public void testSuccessfulOneToOneRelationshipAddNull() throws Exception { * final = (notMine) UNION requested */ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - Parent parent = new Parent(); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) Child child1 = newChild(1); Child child2 = newChild(2); @@ -660,7 +690,7 @@ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { parent.setChildren(allChildren); parent.setSpouses(Sets.newHashSet()); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(allChildren); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(allChildren); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -670,12 +700,11 @@ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { idList.add(new ResourceIdentifier("child", "6").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - - when(tx.loadObject(eq(Child.class), eq(2L), any(), any())).thenReturn(child2); - when(tx.loadObject(eq(Child.class), eq(3L), any(), any())).thenReturn(child3); - when(tx.loadObject(eq(Child.class), eq(-4L), any(), any())).thenReturn(child4); - when(tx.loadObject(eq(Child.class), eq(-5L), any(), any())).thenReturn(child5); - when(tx.loadObject(eq(Child.class), eq(6L), any(), any())).thenReturn(child6); + when(tx.loadObject(any(), eq(2L), any())).thenReturn(child2); + when(tx.loadObject(any(), eq(3L), any())).thenReturn(child3); + when(tx.loadObject(any(), eq(-4L), any())).thenReturn(child4); + when(tx.loadObject(any(), eq(-5L), any())).thenReturn(child5); + when(tx.loadObject(any(), eq(6L), any())).thenReturn(child6); //Final set after operation = (3,4,5,6) Set expected = new HashSet<>(); @@ -743,7 +772,9 @@ public void testGetAttributeSuccess() { fun.setField2("blah"); fun.setField3(null); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); String result = (String) funResource.getAttribute("field2"); assertEquals("blah", result, "The correct attribute should be returned."); @@ -755,7 +786,9 @@ public void testGetAttributeSuccess() { public void testGetAttributeInvalidField() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); assertThrows(InvalidAttributeException.class, () -> funResource.getAttribute("invalid")); } @@ -765,7 +798,9 @@ public void testGetAttributeInvalidFieldPermissions() { FunWithPermissions fun = new FunWithPermissions(); fun.setField1("foo"); - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", scope); assertThrows(ForbiddenAccessException.class, () -> funResource.getAttribute("field1")); } @@ -774,7 +809,9 @@ public void testGetAttributeInvalidFieldPermissions() { public void testGetAttributeInvalidEntityPermissions() { NoReadEntity noread = new NoReadEntity(); - PersistentResource noreadResource = new PersistentResource<>(noread, null, "1", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource noreadResource = new PersistentResource<>(noread, null, "1", scope); assertThrows(ForbiddenAccessException.class, () -> noreadResource.getAttribute("field")); } @@ -788,9 +825,10 @@ public void testGetRelationSuccess() { Set children = Sets.newHashSet(child1, child2, child3); fun.setRelation2(children); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); - when(goodUserScope.getTransaction().getRelation(any(), eq(fun), eq("relation2"), any(), any(), any(), any())).thenReturn(children); + when(scope.getTransaction().getRelation(any(), eq(fun), any(), any())).thenReturn(children); Set results = getRelation(funResource, "relation2"); @@ -807,9 +845,10 @@ public void testGetRelationFilteredSuccess() { Set children = Sets.newHashSet(child1, child2, child3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); - when(goodUserScope.getTransaction().getRelation(any(), eq(fun), eq("relation2"), any(), any(), any(), any())).thenReturn(children); + when(scope.getTransaction().getRelation(any(), eq(fun), any(), any())).thenReturn(children); Set results = getRelation(funResource, "relation2"); @@ -824,13 +863,17 @@ public void testGetRelationWithPredicateSuccess() { Child child3 = newChild(3, "chris smith"); parent.setChildren(Sets.newHashSet(child1, child2, child3)); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); - User goodUser = new User(1); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.add("filter[child.name]", "paul john"); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope("/child", tx, goodUser, queryParams); +======= + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -850,11 +893,12 @@ public void testGetSingleRelationInMemory() { Set children = Sets.newHashSet(child1, child2, child3); parent.setChildren(children); - when(goodUserScope.getTransaction().getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + when(scope.getTransaction().getRelation(any(), eq(parent), any(), any())).thenReturn(children); - PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodUserScope); + PersistentResource parentResource = new PersistentResource<>(parent, null, "1", scope); - PersistentResource childResource = parentResource.getRelation("children", "2"); + PersistentResource childResource = parentResource.getRelation(getRelationship(Parent.class, "children"), "2"); assertEquals("2", childResource.getId()); assertEquals("john buzzard", ((Child) childResource.getObject()).getName()); @@ -864,7 +908,9 @@ public void testGetSingleRelationInMemory() { public void testGetRelationForbiddenByEntity() { NoReadEntity noread = new NoReadEntity(); - PersistentResource noreadResource = new PersistentResource<>(noread, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource noreadResource = new PersistentResource<>(noread, null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> getRelation(noreadResource, "child")); } @@ -872,7 +918,9 @@ public void testGetRelationForbiddenByEntity() { public void testGetRelationForbiddenByField() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", badUserScope); + RequestScope scope = new TestRequestScope(tx, badUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> getRelation(funResource, "relation1")); } @@ -881,6 +929,8 @@ public void testGetRelationForbiddenByField() { public void testGetRelationForbiddenByEntityAllowedByField() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); getRelation(fcResource, "public2"); @@ -890,6 +940,8 @@ public void testGetRelationForbiddenByEntityAllowedByField() { public void testGetAttributeForbiddenByEntityAllowedByField() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); fcResource.getAttribute("public1"); @@ -899,6 +951,8 @@ public void testGetAttributeForbiddenByEntityAllowedByField() { public void testGetRelationForbiddenByEntity2() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope badUserScope = new TestRequestScope(tx, badUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, null, "3", badUserScope); assertThrows(ForbiddenAccessException.class, () -> getRelation(fcResource, "private2")); @@ -908,8 +962,10 @@ public void testGetRelationForbiddenByEntity2() { public void testGetAttributeForbiddenByEntity2() { FirstClassFields firstClassFields = new FirstClassFields(); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + PersistentResource fcResource = new PersistentResource<>(firstClassFields, - null, "3", goodUserScope); + null, "3", scope); assertThrows(ForbiddenAccessException.class, () -> fcResource.getAttribute("private1")); } @@ -918,7 +974,9 @@ public void testGetAttributeForbiddenByEntity2() { public void testGetRelationInvalidRelation() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); assertThrows(InvalidAttributeException.class, () -> getRelation(funResource, "invalid")); } @@ -931,16 +989,12 @@ public void testGetRelationByIdSuccess() { Child child3 = newChild(3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); - PersistentResource result = funResource.getRelation("relation2", "1"); + PersistentResource result = funResource.getRelation(getRelationship(FunWithPermissions.class, "relation2"), "1"); assertEquals(1, ((Child) result.getObject()).getId(), "The correct relationship element should be returned"); } @@ -953,23 +1007,22 @@ public void testGetRelationByInvalidId() { Child child3 = newChild(3); fun.setRelation2(Sets.newHashSet(child1, child2, child3)); - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - when(tx.getRelation(eq(tx), any(), any(), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); + when(tx.getRelation(eq(tx), any(), any(), any())).thenReturn(Sets.newHashSet(child1)); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); - assertThrows(InvalidObjectIdentifierException.class, () -> funResource.getRelation("relation2", "-1000")); + assertThrows(InvalidObjectIdentifierException.class, + () -> funResource.getRelation(getRelationship(FunWithPermissions.class, "relation2"), "-1000")); } @Test public void testGetRelationsNoEntityAccess() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Set set = getRelation(funResource, "relation4"); assertEquals(0, set.size()); @@ -979,7 +1032,9 @@ public void testGetRelationsNoEntityAccess() { public void testGetRelationsNoEntityAccess2() { FunWithPermissions fun = new FunWithPermissions(); - PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource funResource = new PersistentResource<>(fun, null, "3", scope); Set set = getRelation(funResource, "relation5"); assertEquals(0, set.size()); @@ -989,9 +1044,13 @@ public void testGetRelationsNoEntityAccess2() { void testDeleteResourceSuccess() { Parent parent = newParent(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1009,9 +1068,13 @@ void testDeleteCascades() { invoice.setItems(Sets.newHashSet(item)); item.setInvoice(invoice); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource invoiceResource = new PersistentResource<>(invoice, null, "1", goodScope); @@ -1036,9 +1099,7 @@ void testDeleteResourceUpdateRelationshipSuccess() { assertFalse(parent.getChildren().isEmpty()); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(child), eq("parents"), any(), any(), any(), any())).thenReturn(parents); + when(tx.getRelation(any(), eq(child), any(), any())).thenReturn(parents); RequestScope goodScope = buildRequestScope(tx, goodUser); @@ -1058,10 +1119,14 @@ void testDeleteResourceForbidden() { NoDeleteEntity nodelete = new NoDeleteEntity(); nodelete.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource nodeleteResource = new PersistentResource<>(nodelete, null, "1", goodScope); @@ -1077,11 +1142,15 @@ void testAddRelationSuccess() { Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation1", childResource); @@ -1100,10 +1169,14 @@ void testAddRelationForbiddenByField() { Child child = newChild(1); +<<<<<<< HEAD User badUser = new User(-1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", badScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", badScope); assertThrows(ForbiddenAccessException.class, () -> funResource.addRelation("relation1", childResource)); @@ -1116,10 +1189,14 @@ void testAddRelationForbiddenByEntity() { Child child = newChild(2); noUpdate.setChildren(Sets.newHashSet()); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource noUpdateResource = new PersistentResource<>(noUpdate, null, "1", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "2", goodScope); assertThrows(ForbiddenAccessException.class, () -> noUpdateResource.addRelation("children", childResource)); @@ -1131,10 +1208,14 @@ public void testAddRelationInvalidRelation() { Child child = newChild(1); +<<<<<<< HEAD User goodUser = new User(1); DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.addRelation("invalid", childResource)); @@ -1148,9 +1229,13 @@ public void testRemoveToManyRelationSuccess() { Parent parent3 = newParent(3, child); child.setParents(Sets.newHashSet(parent1, parent2, parent3)); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(parent1, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1173,9 +1258,13 @@ public void testRemoveToOneRelationSuccess() { Child child = newChild(1); fun.setRelation3(child); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(child, null, "1", goodScope); @@ -1208,6 +1297,7 @@ public void testNoSaveNonModifications() { child.setReadNoAccess(secret); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); when(tx.getRelation(any(), eq(fun), eq("relation1"), any(), any(), any(), any())).thenReturn(children1); @@ -1221,6 +1311,49 @@ public void testNoSaveNonModifications() { PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource secretResource = new PersistentResource<>(secret, null, "1", goodScope); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); +======= + when(tx.getRelation(any(), eq(fun), eq(com.yahoo.elide.request.Relationship.builder() + .name("relation3") + .alias("relation3") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(child); + + when(tx.getRelation(any(), eq(fun), eq(com.yahoo.elide.request.Relationship.builder() + .name("relation1") + .alias("relation1") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(children1); + + when(tx.getRelation(any(), eq(parent), eq(com.yahoo.elide.request.Relationship.builder() + .name("children") + .alias("children") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(children2); + + when(tx.getRelation(any(), eq(child), eq(com.yahoo.elide.request.Relationship.builder() + .name("readNoAccess") + .alias("readNoAccess") + .projection(EntityProjection.builder() + .type(Child.class) + .build()) + .build()), any())).thenReturn(secret); + + RequestScope funScope = new TestRequestScope(tx, goodUser, dictionary); + RequestScope childScope = new TestRequestScope(tx, goodUser, dictionary); + RequestScope parentScope = new TestRequestScope(tx, goodUser, dictionary); + + + PersistentResource funResource = new PersistentResource<>(fun, null, "1", funScope); + PersistentResource childResource = new PersistentResource<>(child, null, "1", childScope); + PersistentResource secretResource = new PersistentResource<>(secret, null, "1", childScope); + PersistentResource parentResource = new PersistentResource<>(parent, null, "1", parentScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) // Add an existing to-one relationship funResource.addRelation("relation3", childResource); @@ -1252,11 +1385,13 @@ public void testNoSaveNonModifications() { // Clear empty to-one relation secretResource.clearRelation("readNoAccess"); - goodScope.saveOrCreateObjects(); - verify(tx, never()).save(fun, goodScope); - verify(tx, never()).save(child, goodScope); - verify(tx, never()).save(parent, goodScope); - verify(tx, never()).save(secret, goodScope); + parentScope.saveOrCreateObjects(); + childScope.saveOrCreateObjects(); + funScope.saveOrCreateObjects(); + verify(tx, never()).save(fun, funScope); + verify(tx, never()).save(child, childScope); + verify(tx, never()).save(parent, parentScope); + verify(tx, never()).save(secret, childScope); } @Test() @@ -1266,9 +1401,13 @@ public void testRemoveNonexistingToOneRelation() { Child unownedChild = newChild(2); fun.setRelation3(ownedChild); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedChild, null, "1", goodScope); @@ -1291,9 +1430,13 @@ public void testRemoveNonexistingToManyRelation() { Parent unownedParent = newParent(4, null); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedParent, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1318,11 +1461,21 @@ public void testClearToManyRelationSuccess() { Set parents = Sets.newHashSet(parent1, parent2, parent3); child.setParents(parents); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(child), eq("parents"), any(), any(), any(), any())).thenReturn(parents); + when(tx.getRelation(any(), eq(child), any(), any())).thenReturn(parents); +<<<<<<< HEAD User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Child.class) + .relationship("parents", + EntityProjection.builder() + .type(Parent.class) + .build()) + .build()); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); @@ -1346,12 +1499,24 @@ public void testClearToOneRelationSuccess() { Child child = newChild(1); fun.setRelation3(child); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child); +<<<<<<< HEAD when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(FunWithPermissions.class) + .relationship("relation3", + EntityProjection.builder() + .type(Child.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); funResource.clearRelation("relation3"); @@ -1364,11 +1529,17 @@ public void testClearToOneRelationSuccess() { @Test() public void testClearRelationFilteredByReadAccess() { - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); Parent parent = new Parent(); RequestScope goodScope = buildRequestScope(tx, goodUser); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Parent.class) + .relationship("children", + EntityProjection.builder() + .type(Child.class) + .build()) + .build()); + Child child1 = newChild(1); Child child2 = newChild(2); Child child3 = newChild(3); @@ -1377,7 +1548,6 @@ public void testClearRelationFilteredByReadAccess() { Child child5 = newChild(-5); child5.setId(-5); //Not accessible to goodUser - //All = (1,2,3,4,5) //Mine = (1,2,3) Set allChildren = new HashSet<>(); @@ -1389,7 +1559,7 @@ public void testClearRelationFilteredByReadAccess() { parent.setChildren(allChildren); parent.setSpouses(Sets.newHashSet()); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(allChildren); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(allChildren); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1427,9 +1597,22 @@ public void testClearRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noUpdateOne2One", + EntityProjection.builder() + .type(Right.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1448,9 +1631,13 @@ public void testNoChangeRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1474,12 +1661,24 @@ public void testClearRelationInvalidToManyUpdatePermission() { right1.setNoUpdate(Sets.newHashSet(left)); right2.setNoUpdate(Sets.newHashSet(left)); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noInverseUpdate); +<<<<<<< HEAD when(tx.getRelation(any(), eq(left), eq("noInverseUpdate"), any(), any(), any(), any())).thenReturn(noInverseUpdate); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noInverseUpdate", + EntityProjection.builder() + .type(Right.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1497,11 +1696,24 @@ public void testClearRelationInvalidToOneDeletePermission() { noDelete.setId(1); left.setNoDeleteOne2One(noDelete); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.getRelation(any(), eq(left), eq("noDeleteOne2One"), any(), any(), any(), any())).thenReturn(noDelete); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noDelete); + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(EntityProjection.builder() + .type(Left.class) + .relationship("noDeleteOne2One", + EntityProjection.builder() + .type(NoDeleteEntity.class) + .build()) + .build()); + +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertTrue(leftResource.clearRelation("noDeleteOne2One")); assertNull(leftResource.getObject().getNoDeleteOne2One()); @@ -1514,9 +1726,13 @@ public void testClearRelationInvalidRelation() { Child child = newChild(1); fun.setRelation3(child); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.clearRelation("invalid")); } @@ -1525,10 +1741,14 @@ public void testClearRelationInvalidRelation() { public void testUpdateAttributeSuccess() { Parent parent = newParent(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); parentResource.updateAttribute("firstName", "foobar"); @@ -1542,10 +1762,14 @@ public void testUpdateAttributeSuccess() { public void testUpdateAttributeInvalidAttribute() { Parent parent = newParent(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> parentResource.updateAttribute("invalid", "foobar")); } @@ -1555,11 +1779,15 @@ public void testUpdateAttributeInvalidUpdatePermission() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); @@ -1575,10 +1803,14 @@ public void testUpdateAttributeInvalidUpdatePermissionNoChange() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); +<<<<<<< HEAD DataStoreTransaction tx = mock(DataStoreTransaction.class); User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); +======= + RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); @@ -1597,15 +1829,26 @@ public void testLoadRecords() { Child child4 = newChild(4); Child child5 = newChild(5); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) - when(tx.loadObjects(eq(Child.class), any(), any(), any(), any(RequestScope.class))) + .build(); + + when(tx.loadObjects(eq(collection), any(RequestScope.class))) .thenReturn(Lists.newArrayList(child1, child2, child3, child4, child5)); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); Set loaded = PersistentResource.loadRecords(Child.class, new ArrayList<>(), Optional.empty(), Optional.empty(), Optional.empty(), goodScope); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); + + Set loaded = PersistentResource.loadRecords(EntityProjection.builder() + .type(Child.class) + .build(), new ArrayList<>(), goodScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) Set expected = Sets.newHashSet(child1, child4, child5); @@ -1623,55 +1866,84 @@ public void testLoadRecords() { public void testLoadRecordSuccess() { Child child1 = newChild(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) - when(tx.loadObject(eq(Child.class), eq(1L), any(), any())).thenReturn(child1); + .build(); + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(child1); + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource loaded = PersistentResource.loadRecord(Child.class, "1", goodScope); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); + PersistentResource loaded = PersistentResource.loadRecord(EntityProjection.builder() + .type(Child.class) + .build(), "1", goodScope); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertEquals(child1, loaded.getObject(), "The load function should return the requested child object"); } @Test public void testLoadRecordInvalidId() { - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(Child.class) + + .build(); - when(tx.loadObject(eq(Child.class), eq("1"), any(), any())).thenReturn(null); + when(tx.loadObject(eq(collection), eq("1"), any())).thenReturn(null); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertThrows( InvalidObjectIdentifierException.class, - () -> PersistentResource.loadRecord(Child.class, "1", goodScope)); + () -> PersistentResource.loadRecord(EntityProjection.builder() + + .type(Child.class) + .build(), "1", goodScope)); } @Test public void testLoadRecordForbidden() { NoReadEntity noRead = new NoReadEntity(); noRead.setId(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); + EntityProjection collection = EntityProjection.builder() + .type(NoReadEntity.class) - when(tx.loadObject(eq(NoReadEntity.class), eq(1L), any(), any())).thenReturn(noRead); + .build(); + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noRead); + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); assertThrows( ForbiddenAccessException.class, () -> PersistentResource.loadRecord(NoReadEntity.class, "1", goodScope)); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + goodScope.setEntityProjection(collection); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test() public void testCreateObjectSuccess() { Parent parent = newParent(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - when(tx.createNewObject(Parent.class)).thenReturn(parent); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource created = PersistentResource.createObject(null, Parent.class, goodScope, Optional.of("uuid")); +======= + RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + PersistentResource created = PersistentResource.createObject(Parent.class, goodScope, Optional.of("uuid")); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) parent.setChildren(new HashSet<>()); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -1687,11 +1959,15 @@ public void testCreateMappedIdObjectSuccess() { job.setTitle("day job"); job.setParent(newParent(1)); - final DataStoreTransaction tx = mock(DataStoreTransaction.class); when(tx.createNewObject(Job.class)).thenReturn(job); +<<<<<<< HEAD final RequestScope goodScope = buildRequestScope(tx, new User(1)); PersistentResource created = PersistentResource.createObject(null, Job.class, goodScope, Optional.empty()); +======= + final RequestScope goodScope = new RequestScope(null, null, tx, new User(1), null, elideSettings); + PersistentResource created = PersistentResource.createObject(Job.class, goodScope, Optional.empty()); +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) created.getRequestScope().getPermissionExecutor().executeCommitChecks(); assertEquals("day job", created.getObject().getTitle(), @@ -1699,7 +1975,7 @@ public void testCreateMappedIdObjectSuccess() { ); assertNull(created.getObject().getJobId(), "The create function should not override the ID"); - created = PersistentResource.createObject(null, Job.class, goodScope, Optional.of("1234")); + created = PersistentResource.createObject(Job.class, goodScope, Optional.of("1234")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); assertEquals("day job", created.getObject().getTitle(), @@ -1712,8 +1988,6 @@ public void testCreateMappedIdObjectSuccess() { public void testCreateObjectForbidden() { NoCreateEntity noCreate = new NoCreateEntity(); noCreate.setId(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); when(tx.createNewObject(NoCreateEntity.class)).thenReturn(noCreate); @@ -1722,7 +1996,7 @@ public void testCreateObjectForbidden() { assertThrows( ForbiddenAccessException.class, () -> { - PersistentResource created = PersistentResource.createObject(null, NoCreateEntity.class, goodScope, Optional.of("1")); + PersistentResource created = PersistentResource.createObject(NoCreateEntity.class, goodScope, Optional.of("1")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } ); @@ -1740,9 +2014,7 @@ public void testDeletePermissionCheckedOnInverseRelationship() { right.setAllowDeleteAtFieldLevel(Sets.newHashSet(left)); //Bad User triggers the delete permission failure - User badUser = new User(-1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(left), eq("fieldLevelDelete"), any(), any(), any(), any())).thenReturn(rights); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(rights); RequestScope badScope = buildRequestScope(tx, badUser); PersistentResource leftResource = new PersistentResource<>(left, null, badScope.getUUIDFor(left), badScope); @@ -1765,9 +2037,7 @@ public void testUpdatePermissionCheckedOnInverseRelationship() { List empty = new ArrayList<>(); Relationship ids = new Relationship(null, new Data<>(empty)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(left), eq("noInverseUpdate"), any(), any(), any(), any())).thenReturn(rights); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(rights); RequestScope goodScope = buildRequestScope(tx, goodUser); PersistentResource leftResource = new PersistentResource<>(left, null, goodScope.getUUIDFor(left), goodScope); @@ -1785,7 +2055,6 @@ public void testFieldLevelAudit() throws Exception { Parent parent = newParent(7); - User goodUser = new User(1); TestAuditLogger logger = new TestAuditLogger(); RequestScope requestScope = getUserScope(goodUser, logger); PersistentResource parentResource = new PersistentResource<>(parent, null, requestScope.getUUIDFor(parent), requestScope); @@ -1806,7 +2075,6 @@ public void testClassLevelAudit() throws Exception { Child child = newChild(5); Parent parent = newParent(7); - User goodUser = new User(1); TestAuditLogger logger = new TestAuditLogger(); RequestScope requestScope = getUserScope(goodUser, logger); PersistentResource parentResource = new PersistentResource<>( @@ -1829,9 +2097,7 @@ public void testOwningRelationshipInverseUpdates() { Parent parent = newParent(1); Child child = newChild(2); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(parent.getChildren()); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(parent.getChildren()); RequestScope goodScope = buildRequestScope(tx, goodUser); @@ -1852,7 +2118,7 @@ public void testOwningRelationshipInverseUpdates() { assertTrue(child.getParents().contains(parent), "The non-owning relationship should also be updated"); reset(tx); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(parent.getChildren()); + when(tx.getRelation(any(), eq(parent), any(), any())).thenReturn(parent.getChildren()); parentResource.clearRelation("children"); @@ -1866,13 +2132,16 @@ public void testOwningRelationshipInverseUpdates() { @Test public void testIsIdGenerated() { + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); - PersistentResource generated = new PersistentResource<>(new Child(), null, "1", goodUserScope); + PersistentResource generated = new PersistentResource<>(new Child(), null, "1", scope); assertTrue(generated.isIdGenerated(), "isIdGenerated returns true when ID field has the GeneratedValue annotation"); - PersistentResource notGenerated = new PersistentResource<>(new NoCreateEntity(), null, "1", goodUserScope); + scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource notGenerated = new PersistentResource<>(new NoCreateEntity(), null, "1", scope); assertFalse(notGenerated.isIdGenerated(), "isIdGenerated returns false when ID field does not have the GeneratedValue annotation"); @@ -1890,11 +2159,20 @@ public void testSharePermissionErrorOnUpdateSingularRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare); + EntityProjection collection = EntityProjection.builder() + .type(NoShareEntity.class) + + .build(); + + when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noShare); + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + + +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -1913,11 +2191,14 @@ public void testSharePermissionErrorOnUpdateRelationshipPackageLevel() { unShareableList.add(new ResourceIdentifier("unshareableWithEntityUnshare", "1").castToResource()); Relationship unShareales = new Relationship(null, new Data<>(unShareableList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(UnshareableWithEntityUnshare.class), eq(1L), any(), any())).thenReturn(unshareableWithEntityUnshare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(unshareableWithEntityUnshare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); assertThrows( @@ -1937,11 +2218,14 @@ public void testSharePermissionSuccessOnUpdateManyRelationshipPackageLevel() { shareableList.add(new ResourceIdentifier("shareableWithPackageShare", "1").castToResource()); Relationship shareables = new Relationship(null, new Data<>(shareableList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(ShareableWithPackageShare.class), eq(1L), any(), any())).thenReturn(shareableWithPackageShare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(shareableWithPackageShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); containerResource.updateRelation("shareableWithPackageShares", shareables.toPersistentResources(goodScope)); @@ -1965,12 +2249,15 @@ public void testSharePermissionErrorOnUpdateManyRelationship() { idList.add(new ResourceIdentifier("noshare", "2").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare1); - when(tx.loadObject(eq(NoShareEntity.class), eq(2L), any(), any())).thenReturn(noShare2); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); + when(tx.loadObject(any(), eq(2L), any())).thenReturn(noShare2); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -1996,12 +2283,15 @@ public void testSharePermissionSuccessOnUpdateManyRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare1); - when(tx.getRelation(any(), eq(userModel), eq("noShares"), any(), any(), any(), any())).thenReturn(noshares); + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); + + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noshares); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShares", ids.toPersistentResources(goodScope)); @@ -2025,13 +2315,16 @@ public void testSharePermissionSuccessOnUpdateSingularRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - when(tx.getRelation(any(), eq(userModel), eq("noShare"), any(), any(), any(), any())).thenReturn(noShare); - when(tx.loadObject(eq(NoShareEntity.class), eq(1L), any(), any())).thenReturn(noShare); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2053,11 +2346,14 @@ public void testSharePermissionSuccessOnClearSingularRelationship() { List empty = new ArrayList<>(); Relationship ids = new Relationship(null, new Data<>(empty)); - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(userModel), eq("noShare"), any(), any(), any(), any())).thenReturn(noShare); + when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); + + RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); +<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); +======= +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2077,7 +2373,6 @@ public void testCollectionChangeSpecType() { return condFn.apply((Collection) spec.getOriginal(), (Collection) spec.getModified()); }; - DataStoreTransaction tx = mock(DataStoreTransaction.class); // Ensure that change specs coming from collections work properly ChangeSpecModel csModel = new ChangeSpecModel((spec) -> collectionCheck @@ -2086,7 +2381,7 @@ public void testCollectionChangeSpecType() { PersistentResource model = bootstrapPersistentResource(csModel, tx); - when(tx.getRelation(any(), eq(model.obj), eq("otherKids"), any(), any(), any(), any())).thenReturn(new HashSet<>()); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(new HashSet<>()); /* Attributes */ // Set new data from null @@ -2124,7 +2419,8 @@ public void testCollectionChangeSpecType() { model.getObject().checkFunction = (spec) -> collectionCheck.apply("otherKids").apply(spec, (original, modified) -> original.size() == 3 && original.contains(new ChangeSpecChild(1)) && original.contains(new ChangeSpecChild(2)) && original.contains(new ChangeSpecChild(3)) && modified.size() == 2 && modified.contains(new ChangeSpecChild(1)) && modified.contains(new ChangeSpecChild(3))); model.removeRelation("otherKids", bootstrapPersistentResource(child2)); - when(tx.getRelation(any(), eq(model.obj), eq("otherKids"), any(), any(), any(), any())).thenReturn(Sets.newHashSet(child1, child3)); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(Sets.newHashSet(child1, child3)); + // Clear the rest model.getObject().checkFunction = (spec) -> collectionCheck.apply("otherKids").apply(spec, (original, modified) -> original.size() <= 2 && modified.size() < original.size()); @@ -2168,24 +2464,23 @@ public void testRelationChangeSpecType() { } return checkFn.apply((ChangeSpecChild) spec.getOriginal(), (ChangeSpecChild) spec.getModified()); }; - DataStoreTransaction tx = mock(DataStoreTransaction.class); PersistentResource model = bootstrapPersistentResource(new ChangeSpecModel((spec) -> relCheck.apply(spec, (original, modified) -> (original == null) && new ChangeSpecChild(1).equals(modified))), tx); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(null); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(null); ChangeSpecChild child1 = new ChangeSpecChild(1); assertTrue(model.updateRelation("child", Sets.newHashSet(bootstrapPersistentResource(child1, tx)))); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(child1); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(child1); model.getObject().checkFunction = (spec) -> relCheck.apply(spec, (original, modified) -> new ChangeSpecChild(1).equals(original) && new ChangeSpecChild(2).equals(modified)); ChangeSpecChild child2 = new ChangeSpecChild(2); assertTrue(model.updateRelation("child", Sets.newHashSet(bootstrapPersistentResource(child2, tx)))); - when(tx.getRelation(any(), eq(model.obj), eq("child"), any(), any(), any(), any())).thenReturn(child2); + when(tx.getRelation(any(), eq(model.obj), any(), any())).thenReturn(child2); model.getObject().checkFunction = (spec) -> relCheck.apply(spec, (original, modified) -> new ChangeSpecChild(2).equals(original) && modified == null); assertTrue(model.updateRelation("child", null)); @@ -2250,10 +2545,12 @@ public void testEqualsAndHashcode() { Child childWithId = newChild(1); Child childWithoutId = newChild(0); - PersistentResource resourceWithId = new PersistentResource<>(childWithId, null, goodUserScope.getUUIDFor(childWithId), goodUserScope); - PersistentResource resourceWithDifferentId = new PersistentResource<>(childWithoutId, null, goodUserScope.getUUIDFor(childWithoutId), goodUserScope); - PersistentResource resourceWithUUID = new PersistentResource<>(childWithoutId, null, "abc", goodUserScope); - PersistentResource resourceWithIdAndUUID = new PersistentResource<>(childWithId, null, "abc", goodUserScope); + RequestScope scope = new TestRequestScope(tx, goodUser, dictionary); + + PersistentResource resourceWithId = new PersistentResource<>(childWithId, null, scope.getUUIDFor(childWithId), scope); + PersistentResource resourceWithDifferentId = new PersistentResource<>(childWithoutId, null, scope.getUUIDFor(childWithoutId), scope); + PersistentResource resourceWithUUID = new PersistentResource<>(childWithoutId, null, "abc", scope); + PersistentResource resourceWithIdAndUUID = new PersistentResource<>(childWithId, null, "abc", scope); assertNotEquals(resourceWithUUID, resourceWithId); assertNotEquals(resourceWithId, resourceWithUUID); @@ -2270,4 +2567,127 @@ public void testEqualsAndHashcode() { assertNotEquals(resourceWithDifferentId, resourceWithId); assertNotEquals(resourceWithId, resourceWithDifferentId); } +<<<<<<< HEAD +======= + + private PersistentResource bootstrapPersistentResource(T obj) { + return bootstrapPersistentResource(obj, mock(DataStoreTransaction.class)); + } + + private PersistentResource bootstrapPersistentResource(T obj, DataStoreTransaction tx) { + RequestScope requestScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); + return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); + } + + private RequestScope getUserScope(User user, AuditLogger auditLogger) { + return new RequestScope(null, new JsonApiDocument(), null, user, null, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .withAuditLogger(auditLogger) + .build()); + } + + // Testing constructor, setId and non-null empty sets + private static Parent newParent(int id) { + Parent parent = new Parent(); + parent.setId(id); + parent.setChildren(new HashSet<>()); + parent.setSpouses(new HashSet<>()); + return parent; + } + + private Parent newParent(int id, Child child) { + Parent parent = new Parent(); + parent.setId(id); + parent.setChildren(Sets.newHashSet(child)); + parent.setSpouses(new HashSet<>()); + return parent; + } + + /* ChangeSpec-specific test elements */ + @Entity + @Include + @CreatePermission(expression = "allow all") + @ReadPermission(expression = "allow all") + @UpdatePermission(expression = "deny all") + @DeletePermission(expression = "allow all") + public static final class ChangeSpecModel { + @Id + public long id; + + @ReadPermission(expression = "deny all") + @UpdatePermission(expression = "deny all") + public Function checkFunction; + + @UpdatePermission(expression = "changeSpecNonCollection") + public String testAttr; + + @UpdatePermission(expression = "changeSpecCollection") + public List testColl; + + @OneToOne + @UpdatePermission(expression = "changeSpecNonCollection") + public ChangeSpecChild child; + + @ManyToMany + @UpdatePermission(expression = "changeSpecCollection") + public List otherKids; + + public ChangeSpecModel(final Function checkFunction) { + this.checkFunction = checkFunction; + } + } + + @Entity + @Include + @EqualsAndHashCode + @AllArgsConstructor + @CreatePermission(expression = "allow all") + @ReadPermission(expression = "allow all") + @UpdatePermission(expression = "allow all") + @DeletePermission(expression = "allow all") + @SharePermission + public static final class ChangeSpecChild { + @Id + public long id; + } + + public static final class ChangeSpecCollection extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { + ChangeSpec spec = changeSpec.get(); + if (!(spec.getModified() instanceof Collection)) { + return false; + } + return ((ChangeSpecModel) object).checkFunction.apply(spec); + } + throw new IllegalStateException("Something is terribly wrong :("); + } + } + + public static final class ChangeSpecNonCollection extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { + return ((ChangeSpecModel) object).checkFunction.apply(changeSpec.get()); + } + throw new IllegalStateException("Something is terribly wrong :("); + } + } + + public Set getRelation(PersistentResource resource, String relation) { + return resource.getRelationCheckedFiltered(getRelationship(resource.getResourceClass(), relation)); + } + + private com.yahoo.elide.request.Relationship getRelationship(Class type, String name) { + return com.yahoo.elide.request.Relationship.builder() + .name(name) + .alias(name) + .projection(EntityProjection.builder() + .type(type) + .build()) + .build(); + } +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java b/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java new file mode 100644 index 0000000000..245d3c7ed1 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestDictionary.java @@ -0,0 +1,70 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.Injector; +import com.yahoo.elide.security.checks.Check; + +import com.google.inject.Binder; +import com.google.inject.Guice; +import com.google.inject.Module; +import com.google.inject.TypeLiteral; +import com.google.inject.name.Names; +import example.TestCheckMappings; + +import java.util.Map; +import javax.inject.Inject; +import javax.inject.Named; +import javax.inject.Singleton; + +/** + * Test Entity Dictionary. + */ +@Singleton +public class TestDictionary extends EntityDictionary { + + @Inject + public TestDictionary(Injector injector, + @Named("checkMappings") Map> checks) { + super(checks, injector); + } + + @Override + public Class lookupBoundClass(Class objClass) { + // Special handling for mocked Book class which has Entity annotation + if (objClass.getName().contains("$MockitoMock$")) { + objClass = objClass.getSuperclass(); + } + return super.lookupBoundClass(objClass); + } + + /** + * Returns a test dictionary injected with Guice. + * @return a test dictionary. + */ + public static EntityDictionary getTestDictionary() { + return getTestDictionary(TestCheckMappings.MAPPINGS); + } + + /** + * Returns a test dictionary injected with Guice. + * @param checks The security checks to setup the dictionary with. + * @return a test dictionary. + */ + public static EntityDictionary getTestDictionary(Map> checks) { + return Guice.createInjector(new Module() { + @Override + public void configure(Binder binder) { + binder.bind(Injector.class).to(TestInjector.class); + binder.bind(EntityDictionary.class).to(TestDictionary.class); + binder.bind(new TypeLiteral>>() { }) + .annotatedWith(Names.named("checkMappings")) + .toInstance(checks); + } + }).getInstance(EntityDictionary.class); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java b/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java new file mode 100644 index 0000000000..1d88588586 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestInjector.java @@ -0,0 +1,33 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.Injector; + +import javax.inject.Inject; + +/** + * Test Dependency Injector. + */ +public class TestInjector implements Injector { + private final com.google.inject.Injector injector; + + @Inject + public TestInjector(com.google.inject.Injector injector) { + this.injector = injector; + } + + @Override + public void inject(Object entity) { + injector.injectMembers(entity); + } + + @Override + public T instantiate(Class cls) { + return injector.getInstance(cls); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java b/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java new file mode 100644 index 0000000000..4775a44d81 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/core/TestRequestScope.java @@ -0,0 +1,53 @@ +/* + * Copyright 2016, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.core; + +import com.yahoo.elide.ElideSettingsBuilder; +import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.security.User; + +import java.util.Optional; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Utility subclass that helps construct RequestScope objects for testing. + */ +public class TestRequestScope extends RequestScope { + + private MultivaluedMap queryParamOverrides = null; + + public TestRequestScope(DataStoreTransaction transaction, + User user, + EntityDictionary dictionary) { + super(null, new JsonApiDocument(), transaction, user, null, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .build()); + } + + public TestRequestScope(EntityDictionary dictionary, + String path, + MultivaluedMap queryParams) { + super(path, new JsonApiDocument(), null, null, queryParams, + new ElideSettingsBuilder(null) + .withEntityDictionary(dictionary) + .build()); + } + + public void setQueryParams(MultivaluedMap queryParams) { + this.queryParamOverrides = queryParams; + } + + @Override + public Optional> getQueryParams() { + if (queryParamOverrides != null) { + return Optional.of(queryParamOverrides); + } else { + return super.getQueryParams(); + } + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java b/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java index acc34dcdcf..04ef8e8579 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/UpdateOnCreateTest.java @@ -9,83 +9,38 @@ import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.reset; import static org.mockito.Mockito.when; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.User; -import com.yahoo.elide.utils.coerce.CoerceUtil; + import example.Author; import example.Book; -import example.Editor; -import example.Publisher; import example.UpdateAndCreate; +import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; -import java.io.Serializable; import java.util.Optional; public class UpdateOnCreateTest extends PersistenceResourceTestSetup { - private RequestScope userOneScope; - private RequestScope userTwoScope; - private RequestScope userThreeScope; - private RequestScope userFourScope; - - public UpdateOnCreateTest() { - super(); - init(); - } - - public void init() { - dictionary.bindEntity(Author.class); - dictionary.bindEntity(Book.class); - dictionary.bindEntity(Publisher.class); - dictionary.bindEntity(Editor.class); - dictionary.bindEntity(UpdateAndCreate.class); - - UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); - updateAndCreateNewObject.setId(1L); - UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); - updateAndCreateExistingObject.setId(2L); - Book book = new Book(); - Author author = new Author(); - Publisher publisher = new Publisher(); - Editor editor = new Editor(); + private User userOne = new User(1); + private User userTwo = new User(2); + private User userThree = new User(3); + private User userFour = new User(4); - publisher.setEditor(editor); + private DataStoreTransaction tx = mock(DataStoreTransaction.class); - DataStoreTransaction tx = mock(DataStoreTransaction.class); - - User userOne = new User(1); - userOneScope = new RequestScope(null, null, tx, userOne, null, elideSettings); - User userTwo = new User(2); - userTwoScope = new RequestScope(null, null, tx, userTwo, null, elideSettings); - User userThree = new User(3); - userThreeScope = new RequestScope(null, null, tx, userThree, null, elideSettings); - User userFour = new User(4); - userFourScope = new RequestScope(null, null, tx, userFour, null, elideSettings); + @BeforeEach + public void beforeMethod() { + reset(tx); + } - when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); - when(tx.loadObject(eq(UpdateAndCreate.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(updateAndCreateExistingObject); - when(tx.loadObject(eq(Book.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(book); - when(tx.loadObject(eq(Author.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(author); - when(tx.loadObject(eq(Publisher.class), - eq((Serializable) CoerceUtil.coerce(1, Long.class)), - eq(Optional.empty()), - any(RequestScope.class) - )).thenReturn(publisher); + public UpdateOnCreateTest() { + super(); + initDictionary(); } //----------------------------------------- ** Entity Creation ** ------------------------------------------------- @@ -93,30 +48,56 @@ public void init() { //Create allowed based on class level expression @Test public void createPermissionCheckClassAnnotationForCreatingAnEntitySuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("1")); + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("1")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } //Create allowed based on field level expression @Test public void createPermissionCheckFieldAnnotationForCreatingAnEntitySuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("2")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("2")); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } //Create denied based on field level expression @Test public void createPermissionCheckFieldAnnotationForCreatingAnEntityFailureCase() { + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); assertThrows( ForbiddenAccessException.class, - () -> PersistentResource.createObject(null, UpdateAndCreate.class, userFourScope, Optional.of("3"))); + () -> PersistentResource.createObject(UpdateAndCreate.class, userFourScope, Optional.of("3"))); } //----------------------------------------- ** Update Attribute ** ------------------------------------------------ //Expression for field inherited from class level expression @Test public void updatePermissionInheritedForAttributeSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userTwoScope); loaded.updateAttribute("name", ""); @@ -125,7 +106,19 @@ public void updatePermissionInheritedForAttributeSuccessCase() { @Test public void updatePermissionInheritedForAttributeFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userOneScope); assertThrows(ForbiddenAccessException.class, () -> loaded.updateAttribute("name", "")); @@ -134,7 +127,19 @@ public void updatePermissionInheritedForAttributeFailureCase() { //Class level expression overwritten by field level expression @Test public void updatePermissionOverwrittenForAttributeSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userFourScope); loaded.updateAttribute("alias", ""); @@ -143,7 +148,19 @@ public void updatePermissionOverwrittenForAttributeSuccessCase() { @Test public void updatePermissionOverwrittenForAttributeFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userThreeScope); assertThrows(ForbiddenAccessException.class, () -> loaded.updateAttribute("alias", "")); @@ -154,11 +171,31 @@ public void updatePermissionOverwrittenForAttributeFailureCase() { //Expression for relation inherited from class level expression @Test public void updatePermissionInheritedForRelationSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userTwoScope); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userTwoScope); loaded.addRelation("books", loadedBook); loaded.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -166,11 +203,31 @@ public void updatePermissionInheritedForRelationSuccessCase() { @Test public void updatePermissionInheritedForRelationFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userOneScope); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userOneScope); assertThrows(ForbiddenAccessException.class, () -> loaded.addRelation("books", loadedBook)); } @@ -178,11 +235,33 @@ public void updatePermissionInheritedForRelationFailureCase() { //Class level expression overwritten by field level expression @Test public void updatePermissionOverwrittenForRelationSuccessCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userThreeScope = new TestRequestScope(tx, new User(3), dictionary); + + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + updateAndCreateExistingObject.setId(1L); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + .build(), "1", userThreeScope); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userThreeScope); loaded.addRelation("author", loadedAuthor); loaded.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -190,11 +269,32 @@ public void updatePermissionOverwrittenForRelationSuccessCase() { @Test public void updatePermissionOverwrittenForRelationFailureCase() { - PersistentResource loaded = PersistentResource.loadRecord(UpdateAndCreate.class, + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateExistingObject = new UpdateAndCreate(); + + when(tx.loadObject(any(), + eq(1L), + any(RequestScope.class) + )).thenReturn(updateAndCreateExistingObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource loaded = PersistentResource.loadRecord( + EntityProjection.builder() + .type(UpdateAndCreate.class) + + .build(), "1", userTwoScope); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userTwoScope); assertThrows(ForbiddenAccessException.class, () -> loaded.addRelation("author", loadedAuthor)); } @@ -203,54 +303,99 @@ public void updatePermissionOverwrittenForRelationFailureCase() { //Expression for field inherited from class level expression @Test public void createPermissionInheritedForAttributeSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("4")); + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("4")); created.updateAttribute("name", ""); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionInheritedForAttributeFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("5")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("5")); assertThrows(ForbiddenAccessException.class, () -> created.updateAttribute("name", "")); } //Class level expression overwritten by field level expression @Test public void createPermissionOverwrittenForAttributeSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("6")); + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("6")); created.updateAttribute("alias", ""); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionOverwrittenForAttributeFailureCase() { + RequestScope userFourScope = new TestRequestScope(tx, userFour, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); assertThrows( ForbiddenAccessException.class, () -> { PersistentResource created = - PersistentResource.createObject(null, UpdateAndCreate.class, userFourScope, Optional.of("7")); + PersistentResource.createObject(UpdateAndCreate.class, userFourScope, Optional.of("7")); created.updateAttribute("alias", ""); } ); } - //----------------------------------------- ** Update Relation On Create ** -------------------------------------- //Expression for relation inherited from class level expression @Test public void createPermissionInheritedForRelationSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("8")); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("8")); + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userOneScope); + created.addRelation("books", loadedBook); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); } @Test public void createPermissionInheritedForRelationFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userThreeScope, Optional.of("9")); - PersistentResource loadedBook = PersistentResource.loadRecord(Book.class, - "1", + RequestScope userThreeScope = new TestRequestScope(tx, userThree, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Book()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userThreeScope, Optional.of("9")); + PersistentResource loadedBook = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Book.class) + .build(), + "2", userThreeScope); assertThrows(ForbiddenAccessException.class, () -> created.addRelation("books", loadedBook)); } @@ -258,9 +403,22 @@ public void createPermissionInheritedForRelationFailureCase() { //Class level expression overwritten by field level expression @Test public void createPermissionOverwrittenForRelationSuccessCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userTwoScope, Optional.of("10")); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + RequestScope userTwoScope = new TestRequestScope(tx, userTwo, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userTwoScope, Optional.of("10")); + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userTwoScope); created.addRelation("author", loadedAuthor); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -268,9 +426,22 @@ public void createPermissionOverwrittenForRelationSuccessCase() { @Test public void createPermissionOverwrittenForRelationFailureCase() { - PersistentResource created = PersistentResource.createObject(null, UpdateAndCreate.class, userOneScope, Optional.of("11")); - PersistentResource loadedAuthor = PersistentResource.loadRecord(Author.class, - "1", + RequestScope userOneScope = new TestRequestScope(tx, userOne, dictionary); + + UpdateAndCreate updateAndCreateNewObject = new UpdateAndCreate(); + when(tx.createNewObject(UpdateAndCreate.class)).thenReturn(updateAndCreateNewObject); + + when(tx.loadObject(any(), + eq(2L), + any(RequestScope.class) + )).thenReturn(new Author()); + + PersistentResource created = PersistentResource.createObject(UpdateAndCreate.class, userOneScope, Optional.of("11")); + PersistentResource loadedAuthor = PersistentResource.loadRecord( + EntityProjection.builder() + .type(Author.class) + .build(), + "2", userOneScope); assertThrows(ForbiddenAccessException.class, () -> created.addRelation("author", loadedAuthor)); } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java index bf09246bb6..ba2b8b4b61 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java @@ -6,6 +6,7 @@ package com.yahoo.elide.core.datastore.inmemory; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; @@ -27,15 +28,19 @@ import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Lists; import com.google.common.collect.Sets; + import example.Author; import example.Book; import example.Editor; import example.Publisher; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; +import org.mockito.ArgumentCaptor; import java.util.ArrayList; import java.util.Collection; @@ -43,7 +48,6 @@ import java.util.HashSet; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.Set; import java.util.stream.Collectors; @@ -53,7 +57,7 @@ public class InMemoryStoreTransactionTest { private RequestScope scope = mock(RequestScope.class); private InMemoryStoreTransaction inMemoryStoreTransaction = new InMemoryStoreTransaction(wrappedTransaction); private EntityDictionary dictionary; - private Set books = new HashSet<>(); + private Set books = new HashSet<>(); private Book book1; private Book book2; private Book book3; @@ -128,24 +132,17 @@ public void testFullFilterPredicatePushDown() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); - when(wrappedTransaction.supportsFiltering(eq(Book.class), - any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); + when(wrappedTransaction.loadObjects(eq(projection), eq(scope))).thenReturn(books); - verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression)), - eq(Optional.empty()), - eq(Optional.empty()), - eq(scope)); + Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects(projection, scope); + + verify(wrappedTransaction, times(1)).loadObjects(eq(projection), eq(scope)); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); @@ -158,89 +155,66 @@ public void testTransactionRequiresInMemoryFilterDuringGetRelation() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); + Relationship relationship = Relationship.builder() + .projection(EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build()) + .name("books") + .alias("books") + .build(); + + ArgumentCaptor relationshipArgument = ArgumentCaptor.forClass(Relationship.class); + when(scope.getNewPersistentResources()).thenReturn(Sets.newHashSet(mock(PersistentResource.class))); when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.getRelation(eq(inMemoryStoreTransaction), eq(author), eq("books"), - eq(Optional.empty()), eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn(books); + when(wrappedTransaction.getRelation(eq(inMemoryStoreTransaction), eq(author), any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.getRelation( - inMemoryStoreTransaction, - author, - "books", - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + inMemoryStoreTransaction, author, relationship, scope); verify(wrappedTransaction, times(1)).getRelation( eq(inMemoryStoreTransaction), eq(author), - eq("books"), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + relationshipArgument.capture(), eq(scope)); + assertNull(relationshipArgument.getValue().getProjection().getFilterExpression()); + assertNull(relationshipArgument.getValue().getProjection().getSorting()); + assertNull(relationshipArgument.getValue().getProjection().getPagination()); + assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); } @Test - public void testTransactionRequiresInMemoryFilterDuringLoad() { + public void testDataStoreRequiresTotalInMemoryFilter() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); - when(wrappedTransaction.supportsFiltering(eq(Book.class), - any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); - - verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression)), - eq(Optional.empty()), - eq(Optional.empty()), - eq(scope)); - - assertEquals(3, loaded.size()); - assertTrue(loaded.contains(book1)); - assertTrue(loaded.contains(book2)); - assertTrue(loaded.contains(book3)); - } - - @Test - public void testDataStoreRequiresTotalInMemoryFilter() { - FilterExpression expression = - new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); - Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), - scope); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); + + Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects(projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); @@ -254,25 +228,28 @@ public void testDataStoreRequiresPartialInMemoryFilter() { new InPredicate(new Path(Book.class, dictionary, "editor.firstName"), "Jane"); FilterExpression expression = new AndFilterExpression(expression1, expression2); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.PARTIAL); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.of(expression1)), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.of(expression1)), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertEquals(projectionArgument.getValue().getFilterExpression(), expression1); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(1, loaded.size()); assertTrue(loaded.contains(book3)); } @@ -284,28 +261,30 @@ public void testSortingPushDown() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(true); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.of(sorting)), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.of(sorting)), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertEquals(projectionArgument.getValue().getSorting(), sorting); assertEquals(3, loaded.size()); } @@ -316,28 +295,30 @@ public void testDataStoreRequiresInMemorySorting() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(false); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); List bookTitles = loaded.stream().map((o) -> ((Book) o).getTitle()).collect(Collectors.toList()); @@ -354,28 +335,31 @@ public void testFilteringRequiresInMemorySorting() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .sorting(sorting) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(true); - - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.of(sorting), - Optional.empty(), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); List bookTitles = loaded.stream().map((o) -> ((Book) o).getTitle()).collect(Collectors.toList()); @@ -386,27 +370,30 @@ public void testFilteringRequiresInMemorySorting() { public void testPaginationPushDown() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.of(pagination)), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.of(pagination)), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertEquals(projectionArgument.getValue().getPagination(), pagination); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); } @@ -414,27 +401,30 @@ public void testPaginationPushDown() { public void testDataStoreRequiresInMemoryPagination() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(false); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book2)); @@ -448,27 +438,31 @@ public void testFilteringRequiresInMemoryPagination() { Pagination pagination = Pagination.getDefaultPagination(elideSettings); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .filterExpression(expression) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.of(expression), - Optional.empty(), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(2, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book3)); @@ -483,29 +477,33 @@ public void testSortingRequiresInMemoryPagination() { Sorting sorting = new Sorting(sortOrder); + EntityProjection projection = EntityProjection.builder() + .type(Book.class) + .sorting(sorting) + .pagination(pagination) + .build(); + + ArgumentCaptor projectionArgument = ArgumentCaptor.forClass(EntityProjection.class); + when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(false); when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); - when(wrappedTransaction.loadObjects(eq(Book.class), eq(Optional.empty()), - eq(Optional.empty()), eq(Optional.empty()), eq(scope))).thenReturn((Set) books); + when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); Collection loaded = (Collection) inMemoryStoreTransaction.loadObjects( - Book.class, - Optional.empty(), - Optional.of(sorting), - Optional.of(pagination), + projection, scope); verify(wrappedTransaction, times(1)).loadObjects( - eq(Book.class), - eq(Optional.empty()), - eq(Optional.empty()), - eq(Optional.empty()), + projectionArgument.capture(), eq(scope)); + assertNull(projectionArgument.getValue().getFilterExpression()); + assertNull(projectionArgument.getValue().getPagination()); + assertNull(projectionArgument.getValue().getSorting()); assertEquals(3, loaded.size()); assertTrue(loaded.contains(book1)); assertTrue(loaded.contains(book2)); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java index 4a84d6a615..0141bffefb 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java @@ -10,21 +10,20 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.eq; +import static org.mockito.ArgumentMatchers.isA; import static org.mockito.Mockito.mock; import static org.mockito.Mockito.times; import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.request.Attribute; import com.yahoo.elide.security.User; import org.junit.jupiter.api.Test; -import java.util.Optional; - public class TransactionWrapperTest { private class TestTransactionWrapper extends TransactionWrapper { - public TestTransactionWrapper(DataStoreTransaction wrapped) { super(wrapped); } @@ -84,12 +83,11 @@ public void testLoadObjects() throws Exception { DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); Iterable expected = mock(Iterable.class); - when(wrapped.loadObjects(any(), any(), any(), any(), any())).thenReturn(expected); + when(wrapped.loadObjects(any(), any())).thenReturn(expected); - Iterable actual = wrapper.loadObjects(null, Optional.empty(), - Optional.empty(), Optional.empty(), null); + Iterable actual = wrapper.loadObjects(null, null); - verify(wrapped, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrapped, times(1)).loadObjects(any(), any()); assertEquals(expected, actual); } @@ -184,11 +182,11 @@ public void testGetAttribute() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.getAttribute(any(), any(), any())).thenReturn(1L); + when(wrapped.getAttribute(any(), isA(Attribute.class), any())).thenReturn(1L); - Object actual = wrapper.getAttribute(null, null, null); + Object actual = wrapper.getAttribute(null, Attribute.builder().name("foo").type(String.class).build(), null); - verify(wrapped, times(1)).getAttribute(any(), any(), any()); + verify(wrapped, times(1)).getAttribute(any(), isA(Attribute.class), any()); assertEquals(1L, actual); } @@ -197,9 +195,9 @@ public void testSetAttribute() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - wrapper.setAttribute(null, null, null, null); + wrapper.setAttribute(null, null, null); - verify(wrapped, times(1)).setAttribute(any(), any(), any(), any()); + verify(wrapped, times(1)).setAttribute(any(), any(), any()); } @Test @@ -227,12 +225,11 @@ public void testGetRelation() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.getRelation(any(), any(), any(), any(), any(), any(), any())).thenReturn(1L); + when(wrapped.getRelation(any(), any(), any(), any())).thenReturn(1L); - Object actual = wrapper.getRelation(null, null, null, null, - null, null, null); + Object actual = wrapper.getRelation(null, null, null, null); - verify(wrapped, times(1)).getRelation(any(), any(), any(), any(), any(), any(), any()); + verify(wrapped, times(1)).getRelation(any(), any(), any(), any()); assertEquals(1L, actual); } @@ -241,11 +238,11 @@ public void testLoadObject() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.loadObject(any(), any(), any(), any())).thenReturn(1L); + when(wrapped.loadObject(any(), any(), any())).thenReturn(1L); - Object actual = wrapper.loadObject(null, null, null, null); + Object actual = wrapper.loadObject(null, null, null); - verify(wrapped, times(1)).loadObject(any(), any(), any(), any()); + verify(wrapped, times(1)).loadObject(any(), any(), any()); assertEquals(1L, actual); } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java index 132931b5dd..15174cf479 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java @@ -9,6 +9,7 @@ import static org.junit.jupiter.api.Assertions.assertTrue; import com.yahoo.elide.annotation.ReadPermission; +import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.utils.ClassScanner; import org.apache.commons.collections4.IterableUtils; @@ -41,4 +42,14 @@ public void testGetAllAnnotatedClasses() { assertTrue(cls.isAnnotationPresent(ReadPermission.class)); } } + + @Test + public void testGetAnyAnnotatedClasses() { + Set> classes = ClassScanner.getAnnotatedClasses(ReadPermission.class, UpdatePermission.class); + assertEquals(37, classes.size()); + for (Class cls : classes) { + assertTrue(cls.isAnnotationPresent(ReadPermission.class) + || cls.isAnnotationPresent(UpdatePermission.class)); + } + } } diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java new file mode 100644 index 0000000000..e970bb0233 --- /dev/null +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java @@ -0,0 +1,745 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.jsonapi; + +import static org.junit.jupiter.api.Assertions.assertEquals; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestRequestScope; +import com.yahoo.elide.core.filter.InInsensitivePredicate; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import example.Address; +import example.Author; +import example.Book; +import example.Editor; +import example.Publisher; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; +import org.junit.jupiter.api.TestInstance; + +import java.util.HashMap; + +import javax.ws.rs.core.MultivaluedHashMap; +import javax.ws.rs.core.MultivaluedMap; + + +@TestInstance(TestInstance.Lifecycle.PER_CLASS) +public class EntityProjectionMakerTest { + private EntityDictionary dictionary; + + @BeforeAll + public void init() { + dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(Book.class); + dictionary.bindEntity(Author.class); + dictionary.bindEntity(Publisher.class); + dictionary.bindEntity(Editor.class); + } + + @Test + public void testRootCollectionNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionSparseFields() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("fields[book]", "title,publishDate,authors"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/book/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/books/3/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedEntityNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/books/3/publisher/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipNoQueryParams() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + String path = "/author/1/relationships/books"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .pagination(defaultPagination) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book/1/relationships/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .pagination(defaultPagination) + .build()) + .relationship("authors", EntityProjection.builder() + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .type(Author.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + String path = "/book/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithNestedInclude() throws Exception { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + String path = "/author"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("firstName").type(String.class).build()) + .attribute(Attribute.builder().name("lastName").type(String.class).build()) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithNestedInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + String path = "/author/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("firstName").type(String.class).build()) + .attribute(Attribute.builder().name("lastName").type(String.class).build()) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedEntityWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + String path = "/author/1/books/3/publisher/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionWithSingleInclude() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + String path = "/author/1/books/3/publisher"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .relationship("books", EntityProjection.builder() + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootEntityWithNestedIncludeAndSparseFields() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "books"); + queryParams.add("include", "books.publisher,books.editor"); + queryParams.add("fields[publisher]", "name"); + queryParams.add("fields[editor]", "fullName"); + queryParams.add("fields[book]", "publisher,editor,title"); + String path = "/author/1"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .attribute(Attribute.builder().name("fullName").type(String.class).build()) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithGlobalFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter", "genre=='Science Fiction'"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + FilterExpression expression = + new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .filterExpression(expression) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testNestedCollectionWithTypedFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter[publisher]", "name=='Foo'"); + String path = "/author/1/books/3/publisher"; + + FilterExpression expression = + new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo"); + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Author.class) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("updateHookInvoked").type(boolean.class).build()) + .filterExpression(expression) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build()) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRelationshipsAndIncludeWithFilterAndSort() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("include", "authors"); + queryParams.add("filter[author]", "name=='Foo'"); + queryParams.add("filter[publisher]", "name=='Foo'"); + queryParams.add("sort", "name"); + String path = "/book/1/relationships/publisher"; + Sorting sorting = Sorting.parseSortRule("name"); + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .filterExpression(new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo")) + .sorting(sorting) + .pagination(defaultPagination) + .build()) + .relationship("authors", EntityProjection.builder() + .attribute(Attribute.builder().name("name").type(String.class).build()) + .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .filterExpression(new InInsensitivePredicate(new Path(Author.class, dictionary, "name"), "Foo")) + .relationship("books", EntityProjection.builder() + .type(Book.class) + .build()) + .type(Author.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } + + @Test + public void testRootCollectionWithTypedFilter() { + MultivaluedMap queryParams = new MultivaluedHashMap<>(); + queryParams.add("filter[book]", "genre=='Science Fiction'"); + String path = "/book"; + + RequestScope scope = new TestRequestScope(dictionary, path, queryParams); + Pagination defaultPagination = scope.getPagination(); + + FilterExpression expression = + new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); + + EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); + + EntityProjection expected = EntityProjection.builder() + .type(Book.class) + .attribute(Attribute.builder().name("title").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("publishDate").type(long.class).build()) + .filterExpression(expression) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .relationship("publisher", EntityProjection.builder() + .type(Publisher.class) + .build()) + .relationship("editor", EntityProjection.builder() + .type(Editor.class) + .build()) + .pagination(defaultPagination) + .build(); + + EntityProjection actual = maker.parsePath(path); + + assertEquals(expected, actual); + } +} diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java index 31aa9c7093..0a2b134d00 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java @@ -7,16 +7,14 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNull; -import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; -import com.yahoo.elide.ElideSettingsBuilder; -import com.yahoo.elide.audit.AuditLogger; -import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; +import com.yahoo.elide.core.TestRequestScope; import com.yahoo.elide.jsonapi.models.Data; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Meta; @@ -31,10 +29,8 @@ import example.Child; import example.Parent; -import example.TestCheckMappings; - import org.apache.commons.collections4.IterableUtils; -import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.Answers; @@ -50,37 +46,17 @@ * JSON API testing. */ public class JsonApiTest { - private static RequestScope userScope; - private static JsonApiMapper mapper; - - @BeforeAll - static void init() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + private JsonApiMapper mapper; + private User user = new User(0); + private EntityDictionary dictionary; + private DataStoreTransaction tx = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); + + @BeforeEach + void init() { + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Parent.class); dictionary.bindEntity(Child.class); - dictionary.bindInitializer(Parent::doInit, Parent.class); mapper = new JsonApiMapper(dictionary); - AuditLogger testLogger = new TestAuditLogger(); - userScope = new RequestScope(null, new JsonApiDocument(), - mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), new User(0), null, - new ElideSettingsBuilder(null) - .withJsonApiMapper(mapper) - .withAuditLogger(testLogger) - .withEntityDictionary(dictionary) - .build()); - } - - @Test - public void checkInit() { - // Ensure that our object receives its init before serializing - Parent parent = new Parent(); - parent.setId(123L); - parent.setChildren(Sets.newHashSet()); - parent.setSpouses(Sets.newHashSet()); - - new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource(); - - assertTrue(parent.init); } @Test @@ -88,6 +64,8 @@ public void writeSingleNoAttributesNoRel() throws JsonProcessingException { Parent parent = new Parent(); parent.setId(123L); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource())); @@ -112,6 +90,8 @@ public void writeSingle() throws JsonProcessingException { child.setParents(Collections.singleton(parent)); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource())); @@ -136,11 +116,14 @@ public void writeSingleIncluded() throws JsonProcessingException { child.setParents(Collections.singleton(parent)); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + PersistentResource pRec = new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope); JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(pRec.toResource())); - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded( + new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); String expected = "{\"data\":{\"type\":\"parent\",\"id\":\"123\",\"attributes\":{\"firstName\":\"bob\"},\"relationships\":{\"children\":{\"data\":[{\"type\":\"child\",\"id\":\"2\"}]},\"spouses\":{\"data\":[]}}},\"included\":[{\"type\":\"child\",\"id\":\"2\",\"attributes\":{\"name\":null},\"relationships\":{\"friends\":{\"data\":[]},\"parents\":{\"data\":[{\"type\":\"parent\",\"id\":\"123\"}]}}}]}"; @@ -164,6 +147,8 @@ public void writeList() throws JsonProcessingException { parent.setFirstName("bob"); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData( new Data<>(Collections.singletonList(new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope).toResource()))); @@ -189,13 +174,17 @@ public void writeListIncluded() throws JsonProcessingException { parent.setFirstName("bob"); child.setFriends(new HashSet<>()); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + PersistentResource pRec = new PersistentResource<>(parent, null, userScope.getUUIDFor(parent), userScope); JsonApiDocument jsonApiDocument = new JsonApiDocument(); jsonApiDocument.setData(new Data<>(Collections.singletonList(pRec.toResource()))); - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded(new PersistentResource<>(child, + pRec, userScope.getUUIDFor(child), userScope).toResource()); // duplicate will be ignored - jsonApiDocument.addIncluded(new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); + jsonApiDocument.addIncluded( + new PersistentResource<>(child, pRec, userScope.getUUIDFor(child), userScope).toResource()); String expected = "{\"data\":[{\"type\":\"parent\",\"id\":\"123\",\"attributes\":{\"firstName\":\"bob\"},\"relationships\":{\"children\":{\"data\":[{\"type\":\"child\",\"id\":\"2\"}]},\"spouses\":{\"data\":[]}}}],\"included\":[{\"type\":\"child\",\"id\":\"2\",\"attributes\":{\"name\":null},\"relationships\":{\"friends\":{\"data\":[]},\"parents\":{\"data\":[{\"type\":\"parent\",\"id\":\"123\"}]}}}]}"; diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java index c835d130a5..bf3e052990 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/document/processors/IncludedProcessorTest.java @@ -8,14 +8,13 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNull; import static org.mockito.Mockito.mock; +import static org.mockito.Mockito.reset; -import com.yahoo.elide.ElideSettings; -import com.yahoo.elide.ElideSettingsBuilder; -import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; +import com.yahoo.elide.core.TestRequestScope; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.jsonapi.models.Resource; import com.yahoo.elide.security.User; @@ -24,7 +23,6 @@ import example.Child; import example.FunWithPermissions; import example.Parent; -import example.TestCheckMappings; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.mockito.Answers; @@ -38,7 +36,6 @@ import javax.ws.rs.core.MultivaluedHashMap; import javax.ws.rs.core.MultivaluedMap; - public class IncludedProcessorTest { private static final String INCLUDE = "include"; @@ -55,29 +52,22 @@ public class IncludedProcessorTest { private PersistentResource funWithPermissionsRecord; + private DataStoreTransaction mockTransaction = mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS); + private TestRequestScope testScope; + private EntityDictionary dictionary; + @BeforeEach public void setUp() throws Exception { includedProcessor = new IncludedProcessor(); - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(Child.class); dictionary.bindEntity(Parent.class); dictionary.bindEntity(FunWithPermissions.class); - ElideSettings elideSettings = new ElideSettingsBuilder(null) - .withAuditLogger(new TestAuditLogger()) - .withEntityDictionary(dictionary) - .build(); - - RequestScope goodUserScope = new RequestScope(null, - new JsonApiDocument(), mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), - new User(1), null, - elideSettings); + reset(mockTransaction); - RequestScope badUserScope = new RequestScope(null, - new JsonApiDocument(), mock(DataStoreTransaction.class, Answers.CALLS_REAL_METHODS), - new User(-1), null, - elideSettings); + testScope = new TestRequestScope(mockTransaction, new User(1), dictionary); //Create objects Parent parent1 = newParent(1); @@ -102,16 +92,17 @@ public void setUp() throws Exception { child3.setFriends(new HashSet<>(Collections.singletonList(child1))); child4.setFriends(new HashSet<>(Collections.singletonList(child2))); - //Create Persistent Resources - parentRecord1 = new PersistentResource<>(parent1, null, goodUserScope.getUUIDFor(parent1), goodUserScope); - parentRecord2 = new PersistentResource<>(parent2, null, goodUserScope.getUUIDFor(parent2), goodUserScope); - parentRecord3 = new PersistentResource<>(parent3, null, goodUserScope.getUUIDFor(parent3), goodUserScope); - childRecord1 = new PersistentResource<>(child1, null, goodUserScope.getUUIDFor(child1), goodUserScope); - childRecord2 = new PersistentResource<>(child2, null, goodUserScope.getUUIDFor(child2), goodUserScope); - childRecord3 = new PersistentResource<>(child3, null, goodUserScope.getUUIDFor(child3), goodUserScope); - childRecord4 = new PersistentResource<>(child4, null, goodUserScope.getUUIDFor(child4), goodUserScope); - - funWithPermissionsRecord = new PersistentResource<>(funWithPermissions, null, goodUserScope.getUUIDFor(funWithPermissions), badUserScope); + //Create Persistent Resource + parentRecord1 = new PersistentResource<>(parent1, null, String.valueOf(parent1.getId()), testScope); + parentRecord2 = new PersistentResource<>(parent2, null, String.valueOf(parent2.getId()), testScope); + parentRecord3 = new PersistentResource<>(parent3, null, String.valueOf(parent3.getId()), testScope); + childRecord1 = new PersistentResource<>(child1, null, String.valueOf(child1.getId()), testScope); + childRecord2 = new PersistentResource<>(child2, null, String.valueOf(child2.getId()), testScope); + childRecord3 = new PersistentResource<>(child3, null, String.valueOf(child3.getId()), testScope); + childRecord4 = new PersistentResource<>(child4, null, String.valueOf(child4.getId()), testScope); + + funWithPermissionsRecord = new PersistentResource<>(funWithPermissions, null, + String.valueOf(funWithPermissions.getId()), testScope); } @Test @@ -120,6 +111,7 @@ public void testExecuteSingleRelation() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = Collections.singletonList(childRecord1.toResource()); @@ -139,6 +131,7 @@ public void testExecuteSingleRelationOnCollection() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parents, Optional.of(queryParams)); List expectedIncluded = Arrays.asList(childRecord1.toResource(), childRecord2.toResource()); @@ -150,10 +143,12 @@ public void testExecuteSingleRelationOnCollection() throws Exception { @Test public void testExecuteSingleNestedRelation() throws Exception { + JsonApiDocument jsonApiDocument = new JsonApiDocument(); MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children.friends")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = @@ -170,6 +165,7 @@ public void testExecuteMultipleRelations() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Arrays.asList("children", "spouses")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); List expectedIncluded = @@ -186,6 +182,7 @@ public void testExecuteMultipleNestedRelations() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("children.friends")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord3, Optional.of(queryParams)); Set expectedIncluded = @@ -207,6 +204,7 @@ public void testIncludeForbiddenRelationship() { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put(INCLUDE, Collections.singletonList("relation1")); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, funWithPermissionsRecord, Optional.of(queryParams)); assertNull(jsonApiDocument.getIncluded(), @@ -229,6 +227,7 @@ public void testNoQueryIncludeParams() throws Exception { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.put("unused", Collections.emptyList()); + testScope.setQueryParams(queryParams); includedProcessor.execute(jsonApiDocument, parentRecord1, Optional.of(queryParams)); assertNull(jsonApiDocument.getIncluded(), diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java index e620771243..1ba0e7a472 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/CanPaginateVisitorTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.FilterExpressionCheck; @@ -90,7 +91,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -109,7 +110,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -127,7 +128,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -146,7 +147,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -166,7 +167,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -185,7 +186,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -204,7 +205,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -223,7 +224,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -242,7 +243,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -261,7 +262,7 @@ class Book { private String title; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -283,7 +284,7 @@ class Book { private Date publicationDate; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -306,7 +307,7 @@ class Book { private Date publicationDate; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); @@ -331,7 +332,7 @@ class Book { private boolean outOfPrint; } - EntityDictionary dictionary = new EntityDictionary(checkMappings); + EntityDictionary dictionary = TestDictionary.getTestDictionary(checkMappings); dictionary.bindEntity(Book.class); RequestScope scope = mock(RequestScope.class); diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java index 5c463ba470..c83f8b28e3 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java @@ -13,6 +13,7 @@ import com.yahoo.elide.annotation.ReadPermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.Check; @@ -47,7 +48,7 @@ public void setupEntityDictionary() { checks.put("user has all access", Role.ALL.class); checks.put("user has no access", Role.NONE.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); dictionary.bindEntity(Model.class); dictionary.bindEntity(ComplexEntity.class); } diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java index 310b9531c5..ee97249e46 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitorTest.java @@ -19,6 +19,7 @@ import com.yahoo.elide.core.EntityPermissions; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.Operator; import com.yahoo.elide.core.filter.expression.AndFilterExpression; @@ -94,7 +95,7 @@ public void setupEntityDictionary() { checks.put(LT_FILTER, Permissions.LessThanFilterExpression.class); checks.put(GE_FILTER, Permissions.GreaterThanOrEqualFilterExpression.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); elideSettings = new ElideSettingsBuilder(null) .withEntityDictionary(dictionary) .build(); diff --git a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java index db5c42da02..7bdbcad758 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java @@ -16,13 +16,12 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import com.yahoo.elide.security.checks.UserCheck; -import example.TestCheckMappings; - import org.junit.jupiter.api.Test; import java.util.Optional; @@ -413,15 +412,15 @@ public void testUserCheckCache() { requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, cspec); } - public PersistentResource newResource(T obj, Class cls) { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + public PersistentResource newResource(T obj, Class cls) { + EntityDictionary dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(cls); RequestScope requestScope = new RequestScope(null, null, null, null, null, getElideSettings(dictionary)); return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); } - public PersistentResource newResource(Class cls) { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); + public PersistentResource newResource(Class cls) { + EntityDictionary dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(cls); RequestScope requestScope = new RequestScope(null, null, null, null, null, getElideSettings(dictionary)); try { diff --git a/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java b/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java index e09a757a15..2b1df5c2c2 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/permissions/PermissionExpressionBuilderTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.checks.Check; import com.yahoo.elide.security.checks.prefab.Role; @@ -40,7 +41,7 @@ public void setupEntityDictionary() { checks.put("user has all access", Role.ALL.class); checks.put("user has no access", Role.NONE.class); - dictionary = new EntityDictionary(checks); + dictionary = TestDictionary.getTestDictionary(checks); ExpressionResultCache cache = new ExpressionResultCache(); builder = new PermissionExpressionBuilder(cache, dictionary); diff --git a/elide-datastore/elide-datastore-aggregation/.gitignore b/elide-datastore/elide-datastore-aggregation/.gitignore new file mode 100644 index 0000000000..2f7896d1d1 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/.gitignore @@ -0,0 +1 @@ +target/ diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml new file mode 100644 index 0000000000..c2da5362a9 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -0,0 +1,203 @@ + + + + 4.0.0 + elide-datastore-aggregation + jar + Elide Data Store: Aggregation Data Store + Elide Data Store for Aggregation + https://github.com/yahoo/elide/tree/master/elide-datastore/elide-datastore-aggregation + + com.yahoo.elide + elide-datastore-parent-pom + 5.0.0-pr6-SNAPSHOT + + + + + The Apache Software License, Version 2.0 + http://www.apache.org/licenses/LICENSE-2.0.txt + repo + + + + + + Yahoo Inc. + https://github.com/yahoo + + + + + scm:git:ssh://git@github.com/yahoo/elide.git + https://github.com/yahoo/elide.git + HEAD + + + + 5.4.1 + + + + + com.yahoo.elide + elide-core + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-datastore-jpa + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-graphql + 5.0.0-pr6-SNAPSHOT + + + com.yahoo.elide + elide-datastore-multiplex + 5.0.0-pr6-SNAPSHOT + + + + org.projectlombok + lombok + + + + + org.hibernate.javax.persistence + hibernate-jpa-2.1-api + 1.0.0.Final + + + + + org.hibernate + hibernate-entitymanager + ${hibernate5.version} + + + + + + org.junit.jupiter + junit-jupiter-api + ${junit.version} + test + + + + org.junit.jupiter + junit-jupiter-params + ${junit.version} + test + + + + org.junit.jupiter + junit-jupiter-engine + ${junit.version} + test + + + + + com.h2database + h2 + 1.4.197 + test + + + + + org.mockito + mockito-core + test + + + + + com.yahoo.elide + elide-integration-tests + 5.0.0-pr6-SNAPSHOT + test + test-jar + + + + + org.eclipse.jetty + jetty-servlet + test + + + + org.eclipse.jetty + jetty-webapp + test + + + + + org.glassfish.jersey.containers + jersey-container-servlet-core + 2.29.1 + test + + + + org.glassfish.jersey.inject + jersey-hk2 + test + + + + org.glassfish.jersey.containers + jersey-container-servlet + test + + + + io.rest-assured + rest-assured + 4.0.0 + test + + + + org.glassfish.hk2 + hk2-api + 2.5.0 + test + + + + + + + org.codehaus.mojo + build-helper-maven-plugin + + + org.apache.maven.plugins + maven-dependency-plugin + + + org.apache.maven.plugins + maven-surefire-plugin + + + org.apache.maven.plugins + maven-checkstyle-plugin + + + + diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java new file mode 100644 index 0000000000..b32a42dd14 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -0,0 +1,72 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.ArgumentType; +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.utils.ClassScanner; + +import java.lang.annotation.Annotation; + +/** + * DataStore that supports Aggregation. Uses {@link QueryEngine} to return results. + */ +public class AggregationDataStore implements DataStore { + + private final QueryEngineFactory queryEngineFactory; + + private final MetaDataStore metaDataStore; + + private QueryEngine queryEngine; + + /** + * These are the classes the Aggregation Store manages. + */ + private static final Class[] AGGREGATION_STORE_CLASSES = { + FromTable.class, FromSubquery.class }; + + public AggregationDataStore(QueryEngineFactory queryEngineFactory, + MetaDataStore metaDataStore) { + this.queryEngineFactory = queryEngineFactory; + this.metaDataStore = metaDataStore; + } + + /** + * Populate an {@link EntityDictionary} and use this dictionary to construct a {@link QueryEngine}. + * @param dictionary the dictionary + */ + @Override + public void populateEntityDictionary(EntityDictionary dictionary) { + for (Class cls : AGGREGATION_STORE_CLASSES) { + // bind non-jpa entities, including analyticViews and views + ClassScanner.getAnnotatedClasses(cls).forEach(dictionary::bindEntity); + } + + queryEngine = queryEngineFactory.buildQueryEngine(metaDataStore); + + /* Add 'grain' argument to each TimeDimensionColumn */ + for (AnalyticView table : metaDataStore.getMetaData(AnalyticView.class)) { + for (TimeDimension timeDim : table.getColumns(TimeDimension.class)) { + dictionary.addArgumentToAttribute( + table.getCls(), + timeDim.getName(), + new ArgumentType("grain", String.class)); + } + } + } + + @Override + public DataStoreTransaction beginTransaction() { + return new AggregationDataStoreTransaction(queryEngine); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java new file mode 100644 index 0000000000..fd6daed82b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java @@ -0,0 +1,71 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.request.EntityProjection; + +import com.google.common.annotations.VisibleForTesting; + +import java.io.IOException; + +/** + * Transaction handler for {@link AggregationDataStore}. + */ +public class AggregationDataStoreTransaction implements DataStoreTransaction { + private QueryEngine queryEngine; + + public AggregationDataStoreTransaction(QueryEngine queryEngine) { + this.queryEngine = queryEngine; + } + + @Override + public void save(Object entity, RequestScope scope) { + + } + + @Override + public void delete(Object entity, RequestScope scope) { + + } + + @Override + public void flush(RequestScope scope) { + + } + + @Override + public void commit(RequestScope scope) { + + } + + @Override + public void createObject(Object entity, RequestScope scope) { + + } + + @Override + public Iterable loadObjects(EntityProjection entityProjection, RequestScope scope) { + Query query = buildQuery(entityProjection, scope); + return queryEngine.executeQuery(query); + } + + @Override + public void close() throws IOException { + + } + + @VisibleForTesting + private Query buildQuery(EntityProjection entityProjection, RequestScope scope) { + Table table = queryEngine.getTable(entityProjection.getType()); + EntityProjectionTranslator translator = new EntityProjectionTranslator(table, + entityProjection, scope.getDictionary()); + return translator.getQuery(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java new file mode 100644 index 0000000000..86ed3dfd7b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java @@ -0,0 +1,206 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import com.google.common.collect.Sets; + +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Locale; +import java.util.Objects; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * Helper for Aggregation Data Store which does the work associated with extracting {@link Query}. + */ +public class EntityProjectionTranslator { + private AnalyticView queriedTable; + + private EntityProjection entityProjection; + private Set dimensionProjections; + private Set timeDimensions; + private List metrics; + private FilterExpression whereFilter; + private FilterExpression havingFilter; + private EntityDictionary dictionary; + + public EntityProjectionTranslator(Table table, EntityProjection entityProjection, EntityDictionary dictionary) { + if (!(table instanceof AnalyticView)) { + throw new InvalidOperationException("Queried table is not analyticView: " + table.getName()); + } + + this.queriedTable = (AnalyticView) table; + this.entityProjection = entityProjection; + this.dictionary = dictionary; + dimensionProjections = resolveNonTimeDimensions(); + timeDimensions = resolveTimeDimensions(); + metrics = resolveMetrics(); + splitFilters(); + } + + /** + * Builds the query from internal state. + * @return {@link Query} query object with all the parameters provided by user. + */ + public Query getQuery() { + Query query = Query.builder() + .analyticView(queriedTable) + .metrics(metrics) + .groupByDimensions(dimensionProjections) + .timeDimensions(timeDimensions) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .sorting(entityProjection.getSorting()) + .pagination(entityProjection.getPagination()) + .build(); + QueryValidator validator = new QueryValidator(query, getAllFields(), dictionary); + validator.validate(); + return query; + } + + /** + * Gets whereFilter and havingFilter based on provided filter expression from {@link EntityProjection}. + */ + private void splitFilters() { + FilterExpression filterExpression = entityProjection.getFilterExpression(); + if (filterExpression == null) { + whereFilter = null; + havingFilter = null; + return; + } + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(queriedTable); + FilterConstraints constraints = filterExpression.accept(visitor); + whereFilter = constraints.getWhereExpression(); + havingFilter = constraints.getHavingExpression(); + } + + /** + * Gets time dimensions based on relationships and attributes from {@link EntityProjection}. + * + * @return projections for time dimension columns + * @throws InvalidOperationException Thrown if a requested time grain is not supported. + */ + private Set resolveTimeDimensions() { + return entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.getTimeDimension(attribute.getName()) != null) + .map(timeDimAttr -> { + TimeDimension timeDim = queriedTable.getTimeDimension(timeDimAttr.getName()); + + Argument grainArgument = timeDimAttr.getArguments().stream() + .filter(attr -> attr.getName().equals("grain")) + .findAny() + .orElse(null); + + TimeDimensionGrain resolvedGrain; + if (grainArgument == null) { + //The first grain is the default. + resolvedGrain = timeDim.getSupportedGrains().stream() + .findFirst() + .orElseThrow(() -> new IllegalStateException( + String.format("Requested default grain, no grain defined on %s", + timeDimAttr.getName()))); + } else { + String requestedGrainName = grainArgument.getValue().toString(); + + resolvedGrain = timeDim.getSupportedGrains().stream() + .filter(supportedGrain -> supportedGrain.getGrain().name().toLowerCase(Locale.ENGLISH) + .equals(requestedGrainName)) + .findFirst() + .orElseThrow(() -> new InvalidOperationException( + String.format("Unsupported grain %s for field %s", + requestedGrainName, + timeDimAttr.getName()))); + } + + return ColumnProjection.toProjection(timeDim, resolvedGrain.getGrain(), timeDimAttr.getAlias()); + }) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Gets dimensions except time dimensions based on relationships and attributes from {@link EntityProjection}. + */ + private Set resolveNonTimeDimensions() { + Set attributes = entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.getTimeDimension(attribute.getName()) == null) + .map(dimAttr -> { + Dimension dimension = queriedTable.getDimension(dimAttr.getName()); + return dimension == null ? null : ColumnProjection.toProjection(dimension, dimAttr.getAlias()); + }) + .filter(Objects::nonNull) + .collect(Collectors.toSet()); + + Set relationships = entityProjection.getRelationships().stream() + .map(dimAttr -> { + Dimension dimension = queriedTable.getDimension(dimAttr.getName()); + return dimension == null ? null : ColumnProjection.toProjection(dimension, dimAttr.getAlias()); + }) + .filter(Objects::nonNull) + .collect(Collectors.toSet()); + + return Sets.union(attributes, relationships); + } + + /** + * Gets metrics based on attributes from {@link EntityProjection}. + */ + private List resolveMetrics() { + return entityProjection.getAttributes().stream() + .filter(attribute -> queriedTable.isMetric(attribute.getName())) + .map(attribute -> queriedTable.getMetric(attribute.getName()) + .getMetricFunction() + .invoke(attribute.getArguments(), attribute.getAlias())) + .collect(Collectors.toList()); + } + + /** + * Gets relationship names from {@link EntityProjection}. + * @return relationships list of {@link Relationship} names + */ + private Set getRelationships() { + return entityProjection.getRelationships().stream() + .map(Relationship::getName).collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Gets attribute names from {@link EntityProjection}. + * @return relationships list of {@link Attribute} names + */ + private Set getAttributes() { + return entityProjection.getAttributes().stream() + .map(Attribute::getName).collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Helper method to get all field names from the {@link EntityProjection}. + * @return allFields set of all field names + */ + private Set getAllFields() { + Set allFields = getAttributes(); + allFields.addAll(getRelationships()); + return allFields; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java new file mode 100644 index 0000000000..74dc06a008 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -0,0 +1,72 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.Query; + +/** + * A {@link QueryEngine} is an abstraction that an AggregationDataStore leverages to run analytic queries (OLAP style) + * against an underlying persistence layer. + *

+ * The purpose of {@link QueryEngine} is to allow a single {@link DataStore} to utilize multiple query frameworks, such + * as JPA on SQL or NoSQL query engine on Druid shown below. + * 

+ *        +-----------+
+ *        |           |
+ *        | DataStore |
+ *        |           |
+ *        +-----+-----+
+ *              |
+ *              |
+ *   +----------v-----------+
+ *   |                      |
+ *   | DataStoreTransaction |
+ *   |                      |
+ *   +----------+-----------+
+ *              |
+ *              |
+ *       +------v------+
+ *       |             |
+ *       | QueryEngine |
+ *       |             |
+ *       +------+------+
+ *              |
+ *              |
+ *     +--------+---------+
+ *     |                  |
+ * +---v---+          +---v---+
+ * |       |          |       |
+ * | Druid |          | MySQL |
+ * |       |          |       |
+ * +-------+          +-------+
+ *
+ * Implementor must assume that {@link DataStoreTransaction} will never keep reference to any internal state of a + * {@link QueryEngine} object. This ensures the plugability of various {@link QueryEngine} implementations. + *

+ * This is a {@link java.util.function functional interface} whose functional method is {@link #executeQuery(Query)}. + */ +public interface QueryEngine { + + /** + * Executes the specified {@link Query} against a specific persistent storage, which understand the provided + * {@link Query}. + * + * @param query The query customized for a particular persistent storage or storage client + * + * @return query results + */ + Iterable executeQuery(Query query); + + /** + * Returns the schema for a given entity class. + * @param entityClass The class to map to a schema. + * @return The schema that represents the provided entity. + */ + Table getTable(Class entityClass); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java new file mode 100644 index 0000000000..7a77dfdf6d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java @@ -0,0 +1,15 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; + +/** + * Interface that constructs {@link QueryEngine} based on given entityDictionary. + */ +public interface QueryEngineFactory { + QueryEngine buildQueryEngine(MetaDataStore metaDataStore); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java new file mode 100644 index 0000000000..a6ad1583fe --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java @@ -0,0 +1,155 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * Class that checks whether a constructed {@link Query} object can be executed. + * Checks include validate sorting, having clause and make sure there is at least 1 metric queried. + */ +public class QueryValidator { + private Query query; + private Set allFields; + private EntityDictionary dictionary; + private AnalyticView queriedTable; + private Class queriedClass; + private List metrics; + private Set dimensionProjections; + + public QueryValidator(Query query, Set allFields, EntityDictionary dictionary) { + this.query = query; + this.allFields = allFields; + this.dictionary = dictionary; + this.queriedTable = query.getAnalyticView(); + this.queriedClass = queriedTable.getCls(); + this.metrics = query.getMetrics(); + this.dimensionProjections = query.getDimensions(); + } + + /** + * Method that handles all checks to make sure query is valid before we attempt to execute the query. + */ + public void validate() { + validateHavingClause(query.getHavingFilter()); + validateSorting(); + } + + /** + * Validate the having clause before execution. Having clause is not as flexible as where clause, + * the fields in having clause must be either or these two: + * 1. A grouped by dimension in this query + * 2. An aggregated metric in this query + * + * All grouped by dimensions are defined in the entity bean, so the last entity class of a filter path + * must match entity class of the query. + * + * @param havingClause having clause generated from this query + */ + private void validateHavingClause(FilterExpression havingClause) { + // TODO: support having clause for alias + if (havingClause instanceof FilterPredicate) { + Path path = ((FilterPredicate) havingClause).getPath(); + Path.PathElement last = path.lastElement().get(); + Class cls = last.getType(); + String fieldName = last.getFieldName(); + + if (cls != queriedTable.getCls()) { + throw new InvalidOperationException( + String.format( + "Can't filter on relationship field %s in HAVING clause when querying table %s.", + path.toString(), + queriedTable.getCls().getSimpleName())); + } + + if (queriedTable.isMetric(fieldName)) { + if (metrics.stream().noneMatch(m -> m.getAlias().equals(fieldName))) { + throw new InvalidOperationException( + String.format( + "Metric field %s must be aggregated before filtering in having clause.", + fieldName)); + } + } else { + if (dimensionProjections.stream().noneMatch(dim -> dim.getAlias().equals(fieldName))) { + throw new InvalidOperationException( + String.format( + "Dimension field %s must be grouped before filtering in having clause.", + fieldName)); + } + } + } else if (havingClause instanceof AndFilterExpression) { + validateHavingClause(((AndFilterExpression) havingClause).getLeft()); + validateHavingClause(((AndFilterExpression) havingClause).getRight()); + } else if (havingClause instanceof OrFilterExpression) { + validateHavingClause(((OrFilterExpression) havingClause).getLeft()); + validateHavingClause(((OrFilterExpression) havingClause).getRight()); + } else if (havingClause instanceof NotFilterExpression) { + validateHavingClause(((NotFilterExpression) havingClause).getNegated()); + } + } + + /** + * Method to verify that all the sorting options provided + * by the user are valid and supported. + */ + public void validateSorting() { + Sorting sorting = query.getSorting(); + if (sorting == null) { + return; + } + Map sortClauses = sorting.getValidSortingRules(queriedClass, dictionary); + sortClauses.keySet().forEach((path) -> validateSortingPath(path, allFields)); + } + + /** + * Verifies that the current path can be sorted on + * @param path The path that we are validating + * @param allFields Set of all field names included in initial query + */ + private void validateSortingPath(Path path, Set allFields) { + List pathElements = path.getPathElements(); + + // TODO: add support for double nested sorting + if (pathElements.size() > 2) { + throw new UnsupportedOperationException( + "Currently sorting on double nested fields is not supported"); + } + + if (metrics.isEmpty() && pathElements.size() > 1) { + throw new UnsupportedOperationException( + "Query with no metric can't sort on nested field."); + } + + Path.PathElement currentElement = pathElements.get(0); + String currentField = currentElement.getFieldName(); + Class currentClass = currentElement.getType(); + + // TODO: support sorting using alias + if (allFields.stream().noneMatch(field -> field.equals(currentField))) { + throw new InvalidOperationException("Can't sort on " + currentField + " as it is not present in query"); + } + if (dictionary.getIdFieldName(currentClass).equals(currentField) + || currentField.equals(EntityDictionary.REGULAR_ID_NAME)) { + throw new InvalidOperationException("Sorting on id field is not permitted"); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java new file mode 100644 index 0000000000..3be3e35faf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Cardinality.java @@ -0,0 +1,44 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.ElementType.METHOD; +import static java.lang.annotation.ElementType.TYPE; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates whether a dimension has small, medium, or large cardinality. + *

+ * Example: {@literal @}Cardinality(size = {@link CardinalitySize#MEDIUM}). If {@code size} is not specified, + * {@link CardinalitySize#LARGE} is used by default. See {@link CardinalitySize}. + *

+ * In the case of double binding, the following precedence rule is applied: + *

    + *
  1. {@link ElementType#TYPE} + *
  2. {@link ElementType#METHOD} or {@link ElementType#FIELD} + *
+ */ +@Documented +@Target({ TYPE, FIELD, METHOD }) +@Retention(RetentionPolicy.RUNTIME) +public @interface Cardinality { + + /** + * Returns the size category of a dimension. + *

+ * The size category must be from one of the values of type {@link CardinalitySize}. {@link CardinalitySize#LARGE} + * will be the default if size is not specified. + * + * @return dimension size + */ + CardinalitySize size() default CardinalitySize.LARGE; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java new file mode 100644 index 0000000000..13766762df --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/CardinalitySize.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +/** + * A set of constants that indicates how big a dimension is. + */ +public enum CardinalitySize { + + /** + * Size for a small dimension table. + *

+ * TODO: define size range + */ + SMALL, + + /** + * Size for a medium sized dimension table. + *

+ * TODO: define size range + */ + MEDIUM, + + /** + * Size for a large dimension table. + *

+ * TODO: define size range + */ + LARGE + ; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java new file mode 100644 index 0000000000..ae8e06597d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/FriendlyName.java @@ -0,0 +1,23 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that an annotated field is a friendlyName or human displayable column for that dimension. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FriendlyName { + + // intentionally left blank +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java new file mode 100644 index 0000000000..384eb4abd8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Meta.java @@ -0,0 +1,25 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the specified entity field has a configured long name and field description for human to read on UI. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD, ElementType.TYPE}) +@Retention(RetentionPolicy.RUNTIME) +public @interface Meta { + + String longName() default ""; + + String description() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java new file mode 100644 index 0000000000..73da16f13c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricAggregation.java @@ -0,0 +1,24 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Specify that a field in a table is metric field. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface MetricAggregation { + Class function(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java new file mode 100644 index 0000000000..8762d856b3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/MetricComputation.java @@ -0,0 +1,62 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that a field is computed via a {@link #expression() custom metric formula expression}, such as Calcite SQL. + *

+ * Example: {@literal @}ComputedMetric(expression = '(fieldA * fieldB) / 100').The field names should match the Elide + * data model field names. + *

+ * {@code expression} can also be composite. During {@link Table} construction, it will substitute attribute names in + * the provided expression with either: + *

    + *
  • The column alias for that column in the query, or + *
  • Another ComputedMetric expression - recursively expanding expressions until every referenced field is not + * computed. + *
+ * For example, considering the following entity: + * 
+ * {@code
+ * public class FactTable {
+ *
+ *     {@literal @}MetricAggregation(sum.class)
+ *     Long sessions
+ *
+ *     {@literal @}MetricAggregation(sum.class)
+ *     Long timeSpent
+ *
+ *     {@literal @}MetricComputation(expression = "timeSpent / sessions")
+ *     Float timeSpentPerSession
+ *
+ *     {@literal @}MetricComputation(expression = "timeSpentPerSession / gameCount")
+ *     Float timeSpentPerGame
+ * }
+ * }
+ *
+ * During {@link Table} construction, {@code timeSpentPerSession} the provided expression will be substituted with + * {@code timeSpent / sessions}. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface MetricComputation { + + /** + * The custom metric expression that represents this metric computation logic. + * + * @return metric formula + */ + String expression(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java new file mode 100644 index 0000000000..0a9e84ea93 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; +import java.util.TimeZone; + +/** + * Indicates that the annotated entity field is a temporal field and is backed by a temporal column in persistent + * storage. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface Temporal { + + /** + * The set of time grains supported by this time dimension. + * + * @return one or more time gains. + */ + TimeGrainDefinition[] grains() default { @TimeGrainDefinition(grain = TimeGrain.DAY, expression = "") }; + + /** + * The timezone in {@link String} of the column. + *

+ * The String format can be expressed by + *

    + *
  • an abbreviation such as "PST", or + *
  • a full name such as "America/Los_Angeles", or + *
  • a custom ID such as "GMT-8:00" + *
+ * The timezone will be parsed using {@link TimeZone#getTimeZone(String)}. + * + * @return data timezone + */ + String timeZone() default "UTC"; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java new file mode 100644 index 0000000000..6dac86360a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java @@ -0,0 +1,29 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +/** + * A time grain that a time based dimension can be converted to. + */ +public @interface TimeGrainDefinition { + + /** + * The unit into which temporal column can be divided. + * + * @return One of the supported time grains of a persistent storage column + */ + TimeGrain grain() default TimeGrain.DAY; + + /** + * Optional expression used by the QueryEngine to represent the grain natively. + * The value is query engine specific. + * + * @return An expression which defines the grain and is meaningful to the Query Engine. + */ + String expression() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java new file mode 100644 index 0000000000..103df56c34 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraints.java @@ -0,0 +1,111 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import com.yahoo.elide.core.filter.expression.FilterExpression; + +import lombok.Getter; + +import java.util.Objects; + +/** + * {@link FilterConstraints} is an auxiliary class for {@link SplitFilterExpressionVisitor} that wraps a {@code WHERE} + * filter expression and {@code HAVING} filter expression. + *

+ * {@link FilterConstraints} is thread-safe and can be accessed by multiple threads. + */ +public class FilterConstraints { + + /** + * Creates a new {@link FilterConstraints} instance that wraps a specified {@code HAVING} filter expression only. + * + * @param havingExpression A pure {@code HAVING} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code HAVING} filter expression is {@code null} + */ + public static FilterConstraints pureHaving(FilterExpression havingExpression) { + return new FilterConstraints( + null, + Objects.requireNonNull(havingExpression, "havingExpression") + ); + } + + /** + * Creates a new {@link FilterConstraints} instance that wraps a specified {@code WHERE} filter expression only. + * + * @param whereExpression A pure {@code WHERE} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code WHERE} filter expression is {@code null} + */ + public static FilterConstraints pureWhere(FilterExpression whereExpression) { + return new FilterConstraints( + Objects.requireNonNull(whereExpression, "whereExpression"), + null + ); + } + + /** + * Creates a new {@link FilterConstraints} instance that wraps a pair of specified {@code WHERE} filter expression + * and {@code HAVING} filter expression. + * + * @param whereExpression A pure {@code HAVING} filter expression + * @param havingExpression A pure {@code WHERE} filter expression + * + * @return a new instance of {@link FilterConstraints} + * + * @throws NullPointerException if the provided {@code WHERE} or {@code HAVING} filter expression is {@code null} + */ + public static FilterConstraints withWhereAndHaving( + FilterExpression whereExpression, + FilterExpression havingExpression + ) { + return new FilterConstraints( + Objects.requireNonNull(whereExpression, "whereExpression"), + Objects.requireNonNull(havingExpression, "havingExpression") + ); + } + + @Getter + private final FilterExpression whereExpression; + + @Getter + private final FilterExpression havingExpression; + + /** + * Private constructor. + * + * @param whereExpression + * @param havingExpression + */ + private FilterConstraints(FilterExpression whereExpression, FilterExpression havingExpression) { + this.whereExpression = whereExpression; + this.havingExpression = havingExpression; + } + + /** + * Returns whether or not this {@link FilterConstraints} filter expression pair contains only a {@code HAVING} + * expression, i.e. no {@code WHERE} clause. + * + * @return {@code true} if there is {@code HAVING} expression only and not {@code WHERE} expression. + */ + public boolean isPureHaving() { + return getWhereExpression() == null && getHavingExpression() != null; + } + + /** + * Returns whether or not this {@link FilterConstraints} filter expression pair contains only a {@code WHERE} + * expression, i.e. no {@code HAVING} clause. + * + * @return {@code true} if there is {@code HAVING} expression only and not {@code WHERE} expression. + */ + public boolean isPureWhere() { + return getWhereExpression() != null && getHavingExpression() == null; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java new file mode 100644 index 0000000000..8ee3a78ec4 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitor.java @@ -0,0 +1,238 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpressionVisitor; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.parsers.expression.FilterExpressionNormalizationVisitor; + +import org.apache.commons.lang3.tuple.Pair; + +import lombok.AccessLevel; +import lombok.Getter; +import lombok.extern.slf4j.Slf4j; + +import java.util.Objects; + +/** + * {@link SplitFilterExpressionVisitor} splits the {@link FilterExpression} into a {@code WHERE} expression and a + * {@code Having} expression. + *

+ * {@link SplitFilterExpressionVisitor} is leveraged by the AggregationStore to construct the JPQL query. + * {@link FilterExpression} for AggregationDataStore must be split into those that apply to metric aggregations + * ({@code HAVING} clauses) and those that apply to dimensions ({@code WHERE} clauses), although only a single + * {@link FilterExpression} is passed to the datastore in each query. The split groups {@code HAVING} clauses and + * {@code WHERE} clauses separately. For example: + * 

+ * +-------------------+-----------------------------+------------------------+
+ * |    Expression     |             SQL             | WHERE-clause Promotion |
+ * +-------------------+-----------------------------+------------------------+
+ * | H1 AND W1 AND H2  | WHERE W1 HAVING (H1 AND H2) | No                     |
+ * | (H1 OR H2) AND W1 | WHERE W1 HAVING (H1 OR H2)  | No                     |
+ * | H1 OR W1          | HAVING (H1 OR W1)           | Yes                    |
+ * | (W1 AND H1) OR W2 | HAVING ((W1 AND H1) OR W2)  | Yes                    |
+ * +-------------------+-----------------------------+------------------------+
+ *
+ * Note that {@link SplitFilterExpressionVisitor} might incur more-than-expected network I/O in the case of WHERE-clause + * promotion. + *

+ * {@link SplitFilterExpressionVisitor} splits by storing {@code WHERE} and {@code HAVING} clauses in + * {@link Pair#getLeft() left} and {@link Pair#getRight() right} of a {@link Pair}, respectively. For example: + * 

+ * {@code
+ * Pair filterPair = filterExpression.accept(splitFilterExpressionVisitor);
+ *
+ * FilterExpression whereClauseFilter = filterPair.getLeft();
+ * FilterExpression havingClauseFilter = filterPair.getRight();
+ * }
+ *
+ * {@link SplitFilterExpressionVisitor} is thread-safe and can be accessed by multiple threads at the same time. + */ +@Slf4j +public class SplitFilterExpressionVisitor implements FilterExpressionVisitor { + + @Getter(value = AccessLevel.PRIVATE) + private final Table table; + + @Getter(value = AccessLevel.PRIVATE) + private final FilterExpressionNormalizationVisitor normalizationVisitor; + + /** + * Constructor. + * + * @param table Object that offers meta information about an entity field + * + * @throws NullPointerException if any one of the argument is {@code null} + */ + public SplitFilterExpressionVisitor(final Table table) { + this.table = Objects.requireNonNull(table, "table"); + this.normalizationVisitor = new FilterExpressionNormalizationVisitor(); + } + + @Override + public FilterConstraints visitPredicate(final FilterPredicate filterPredicate) { + return isHavingPredicate(filterPredicate) + ? FilterConstraints.pureHaving(filterPredicate) // this filterPredicate belongs to a HAVING clause + : FilterConstraints.pureWhere(filterPredicate); // this filterPredicate belongs to a WHERE clause + } + + @Override + public FilterConstraints visitAndExpression(final AndFilterExpression expression) { + /* + * Definition: + * C = condition + * pure-W = WHERE C + * pure-H = HAVING C + * mix-HW = WHERE C HAVING C' + * + * Given that L and R operands of an AndFilterExpression can only be one of "pure-H", "pure-W", or "mix-HW", + * then: + * + * pure-W1 AND pure-W2 = WHERE C1 AND WHERE C2 = WHERE (C1 AND C2) = pure-W + * pure-H1 AND pure-H2 = HAVING C1 AND HAVING C2 = HAVING (C1 AND C2) = pure-H + * + * pure-H1 AND pureW2 = HAVING C1 AND WHERE C2 = WHERE C2 HAVING C1 = mix-HW + * pure-W1 AND pureH2 = WHERE C1 HAVING C2 = mix-HW + * + * mix-HW1 AND pure-W2 = WHERE C1 HAVING C1' AND WHERE C2 = WHERE (C1 & C2) HAVING C1' = mix-HW + * mix-HW1 AND pure-H2 = WHERE C1 HAVING C1' AND HAVING C2 = WHERE C1 HAVING (C1' & C2) = mix-HW + * + * mix-HW1 AND mim-HW2 = WHERE C1 HAVING C1' AND WHERE C2 HAVING C2' = WHERE (C1 & C2) HAVING (C1' & C2') + * = mix-HW + */ + + FilterConstraints left = expression.getLeft().accept(this); + FilterConstraints right = expression.getRight().accept(this); + + if (left.isPureWhere() && right.isPureWhere()) { + // pure-W1 AND pure-W2 = WHERE (C1 & C2) = pure-W + return FilterConstraints.pureWhere( + new AndFilterExpression( + left.getWhereExpression(), + right.getWhereExpression() + ) + ); + } else if (left.isPureHaving() && right.isPureHaving()) { + // pure-H1 AND pure-H2 = HAVING (C1 AND C2) = pure-H + return FilterConstraints.pureHaving( + new AndFilterExpression( + left.getHavingExpression(), + right.getHavingExpression() + ) + ); + } else { + // all of the rests are mix-HW + return FilterConstraints.withWhereAndHaving( + AndFilterExpression.fromPair( + left.getWhereExpression(), + right.getWhereExpression() + ), + AndFilterExpression.fromPair( + left.getHavingExpression(), + right.getHavingExpression() + ) + ); + } + } + + @Override + public FilterConstraints visitOrExpression(final OrFilterExpression expression) { + /* + * Definition: + * C = condition + * pure-W = WHERE C + * pure-H = HAVING C + * mix-HW = WHERE C HAVING C' + * + * Given that L and R operands of an OrFilterExpression can only be one of "pure-H", "pure-W", or "mix-HW", + * then: + * + * pure-W1 OR pure-W2 = WHERE C1 OR WHERE C2 = WHERE (C1 OR C2) = pure-W + * pure-H1 OR pure-H2 = HAVING C1 OR HAVING C2 = HAVING (C1 OR C2) = pure-H + * + * pure-H1 OR pureW2 = HAVING C1 OR WHERE C2 = HAVING (C1 OR C2) = pure-H + * pure-W1 OR pureH2 = WHERE C1 OR HAVING C2 = HAVING (C1 OR C2) = pure-H + * + * mix-HW1 OR pure-W2 = (WHERE C1 HAVING C1') OR WHERE C2 = HAVING (C1 & C1' | C2) = pure-H + * mix-HW1 OR pure-H2 = (WHERE C1 HAVING C1') OR HAVING C2 = HAVING (C1 & C1' | C2) = pure-H + * + * mix-HW1 OR mim-HW2 = (WHERE C1 HAVING C1') OR (WHERE C2 HAVING C2') = HAVING ((C1 & C1') | (C2 & C2')) + * = pure-H + */ + + FilterConstraints left = expression.getLeft().accept(this); + FilterConstraints right = expression.getRight().accept(this); + + if (left.isPureWhere() && right.isPureWhere()) { + // pure-W1 OR pure-W2 = WHERE (C1 OR C2) = pure-W + return FilterConstraints.pureWhere( + OrFilterExpression.fromPair( + left.getWhereExpression(), + right.getWhereExpression() + ) + ); + } else { + // all of the rests are pure-H + return FilterConstraints.pureHaving( + OrFilterExpression.fromPair( + AndFilterExpression.fromPair( + left.getWhereExpression(), + left.getHavingExpression() + ), + AndFilterExpression.fromPair( + right.getWhereExpression(), + right.getHavingExpression() + ) + ) + ); + } + } + + @Override + public FilterConstraints visitNotExpression(NotFilterExpression expression) { + FilterExpression normalized = getNormalizationVisitor().visitNotExpression(expression); + + if (normalized instanceof AndFilterExpression) { + return visitAndExpression((AndFilterExpression) normalized); + } else if (normalized instanceof OrFilterExpression) { + return visitOrExpression((OrFilterExpression) normalized); + } else if (normalized instanceof NotFilterExpression) { + FilterConstraints negatedConstraint = visitNotExpression((NotFilterExpression) normalized); + + if (negatedConstraint.isPureWhere()) { + return FilterConstraints.pureWhere(new NotFilterExpression(negatedConstraint.getWhereExpression())); + } else { + // It is not possible to have a mixed where/having for a NotFilterExpression after normalization + // so this must be a pure HAVING + return FilterConstraints.pureHaving(new NotFilterExpression(negatedConstraint.getHavingExpression())); + } + } else { + return visitPredicate((FilterPredicate) normalized); + } + } + + /** + * Returns whether or not a {@link FilterPredicate} corresponds to a {@code HAVING} clause in JPQL query. + *

+ * A {@link FilterPredicate} corresponds to a {@code HAVING} clause iff the predicate field has + * {@link MetricAggregation} or MetricComputation annotation on it. + * + * @param filterPredicate The terminal filter expression to check for + * + * @return {@code true} if the {@link FilterPredicate} is a HAVING clause + */ + private boolean isHavingPredicate(final FilterPredicate filterPredicate) { + String fieldName = filterPredicate.getField(); + + return getTable().isMetric(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java new file mode 100644 index 0000000000..eaeb3ab5fa --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -0,0 +1,183 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; +import com.yahoo.elide.core.exceptions.DuplicateMappingException; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.DataType; +import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; +import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.utils.ClassScanner; + +import org.hibernate.annotations.Subselect; + +import java.util.HashMap; +import java.util.Set; +import java.util.stream.Collectors; + +/** + * MetaDataStore is a in-memory data store that manage data models for an {@link AggregationDataStore}. + */ +public class MetaDataStore extends HashMapDataStore { + public static final Package META_DATA_PACKAGE = + Package.getPackage("com.yahoo.elide.datastores.aggregation.metadata.models"); + + private static final Class[] METADATA_STORE_ANNOTATIONS = { + FromTable.class, FromSubquery.class, Subselect.class, javax.persistence.Table.class}; + + public MetaDataStore() { + super(META_DATA_PACKAGE); + + this.dictionary = new EntityDictionary(new HashMap<>()); + + ClassScanner.getAllClasses(Table.class.getPackage().getName()).forEach(cls -> dictionary.bindEntity(cls)); + + Set> modelsToBind = ClassScanner.getAnnotatedClasses(METADATA_STORE_ANNOTATIONS); + + // bind data models in the package + modelsToBind.forEach(modelClass -> { + dictionary.bindEntity(modelClass); + }); + + // resolve meta data from the bound models + modelsToBind.forEach(modelClass -> { + addTable(isAnalyticView(modelClass) + ? new AnalyticView(modelClass, dictionary) + : new Table(modelClass, dictionary)); + }); + } + + @Override + public void populateEntityDictionary(EntityDictionary dictionary) { + ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()).stream().forEach(cls -> { + dictionary.bindEntity(cls); + }); + } + + /** + * Add a table metadata object. + * + * @param table table metadata + */ + private void addTable(Table table) { + addMetaData(table); + table.getColumns().forEach(this::addColumn); + } + + /** + * Add a column metadata object. + * + * @param column column metadata + */ + private void addColumn(Column column) { + addMetaData(column); + addDataType(column.getDataType()); + + if (column instanceof TimeDimension) { + ((TimeDimension) column).getSupportedGrains().forEach(this::addTimeDimensionGrain); + } else if (column instanceof Metric) { + addMetricFunction(((Metric) column).getMetricFunction()); + } + } + + /** + * Add a metric function metadata object. + * + * @param metricFunction metric function metadata + */ + private void addMetricFunction(MetricFunction metricFunction) { + addMetaData(metricFunction); + metricFunction.getArguments().forEach(this::addFunctionArgument); + } + + /** + * Add a datatype metadata object. + * + * @param dataType datatype metadata + */ + private void addDataType(DataType dataType) { + addMetaData(dataType); + } + + /** + * Add a function argument metadata object. + * + * @param functionArgument function argument metadata + */ + private void addFunctionArgument(FunctionArgument functionArgument) { + addMetaData(functionArgument); + } + + /** + * Add a time dimension grain metadata object. + * + * @param timeDimensionGrain time dimension grain metadata + */ + private void addTimeDimensionGrain(TimeDimensionGrain timeDimensionGrain) { + addMetaData(timeDimensionGrain); + } + + /** + * Add a meta data object into this data store, check for duplication. + * + * @param object a meta data object + */ + private void addMetaData(Object object) { + Class cls = dictionary.lookupBoundClass(object.getClass()); + String id = dictionary.getId(object); + + if (dataStore.get(cls).containsKey(id)) { + if (!dataStore.get(cls).get(id).equals(object)) { + throw new DuplicateMappingException("Duplicated " + cls.getSimpleName() + " metadata " + id); + } + } else { + dataStore.get(cls).put(id, object); + } + } + + public Set getMetaData(Class cls) { + return dataStore.get(cls).values().stream().map(cls::cast).collect(Collectors.toSet()); + } + + /** + * Returns whether or not an entity field is a metric field. + *

+ * A field is a metric field iff that field is annotated by at least one of + *

    + *
  1. {@link MetricAggregation} + *
+ * + * @param dictionary entity dictionary in current Elide instance + * @param cls entity class + * @param fieldName The entity field + * + * @return {@code true} if the field is a metric field + */ + public static boolean isMetricField(EntityDictionary dictionary, Class cls, String fieldName) { + return dictionary.attributeOrRelationAnnotationExists(cls, fieldName, MetricAggregation.class); + } + + /** + * Returns whether an entity class is analytic view. + * + * @param cls entity class + * @return True if {@link FromTable} or {@link FromSubquery} is presented. + */ + private static boolean isAnalyticView(Class cls) { + return cls.isAnnotationPresent(FromTable.class) || cls.isAnnotationPresent(FromSubquery.class); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java new file mode 100644 index 0000000000..828f00b523 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java @@ -0,0 +1,15 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Aggregation functions. + */ +public enum Aggregation { + SUM, + MIN, + MAX; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java new file mode 100644 index 0000000000..1c164e8025 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Format.java @@ -0,0 +1,14 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Format of a value field, e.g. decimal for numbers + */ +public enum Format { + NONE +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java new file mode 100644 index 0000000000..92065a06d3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java @@ -0,0 +1,13 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Tag attached to fields. + */ +public enum Tag { + DISPLAY +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java new file mode 100644 index 0000000000..6df6643e9e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Actual value type of a data type. + */ +public enum ValueType { + DATE, + NUMBER, + TEXT, + COORDINATE, + BOOLEAN, + RELATIONSHIP, + ID +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java new file mode 100644 index 0000000000..b9dd81dcab --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/metric/MetricFunctionInvocation.java @@ -0,0 +1,69 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.metric; + +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.request.Argument; + +import com.google.common.base.Functions; + +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * An invoked metric function instance applied on an aggregated field with provided arguments to project the result + * as the alias. + */ +public interface MetricFunctionInvocation { + /** + * Get all arguments provided for this metric function. + * + * @return request arguments + */ + List getArguments(); + + /** + * Get a name-argument map contains all arguments. + * + * @return argument map + */ + default Map getArgumentMap() { + return getArguments().stream() + .collect(Collectors.toMap(Argument::getName, Functions.identity())); + } + + /** + * Get argument for a specific name. + * + * @param argumentName argument name + * @return an argument + */ + Argument getArgument(String argumentName); + + /** + * Get invoked metric function. + * + * @return metric function + */ + MetricFunction getFunction(); + + /** + * Get full expression with provided arguments. + * + * @return function expression + */ + default String getFunctionExpression() { + return getFunction().constructExpression(getArgumentMap()); + } + + /** + * Get alias of this invocation. + * + * @return alias + */ + String getAlias(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java new file mode 100644 index 0000000000..65b03048bb --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java @@ -0,0 +1,48 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; + +import lombok.Data; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import java.util.Set; +import java.util.stream.Collectors; +import javax.persistence.OneToMany; + +/** + * AnalyticViews are logical tables that support aggregation functionality, but don't support join or relationship. + */ +@EqualsAndHashCode(callSuper = true) +@Include(rootLevel = true, type = "analyticView") +@Data +public class AnalyticView extends Table { + + @OneToMany + @ToString.Exclude + private Set metrics; + + @OneToMany + @ToString.Exclude + private Set dimensions; + + public AnalyticView(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + + metrics = getColumns().stream() + .filter(col -> col instanceof Metric) + .map(Metric.class::cast) + .collect(Collectors.toSet()); + + dimensions = getColumns().stream() + .filter(col -> !(col instanceof Metric)) + .map(Dimension.class::cast) + .collect(Collectors.toSet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java new file mode 100644 index 0000000000..8bc9c02baf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -0,0 +1,87 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.metadata.enums.Tag; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.Data; +import lombok.ToString; + +import java.util.Date; +import java.util.HashSet; +import java.util.Locale; +import java.util.Set; +import javax.persistence.Id; +import javax.persistence.ManyToOne; + +/** + * Column is the super class of a field in a table, it can be either dimension or metric. + */ +@Include(type = "column") +@Data +@ToString +public abstract class Column { + @Id + private String id; + + private String name; + + private String longName; + + private String tableName; + + private String description; + + private String category; + + @ManyToOne + private DataType dataType; + + @ToString.Exclude + private Set columnTags; + + protected Column(Class tableClass, String fieldName, EntityDictionary dictionary) { + this.tableName = dictionary.getJsonAliasFor(tableClass); + this.id = tableName + "." + fieldName; + this.name = fieldName; + this.columnTags = new HashSet<>(); + + Meta meta = dictionary.getAttributeOrRelationAnnotation(tableClass, Meta.class, fieldName); + if (meta != null) { + this.longName = meta.longName(); + this.description = meta.description(); + } + + dataType = getDataType(tableClass, fieldName, dictionary); + if (dataType == null) { + throw new IllegalArgumentException("Unknown data type for " + this.id); + } + } + + public static DataType getDataType(Class tableClass, String fieldName, EntityDictionary dictionary) { + String tableName = dictionary.getJsonAliasFor(tableClass); + DataType dataType; + if (dictionary.isRelation(tableClass, fieldName)) { + Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); + dataType = new RelationshipType(dictionary.getJsonAliasFor(relationshipClass)); + } else { + Class fieldClass = dictionary.getType(tableClass, fieldName); + + if (fieldName.equals(dictionary.getIdFieldName(tableClass))) { + dataType = new DataType(tableName + "." + fieldName, ValueType.ID); + } else if (Date.class.isAssignableFrom(fieldClass)) { + dataType = new DataType(fieldClass.getSimpleName().toLowerCase(Locale.ENGLISH), ValueType.DATE); + } else { + dataType = DataType.getScalarType(fieldClass); + } + } + return dataType; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java new file mode 100644 index 0000000000..de8a0d2b81 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java @@ -0,0 +1,58 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.BOOLEAN; +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.NUMBER; +import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.TEXT; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.ToString; + +import java.math.BigDecimal; +import java.util.HashMap; +import java.util.Map; +import javax.persistence.Id; + +/** + * Data type of a column. + */ +@Include(rootLevel = true, type = "dataType") +@Data +@AllArgsConstructor +@ToString +public class DataType { + private static final Map, DataType> SCALAR_TYPES = new HashMap, DataType>() {{ + put(short.class, new DataType("p_short", NUMBER)); + put(Short.class, new DataType("short", NUMBER)); + put(int.class, new DataType("p_int", NUMBER)); + put(Integer.class, new DataType("int", NUMBER)); + put(long.class, new DataType("p_bigint", NUMBER)); + put(Long.class, new DataType("bigint", NUMBER)); + put(BigDecimal.class, new DataType("bigDecimal", NUMBER)); + put(float.class, new DataType("p_float", NUMBER)); + put(Float.class, new DataType("float", NUMBER)); + put(double.class, new DataType("p_double", NUMBER)); + put(Double.class, new DataType("double", NUMBER)); + put(boolean.class, new DataType("p_boolean", BOOLEAN)); + put(Boolean.class, new DataType("boolean", BOOLEAN)); + put(char.class, new DataType("p_char", TEXT)); + put(String.class, new DataType("string", TEXT)); + }}; + + @Id + private String name; + + private ValueType valueType; + + public static DataType getScalarType(Class valueClass) { + return SCALAR_TYPES.get(valueClass); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java new file mode 100644 index 0000000000..3ed5f7f56a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java @@ -0,0 +1,24 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +/** + * Regular field in tables, can be grouped by if the table is an AnalyticView. + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "dimension") +@Data +public class Dimension extends Column { + public Dimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java new file mode 100644 index 0000000000..d7b1de386b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java @@ -0,0 +1,39 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; + +import lombok.Data; +import lombok.ToString; + +import javax.persistence.Id; +import javax.persistence.ManyToOne; + +/** + * Arguments that can be provided into a metric function. + */ +@Include(type = "functionArgument") +@Data +@ToString +public class FunctionArgument { + @Id + private String id; + + private String name; + + private String description; + + @ManyToOne + private DataType dataType; + + public FunctionArgument(String functionName, FunctionArgument argument) { + this.id = functionName + "." + argument.getName(); + this.name = argument.getName(); + this.description = argument.getDescription(); + this.dataType = argument.getDataType(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java new file mode 100644 index 0000000000..8dcfaf3d16 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java @@ -0,0 +1,50 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.metadata.enums.Format; + +import lombok.Data; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import javax.persistence.ManyToOne; + +/** + * Special column for AnalyticView which supports aggregation. + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "metric") +@Data +public class Metric extends Column { + private Format defaultFormat; + + @ManyToOne + @ToString.Exclude + private MetricFunction metricFunction; + + public Metric(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + MetricAggregation metric = dictionary.getAttributeOrRelationAnnotation( + tableClass, + MetricAggregation.class, + fieldName); + + try { + this.metricFunction = metric.function().newInstance(); + metricFunction.setName(getId() + "[" + metricFunction.getName() + "]"); + metricFunction.setExpression(String.format( + metricFunction.getExpression(), + dictionary.getAnnotatedColumnName(tableClass, fieldName))); + } catch (InstantiationException | IllegalAccessException e) { + throw new IllegalArgumentException("Can't initialize function for metric " + getId()); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java new file mode 100644 index 0000000000..6c3b06ed6b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/MetricFunction.java @@ -0,0 +1,114 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.request.Argument; + +import lombok.AllArgsConstructor; +import lombok.Data; +import lombok.ToString; + +import java.util.ArrayList; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.persistence.Id; +import javax.persistence.OneToMany; + +/** + * Functions used to compute metrics. + */ +@Include(type = "metricFunction") +@Data +@ToString +@AllArgsConstructor +public class MetricFunction { + @Id + private String name; + + private String longName; + + private String description; + + private String expression; + + @OneToMany + private Set arguments; + + protected MetricFunctionInvocation invoke(Map arguments, String alias) { + final MetricFunction function = this; + return new MetricFunctionInvocation() { + @Override + public List getArguments() { + return new ArrayList<>(arguments.values()); + } + + @Override + public Argument getArgument(String argumentName) { + return arguments.get(argumentName); + } + + @Override + public MetricFunction getFunction() { + return function; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + + /** + * Construct full metric expression using arguments. + * + * @param arguments provided arguments + * @return FUNCTION(field1, field2, ..., arg1, arg2, ...) + */ + public String constructExpression(Map arguments) { + return getExpression(); + } + + /** + * Get all required argument names for this metric function. + * + * @return all argument names + */ + private Set getArgumentNames() { + return getArguments().stream().map(FunctionArgument::getName).collect(Collectors.toSet()); + } + + /** + * Invoke this metric function with arguments, an aggregated field and projection alias. + * + * @param arguments arguments provided in the request + * @param alias result alias + * @return an invoked metric function + */ + public final MetricFunctionInvocation invoke(Set arguments, String alias) { + Set requiredArguments = getArgumentNames(); + Set providedArguments = arguments.stream() + .map(Argument::getName) + .collect(Collectors.toSet()); + + if (!requiredArguments.equals(providedArguments)) { + throw new InvalidPredicateException( + "Provided arguments doesn't match requirement for function " + getName() + "."); + } + + // map arguments to their actual name + Map resolvedArguments = arguments.stream() + .collect(Collectors.toMap(Argument::getName, Function.identity())); + + return invoke(resolvedArguments, alias); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java new file mode 100644 index 0000000000..c0e8f49587 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java @@ -0,0 +1,22 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +/** + * Special data type that represents a relationship between tables. + */ +@Data +@EqualsAndHashCode(callSuper = true) +public class RelationshipType extends DataType { + public RelationshipType(String name) { + super(name, ValueType.RELATIONSHIP); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java new file mode 100644 index 0000000000..b31af3e474 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -0,0 +1,148 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; + +import lombok.Data; +import lombok.ToString; + +import java.util.Set; +import java.util.stream.Collectors; +import javax.persistence.Id; +import javax.persistence.OneToMany; +import javax.persistence.Transient; + +/** + * Super class of all logical or physical tables. + */ +@Include(rootLevel = true, type = "table") +@Data +@ToString +public class Table { + @Transient + private Class cls; + + @Id + private String name; + + private String longName; + + private String description; + + private String category; + + private CardinalitySize cardinality; + + @OneToMany + @ToString.Exclude + private Set columns; + + public Table(Class cls, EntityDictionary dictionary) { + if (!dictionary.getBindings().contains(cls)) { + throw new IllegalArgumentException( + String.format("Table class {%s} is not defined in dictionary.", cls)); + } + + this.cls = cls; + this.name = dictionary.getJsonAliasFor(cls); + + this.columns = resolveColumns(cls, dictionary); + + Meta meta = cls.getAnnotation(Meta.class); + if (meta != null) { + this.longName = meta.longName(); + this.description = meta.description(); + } + + Cardinality cardinality = dictionary.getAnnotation(cls, Cardinality.class); + if (cardinality != null) { + this.cardinality = cardinality.size(); + } + } + + /** + * Add all columns of this table. + * + * @param cls table class + * @param dictionary dictionary contains the table class + * @return all resolved column metadata + */ + private static Set resolveColumns(Class cls, EntityDictionary dictionary) { + Set fields = dictionary.getAllFields(cls).stream() + .filter((field) -> Column.getDataType(cls, field, dictionary) != null) + .map(field -> { + if (isMetricField(dictionary, cls, field)) { + return new Metric(cls, field, dictionary); + } else if (dictionary.attributeOrRelationAnnotationExists(cls, field, Temporal.class)) { + return new TimeDimension(cls, field, dictionary); + } else { + return new Dimension(cls, field, dictionary); + } + }) + .collect(Collectors.toSet()); + + // add id field if exists + if (dictionary.getIdFieldName(cls) != null) { + fields.add(new Dimension(cls, dictionary.getIdFieldName(cls), dictionary)); + } + + return fields; + } + + /** + * Get all columns of a specific class, can be {@link Metric}, {@link TimeDimension} or {@link Dimension}. + * + * @param cls metadata class + * @param metadata class + * @return columns as requested type if found + */ + public Set getColumns(Class cls) { + return columns.stream() + .filter(col -> cls.isAssignableFrom(col.getClass())) + .map(cls::cast) + .collect(Collectors.toSet()); + } + + /** + * Get a field column as a specific class, can be {@link Metric}, {@link TimeDimension} or {@link Dimension}. + * + * @param cls metadata class + * @param fieldName logical column name + * @param metadata class + * @return column as requested type if found + */ + private T getColumn(Class cls, String fieldName) { + return columns.stream() + .filter(col -> cls.isAssignableFrom(col.getClass()) && (col.getName().equals(fieldName))) + .map(cls::cast) + .findFirst() + .orElse(null); + } + + public Metric getMetric(String fieldName) { + return getColumn(Metric.class, fieldName); + } + + public Dimension getDimension(String fieldName) { + return getColumn(Dimension.class, fieldName); + } + + public TimeDimension getTimeDimension(String fieldName) { + return getColumn(TimeDimension.class, fieldName); + } + + public boolean isMetric(String fieldName) { + return getMetric(fieldName) != null; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java new file mode 100644 index 0000000000..22b06e3fd8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java @@ -0,0 +1,44 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Arrays; +import java.util.LinkedHashSet; +import java.util.Set; +import java.util.TimeZone; +import java.util.stream.Collectors; +import javax.persistence.ManyToMany; + +/** + * TimeDimension is a dimension that represents time value. + * This type of dimension can be used to support more specific aggregation logic e.g. DAILY/MONTHLY aggregation + */ +@EqualsAndHashCode(callSuper = true) +@Include(type = "timeDimension") +@Data +public class TimeDimension extends Dimension { + @ManyToMany + Set supportedGrains; + + private TimeZone timezone; + + public TimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + Temporal temporal = dictionary.getAttributeOrRelationAnnotation(tableClass, Temporal.class, fieldName); + + this.supportedGrains = Arrays.stream(temporal.grains()) + .map(grain -> new TimeDimensionGrain(getId(), grain)) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java new file mode 100644 index 0000000000..7ff87e8380 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java @@ -0,0 +1,35 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.models; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.Data; + +import java.util.Locale; +import javax.persistence.Id; + +/** + * Defines how to extract a time dimension for a specific grain from a table. + */ +@Include(type = "timeDimensionGrain") +@Data +public class TimeDimensionGrain { + @Id + private String id; + + private TimeGrain grain; + + private String expression; + + public TimeDimensionGrain(String fieldName, TimeGrainDefinition definition) { + this.id = fieldName + "." + definition.grain().name().toLowerCase(Locale.ENGLISH); + this.grain = definition.grain(); + this.expression = definition.expression(); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java new file mode 100644 index 0000000000..2228751f72 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java @@ -0,0 +1,91 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.io.Serializable; +import java.util.TimeZone; + +/** + * Represents a projected column as an alias in a query. + */ +public interface ColumnProjection extends Serializable { + /** + * Get the projected column. + * + * @return column + */ + Column getColumn(); + + /** + * Get the projection alias. + * + * @return alias + */ + String getAlias(); + + /** + * Project a dimension as alias. + * + * @param dimension dimension column + * @param alias alias + * @return a projection represents that "dimension AS alias" + */ + static ColumnProjection toProjection(Dimension dimension, String alias) { + return new ColumnProjection() { + @Override + public Dimension getColumn() { + return dimension; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + + /** + * Project a time dimension as alias with specific time grain. + * + * @param dimension time dimension column + * @param grain projected time grain + * @param alias alias + * @return a projection represents that "grain(dimension) AS alias" + */ + static TimeDimensionProjection toProjection(TimeDimension dimension, TimeGrain grain, String alias) { + // TODO: get time zone from the request + if (dimension.getSupportedGrains().stream().anyMatch(g -> g.getGrain().equals(grain))) { + return new TimeDimensionProjection() { + @Override + public TimeGrain getGrain() { + return grain; + } + + @Override + public TimeZone getTimeZone() { + return null; + } + + @Override + public TimeDimension getTimeDimension() { + return dimension; + } + + @Override + public String getAlias() { + return alias; + } + }; + } + throw new InvalidValueException(dimension.getId() + " doesn't support grain " + grain); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java new file mode 100644 index 0000000000..9d6687ca79 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java @@ -0,0 +1,57 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; + +import lombok.Builder; +import lombok.Data; +import lombok.Singular; + +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Set; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * A {@link Query} is an object representing a query executed by {@link QueryEngine}. + */ +@Data +@Builder +public class Query { + private final AnalyticView analyticView; + + @Singular + private final List metrics; + + @Singular + private final Set groupByDimensions; + + @Singular + private final Set timeDimensions; + + private final FilterExpression whereFilter; + private final FilterExpression havingFilter; + private final Sorting sorting; + private final Pagination pagination; + private final RequestScope scope; + + /** + * Returns all the dimensions regardless of type. + * @return All the dimensions. + */ + public Set getDimensions() { + return Stream.concat(getGroupByDimensions().stream(), getTimeDimensions().stream()) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java new file mode 100644 index 0000000000..bab8385207 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java @@ -0,0 +1,86 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.query; + +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.util.TimeZone; + +/** + * Represents a requested time dimension in a query. + */ +public interface TimeDimensionProjection extends ColumnProjection { + /** + * Get the projected time dimension. + * + * @return time dimension + */ + TimeDimension getTimeDimension(); + + /** + * The time dimension is the projected column. + * + * @return project column + */ + @Override + default Column getColumn() { + return getTimeDimension(); + } + + /** + * Get the requested time grain. + * + * @return time grain + */ + TimeGrain getGrain(); + + /** + * Get the requested time zone. + * + * @return time zone + */ + TimeZone getTimeZone(); + + /** + * Convert this projection to a new time grain. + * + * @param newGrain new time grain + * @return a new projection + */ + default TimeDimensionProjection toTimeGrain(TimeGrain newGrain) { + if (getTimeDimension().getSupportedGrains().stream() + .noneMatch(supportedGrain -> supportedGrain.getGrain().equals(newGrain))) { + throw new InvalidValueException(getTimeDimension().getId() + " doesn't support grain " + newGrain); + } + + TimeDimensionProjection projection = this; + return new TimeDimensionProjection() { + @Override + public TimeDimension getTimeDimension() { + return projection.getTimeDimension(); + } + + @Override + public TimeGrain getGrain() { + return newGrain; + } + + @Override + public TimeZone getTimeZone() { + return projection.getTimeZone(); + } + + @Override + public String getAlias() { + return projection.getAlias(); + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java new file mode 100644 index 0000000000..b8e2151a1c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java @@ -0,0 +1,203 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.RelationshipType; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import com.google.common.base.Preconditions; +import org.apache.commons.lang3.mutable.MutableInt; + +import lombok.AccessLevel; +import lombok.Getter; + +import java.util.ArrayList; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * {@link AbstractEntityHydrator} hydrates the entity loaded by {@link QueryEngine#executeQuery(Query)}. + *

+ * {@link AbstractEntityHydrator} is not thread-safe and should be accessed by only 1 thread in this application, + * because it uses {@link StitchList}. See {@link StitchList} for more details. + */ +public abstract class AbstractEntityHydrator { + + @Getter(AccessLevel.PROTECTED) + private final EntityDictionary entityDictionary; + + @Getter(AccessLevel.PRIVATE) + private final StitchList stitchList; + + @Getter(AccessLevel.PROTECTED) + private final List> results = new ArrayList<>(); + + @Getter(AccessLevel.PRIVATE) + private final Query query; + + /** + * Constructor. + * + * @param results The loaded objects from {@link QueryEngine#executeQuery(Query)} + * @param query The query passed to {@link QueryEngine#executeQuery(Query)} to load the objects + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + */ + public AbstractEntityHydrator(List results, Query query, EntityDictionary entityDictionary) { + this.stitchList = new StitchList(entityDictionary); + this.query = query; + this.entityDictionary = entityDictionary; + + //Get all the projections from the client query. + List projections = this.query.getMetrics().stream() + .map(MetricFunctionInvocation::getAlias) + .collect(Collectors.toList()); + + projections.addAll(this.query.getDimensions().stream() + .map(ColumnProjection::getAlias) + .collect(Collectors.toList())); + + results.forEach(result -> { + Map row = new HashMap<>(); + + Object[] resultValues = result instanceof Object[] ? (Object[]) result : new Object[] { result }; + + Preconditions.checkArgument(projections.size() == resultValues.length); + + for (int idx = 0; idx < resultValues.length; idx++) { + Object value = resultValues[idx]; + String fieldName = projections.get(idx); + row.put(fieldName, value); + } + + this.results.add(row); + }); + } + + /** + * Loads a map of relationship object ID to relationship object instance. + *

+ * Note the relationship cannot be toMany. This method will be invoked for every relationship field of the + * requested entity. Its implementation should return the result of the following query + *

+ * Given a relationship with type {@code relationshipType} in an entity, loads all relationship + * objects whose foreign keys are one of the specified list, {@code joinFieldIds}. + *

+ * For example, when the relationship is loaded from SQL and we have the following example identity: + * 

+     * public class PlayerStats {
+     *     private String id;
+     *     private Country country;
+     *
+     *     @OneToOne
+     *     @JoinColumn(name = "country_id")
+     *     public Country getCountry() {
+     *         return country;
+     *     }
+     * }
+     *
+ * In this case {@code relationshipType = Country.class}. If {@code country} is + * requested in {@code PlayerStats} query and 3 stats, for example, are found in database whose country ID's are + * {@code joinFieldIds = [840, 344, 840]}, then this method should effectively run the following query (JPQL as + * example) + * 
+     * {@code
+     *     SELECT e FROM country_table e WHERE country_id IN (840, 344);
+     * }
+     *
+ * and returns the map of [840: Country(id:840), 344: Country(id:344)] + * + * @param relationshipType The type of relationship + * @param joinFieldIds The specified list of join ID's against the relationship + * + * @return a list of hydrating values + */ + protected abstract Map getRelationshipValues( + Class relationshipType, + List joinFieldIds + ); + + public Iterable hydrate() { + //Coerce the results into entity objects. + MutableInt counter = new MutableInt(0); + + List queryResults = getResults().stream() + .map((result) -> coerceObjectToEntity(result, counter)) + .collect(Collectors.toList()); + + if (getStitchList().shouldStitch()) { + // relationship is requested, stitch relationship then + populateObjectLookupTable(); + getStitchList().stitch(); + } + + return queryResults; + } + + /** + * Coerces results from a {@link Query} into an Object. + * + * @param result a fieldName-value map + * @param counter Monotonically increasing number to generate IDs. + * @return A hydrated entity object. + */ + protected Object coerceObjectToEntity(Map result, MutableInt counter) { + Class entityClass = query.getAnalyticView().getCls(); + + //Construct the object. + Object entityInstance; + try { + entityInstance = entityClass.newInstance(); + } catch (InstantiationException | IllegalAccessException e) { + throw new IllegalStateException(e); + } + + result.forEach((fieldName, value) -> { + Dimension dim = query.getAnalyticView().getDimension(fieldName); + + if (dim != null && dim.getDataType() instanceof RelationshipType) { + getStitchList().todo(entityInstance, fieldName, value); // We don't hydrate relationships here. + } else { + getEntityDictionary().setValue(entityInstance, fieldName, value); + } + }); + + //Set the ID (it must be coerced from an integer) + getEntityDictionary().setValue( + entityInstance, + getEntityDictionary().getIdFieldName(entityClass), + counter.getAndIncrement() + ); + + return entityInstance; + } + + /** + * Foe each requested relationship, run a single query to load all relationship objects whose ID's are involved in + * the request. + */ + private void populateObjectLookupTable() { + // mapping: relationship field name -> join ID's + Map> hydrationIdsByRelationship = getStitchList().getHydrationMapping(); + + // hydrate each relationship + for (Map.Entry> entry : hydrationIdsByRelationship.entrySet()) { + String joinField = entry.getKey(); + List joinFieldIds = entry.getValue(); + Class relationshipType = getEntityDictionary().getParameterizedType( + getQuery().getAnalyticView().getCls(), + joinField); + + getStitchList().populateLookup(relationshipType, getRelationshipValues(relationshipType, joinFieldIds)); + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java new file mode 100644 index 0000000000..7dd9a95b1a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/StitchList.java @@ -0,0 +1,162 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.utils.coerce.CoerceUtil; +import lombok.AccessLevel; +import lombok.Data; +import lombok.Getter; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +/** + * An auxiliary class for {@link AbstractEntityHydrator} and is responsible for setting relationship values of an entity + * instance. + *

+ * {@link StitchList} should not be subclassed. + */ +public final class StitchList { + /** + * Maps an relationship entity class to a map of object ID to object instance. + *

+ * For example, [Country.class: [340: Country(id:340), 100: Country(id:100)]] + */ + @Getter(AccessLevel.PRIVATE) + private final Map, Map> objectLookups; + + /** + * List of relationships to hydrate. + */ + @Getter(AccessLevel.PRIVATE) + private final List todoList; + + @Getter(AccessLevel.PRIVATE) + private final EntityDictionary entityDictionary; + + /** + * A representation of an TODO item in a {@link StitchList}. + */ + @Data + public static class Todo { + private final Object entityInstance; + private final String relationshipName; + private final Object foreignKey; + } + + /** + * Constructor. + * + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + */ + public StitchList(EntityDictionary entityDictionary) { + this.objectLookups = new HashMap<>(); + this.todoList = new ArrayList<>(); + this.entityDictionary = entityDictionary; + } + + /** + * Returns whether or not the entity instances in this {@link StitchList} have relationships that are unset. + * + * @return {@code true} if the entity instances in this {@link StitchList} should be further hydrated because they + * have one or more relationship fields. + */ + public boolean shouldStitch() { + return !getTodoList().isEmpty(); + } + + /** + * Enqueues an entity instance which will be further hydrated on one of its relationship fields later. + * + * @param entityInstance The entity instance to be hydrated + * @param fieldName The relationship field to hydrate in the entity instance + * @param value The foreign key between the entity instance and the field entity. + */ + public void todo(Object entityInstance, String fieldName, Object value) { + Object coercedValue = CoerceUtil.coerce(value, getEntityDictionary() + .getIdType(getEntityDictionary().getParameterizedType(entityInstance, fieldName))); + getTodoList().add(new Todo(entityInstance, fieldName, coercedValue)); + } + + /** + * Sets all the relationship values of an requested entity. + *

+ * Values associated with the existing key will be overwritten. + * + * @param relationshipType The type of the relationship to set + * @param idToInstance A map from relationship ID to the actual relationship instance with that ID + */ + public void populateLookup(Class relationshipType, Map idToInstance) { + if (getObjectLookups().containsKey(relationshipType)) { + getObjectLookups().get(relationshipType).putAll(idToInstance); + } else { + getObjectLookups().put(relationshipType, idToInstance); + } + } + + /** + * Stitch all entity instances currently in this {@link StitchList} by setting their relationship fields whose + * values are determined by relationship ID's. + */ + public void stitch() { + for (Todo todo : getTodoList()) { + Object entityInstance = todo.getEntityInstance(); + String relationshipName = todo.getRelationshipName(); + Object foreignKey = todo.getForeignKey(); + + Class relationshipType = getEntityDictionary().getParameterizedType(entityInstance, relationshipName); + Object relationshipValue = getObjectLookups().get(relationshipType).get(foreignKey); + + getEntityDictionary().setValue(entityInstance, relationshipName, relationshipValue); + } + } + + /** + * Returns a mapping from relationship name to an immutable list of foreign key ID objects. + *

+ * For example, given the following {@code todoList}: + * 

+     * {@code
+     *     [PlayerStats, country, 344]
+     *     [PlayerStats, country, 840]
+     *     [PlayerStats, country, 344]
+     *     [PlayerStats, player, 1]
+     *     [PlayerStats, player, 1]
+     *     [PlayerStats, player, 1]
+     * }
+     *
+ * this method returns a map of the following: + * 
+     *     [
+     *         "country": [344, 840]
+     *         "player": [1]
+     *     ]
+     *
+ * + * @return a mapping from relationship name to an ordered list of relationship join ID's + */ + public Map> getHydrationMapping() { + return getTodoList().stream() + .collect( + Collectors.groupingBy( + Todo::getRelationshipName, + Collectors.mapping( + Todo::getForeignKey, + Collectors.collectingAndThen( + Collectors.toCollection(LinkedList::new), + Collections::unmodifiableList + ) + ) + ) + ); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java new file mode 100644 index 0000000000..ac93a6d920 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLEntityHydrator.java @@ -0,0 +1,83 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.AbstractEntityHydrator; +import com.yahoo.elide.utils.coerce.CoerceUtil; +import lombok.AccessLevel; +import lombok.Getter; + +import java.util.AbstractMap; +import java.util.Collections; +import java.util.LinkedList; +import java.util.List; +import java.util.Map; +import java.util.stream.Collectors; + +import javax.persistence.EntityManager; + +/** + * {@link SQLEntityHydrator} hydrates the entity loaded by {@link SQLQueryEngine#executeQuery(Query)}. + */ +public class SQLEntityHydrator extends AbstractEntityHydrator { + + @Getter(AccessLevel.PRIVATE) + private final EntityManager entityManager; + + /** + * Constructor. + * + * @param results The loaded objects from {@link SQLQueryEngine#executeQuery(Query)} + * @param query The query passed to {@link SQLQueryEngine#executeQuery(Query)} to load the objects + * @param entityDictionary An object that sets entity instance values and provides entity metadata info + * @param entityManager An service that issues JPQL queries to load relationship objects + */ + public SQLEntityHydrator( + List results, + Query query, + EntityDictionary entityDictionary, + EntityManager entityManager + ) { + super(results, query, entityDictionary); + this.entityManager = entityManager; + } + + @Override + protected Map getRelationshipValues( + Class relationshipType, + List joinFieldIds + ) { + if (joinFieldIds.isEmpty()) { + return Collections.emptyMap(); + } + + List uniqueIds = joinFieldIds.stream() + .distinct() + .collect(Collectors.toCollection(LinkedList::new)); + + List loaded = getEntityManager() + .createQuery( + String.format( + "SELECT e FROM %s e WHERE %s IN (:idList)", + relationshipType.getCanonicalName(), + getEntityDictionary().getIdFieldName(relationshipType) + ) + ) + .setParameter("idList", uniqueIds) + .getResultList(); + + return loaded.stream() + .map(obj -> new AbstractMap.SimpleImmutableEntry<>( + CoerceUtil.coerce( + (Object) getEntityDictionary().getId(obj), + getEntityDictionary().getIdType(relationshipType) + ), + obj)) + .collect(Collectors.toMap(Map.Entry::getKey, Map.Entry::getValue)); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java new file mode 100644 index 0000000000..624d13b02a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQuery.java @@ -0,0 +1,53 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.datastores.aggregation.query.Query; + +import lombok.Builder; +import lombok.Data; +import lombok.NonNull; + +/** + * Aids in constructing a SQL query from String fragments. + */ +@Data +@Builder +public class SQLQuery { + + private static final String SPACE = " "; + + @NonNull + private Query clientQuery; + + @NonNull + private String fromClause; + + @NonNull + private String projectionClause; + + @Builder.Default + private String joinClause = ""; + @Builder.Default + private String whereClause = ""; + @Builder.Default + private String groupByClause = ""; + @Builder.Default + private String havingClause = ""; + @Builder.Default + private String orderByClause = ""; + + @Override + public String toString() { + return String.format("SELECT %s FROM %s", projectionClause, fromClause) + + SPACE + joinClause + + SPACE + whereClause + + SPACE + groupByClause + + SPACE + havingClause + + SPACE + orderByClause; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java new file mode 100644 index 0000000000..e35ae1184c --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -0,0 +1,546 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; +import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.FilterTranslator; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimensionGrain; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; + +import org.hibernate.annotations.Subselect; + +import java.util.Collection; +import java.util.HashSet; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import java.util.stream.Stream; + +/** + * Class to construct query template into real sql query. + */ +public class SQLQueryConstructor { + private final EntityDictionary dictionary; + + public SQLQueryConstructor(EntityDictionary dictionary) { + this.dictionary = dictionary; + } + + /** + * Construct sql query with a template and sorting, where and having clause. + * + * @param clientQuery original query object + * @param template query template constructed from client query + * @param sorting sorting clause + * @param whereClause where clause + * @param havingClause having clause + * @return constructed SQLQuery object contains all information above + */ + public SQLQuery resolveTemplate(Query clientQuery, + SQLQueryTemplate template, + Sorting sorting, + FilterExpression whereClause, + FilterExpression havingClause) { + SQLAnalyticView queriedTable = (SQLAnalyticView) clientQuery.getAnalyticView(); + Class tableCls = clientQuery.getAnalyticView().getCls(); + String tableAlias = getClassAlias(tableCls); + + SQLQuery.SQLQueryBuilder builder = SQLQuery.builder().clientQuery(clientQuery); + + Set joinPaths = new HashSet<>(); + + String tableStatement = tableCls.isAnnotationPresent(FromSubquery.class) + ? "(" + tableCls.getAnnotation(FromSubquery.class).sql() + ")" + : tableCls.isAnnotationPresent(FromTable.class) + ? tableCls.getAnnotation(FromTable.class).name() + : queriedTable.getName(); + + builder.fromClause(String.format("%s AS %s", tableStatement, tableAlias)); + + builder.projectionClause(constructProjectionWithReference(template, queriedTable)); + + Set groupByDimensions = template.getGroupByDimensions(); + + if (!groupByDimensions.isEmpty()) { + if (!clientQuery.getMetrics().isEmpty()) { + builder.groupByClause(constructGroupByWithReference(groupByDimensions, queriedTable)); + } + + joinPaths.addAll(extractJoinPaths(groupByDimensions, queriedTable)); + } + + if (whereClause != null) { + builder.whereClause("WHERE " + translateFilterExpression(whereClause, this::generatePredicateReference)); + + joinPaths.addAll(extractJoinPaths(whereClause)); + } + + if (havingClause != null) { + builder.havingClause("HAVING " + translateFilterExpression( + havingClause, + (predicate) -> constructHavingClauseWithReference(predicate, queriedTable, template))); + + joinPaths.addAll(extractJoinPaths(havingClause)); + } + + if (sorting != null) { + Map sortClauses = sorting.getValidSortingRules(tableCls, dictionary); + builder.orderByClause(extractOrderBy(sortClauses, template)); + + joinPaths.addAll(extractJoinPaths(sortClauses)); + } + + builder.joinClause(extractJoin(joinPaths)); + + return builder.build(); + } + + /** + * Construct directly projection GROUP BY clause using column reference. + * + * @param groupByDimensions columns to project out + * @param queriedTable queried analytic view + * @return GROUP BY tb1.col1, tb2.col2, ... + */ + private String constructGroupByWithReference(Set groupByDimensions, + SQLAnalyticView queriedTable) { + return "GROUP BY " + groupByDimensions.stream() + .map(dimension -> resolveSQLColumnReference(dimension, queriedTable)) + .collect(Collectors.joining(", ")); + } + + /** + * Construct HAVING clause filter using physical column references. Metric fields need to be aggregated in HAVING. + * + * @param predicate a filter predicate in HAVING clause + * @param table Elide logical table this query is querying + * @param template query template + * @return an filter/constraint expression that can be put in HAVING clause + */ + private String constructHavingClauseWithReference(FilterPredicate predicate, + Table table, + SQLQueryTemplate template) { + Path.PathElement last = predicate.getPath().lastElement().get(); + Class lastClass = last.getType(); + String fieldName = last.getFieldName(); + + if (!lastClass.equals(table.getCls())) { + throw new InvalidPredicateException("The having clause can only reference fact table aggregations."); + } + + MetricFunctionInvocation metric = template.getMetrics().stream() + // TODO: filter predicate should support alias + .filter(invocation -> invocation.getAlias().equals(fieldName)) + .findFirst() + .orElse(null); + + if (metric != null) { + return metric.getFunctionExpression(); + } else { + return generatePredicateReference(predicate); + } + } + + /** + * Construct SELECT statement expression with metrics and dimensions directly using physical table column + * references. + * + * @param template query template with nested subquery + * @param queriedTable queried analytic view + * @return SELECT function(metric1) AS alias1, tb1.dimension1 AS alias2 + */ + private String constructProjectionWithReference(SQLQueryTemplate template, SQLAnalyticView queriedTable) { + // TODO: project metric field using table column reference + List metricProjections = template.getMetrics().stream() + .map(invocation -> invocation.getFunctionExpression() + " AS " + invocation.getAlias()) + .collect(Collectors.toList()); + + Class tableClass = queriedTable.getCls(); + + List dimensionProjections = template.getGroupByDimensions().stream() + .map(dimension -> { + String fieldName = dimension.getColumn().getName(); + + // relation to Non-JPA Entities object can't be projected + if (dictionary.isRelation(tableClass, fieldName)) { + Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); + if (!dictionary.isJPAEntity(relationshipClass)) { + throw new InvalidPredicateException( + "Can't query on non-JPA relationship field: " + dimension.getColumn().getName()); + } + } + + return resolveSQLColumnReference(dimension, queriedTable) + " AS " + dimension.getAlias(); + }) + .collect(Collectors.toList()); + + if (metricProjections.isEmpty()) { + return "DISTINCT " + String.join(",", dimensionProjections); + } + + return Stream.concat(metricProjections.stream(), dimensionProjections.stream()) + .collect(Collectors.joining(",")); + } + + /** + * Build full join clause for all join paths. + * + * @param joinPaths paths that require joins + * @return built join clause that contains all needed relationship dimension joins for this query. + */ + private String extractJoin(Set joinPaths) { + Set joinClauses = new LinkedHashSet<>(); + + joinPaths.forEach(path -> addJoinClauses(path, joinClauses)); + + return String.join(" ", joinClauses); + } + + /** + * Add a join clause to a set of join clauses. + * + * @param joinPath join path + * @param alreadyJoined A set of joins that have already been computed. + */ + private void addJoinClauses(Path joinPath, Set alreadyJoined) { + String parentAlias = getTypeAlias(joinPath.getPathElements().get(0).getType()); + + for (Path.PathElement pathElement : joinPath.getPathElements()) { + String fieldName = pathElement.getFieldName(); + Class parentClass = pathElement.getType(); + + // Nothing left to join. + if (! dictionary.isRelation(parentClass, fieldName)) { + return; + } + + String joinFragment = extractJoinClause( + parentClass, + parentAlias, + pathElement.getFieldType(), + fieldName); + + alreadyJoined.add(joinFragment); + + parentAlias = appendAlias(parentAlias, fieldName); + } + } + + /** + * Build a single dimension join clause for joining a relationship table to the parent table. + * + * @param parentClass parent class + * @param parentAlias parent table alias + * @param relationshipClass relationship class + * @param relationshipName relationship field name + * @return built join clause i.e. LEFT JOIN table1 AS dimension1 ON table0.dim_id = dimension1.id + */ + private String extractJoinClause(Class parentClass, + String parentAlias, + Class relationshipClass, + String relationshipName) { + //TODO - support composite join keys. + //TODO - support joins where either side owns the relationship. + //TODO - Support INNER and RIGHT joins. + //TODO - Support toMany joins. + + String relationshipAlias = appendAlias(parentAlias, relationshipName); + String relationshipColumnName = dictionary.getAnnotatedColumnName(parentClass, relationshipName); + + // resolve the right hand side of JOIN + String joinSource = constructTableOrSubselect(relationshipClass); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation( + parentClass, + JoinTo.class, + relationshipColumnName); + + String joinClause = joinTo == null + ? String.format("%s.%s = %s.%s", + parentAlias, + relationshipColumnName, + relationshipAlias, + dictionary.getAnnotatedColumnName( + relationshipClass, + dictionary.getIdFieldName(relationshipClass))) + : extractJoinExpression(joinTo.joinClause(), parentAlias, relationshipAlias); + + return String.format("LEFT JOIN %s AS %s ON %s", + joinSource, + relationshipAlias, + joinClause); + } + + + /** + * Make a select statement for a table a sub select query. + * + * @param cls entity class + * @return tableName or (subselect query) + */ + private String constructTableOrSubselect(Class cls) { + return isSubselect(cls) + ? "(" + resolveTableOrSubselect(dictionary, cls) + ")" + : resolveTableOrSubselect(dictionary, cls); + } + + /** + * Given a list of columns to sort on, constructs an ORDER BY clause in SQL. + * @param sortClauses The list of sort columns and their sort order (ascending or descending). + * @return A SQL expression + */ + private String extractOrderBy(Map sortClauses, SQLQueryTemplate template) { + if (sortClauses.isEmpty()) { + return ""; + } + + //TODO - Ensure that order by columns are also present in the group by. + + return " ORDER BY " + sortClauses.entrySet().stream() + .map((entry) -> { + Path expandedPath = expandJoinToPath(entry.getKey()); + Sorting.SortOrder order = entry.getValue(); + + Path.PathElement last = expandedPath.lastElement().get(); + + MetricFunctionInvocation metric = template.getMetrics().stream() + // TODO: filter predicate should support alias + .filter(invocation -> invocation.getAlias().equals(last.getFieldName())) + .findFirst() + .orElse(null); + + String orderByClause = metric == null + ? generateColumnReference(expandedPath, dictionary) + : metric.getFunctionExpression(); + + return orderByClause + (order.equals(Sorting.SortOrder.desc) ? " DESC" : " ASC"); + }) + .collect(Collectors.joining(",")); + } + + /** + * Expands a predicate path (from a sort or filter predicate) to the path contained in + * the JoinTo annotation. If no JoinTo annotation is present, the original path is returned. + * + * @param path The path to expand. + * @return The expanded path. + */ + private Path expandJoinToPath(Path path) { + Path.PathElement pathRoot = path.getPathElements().get(0); + + Class entityClass = pathRoot.getType(); + String fieldName = pathRoot.getFieldName(); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(entityClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + return path; + } + + return new Path(entityClass, dictionary, joinTo.path()); + } + + /** + * Given a filter expression, extracts any entity relationship traversals that require joins. + * + * @param expression The filter expression + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(FilterExpression expression) { + Collection predicates = expression.accept(new PredicateExtractionVisitor()); + + return predicates.stream() + .map(FilterPredicate::getPath) + .map(this::expandJoinToPath) + .filter(path -> path.getPathElements().size() > 1) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Given a list of columns to sort on, extracts any entity relationship traversals that require joins. + * + * @param sortClauses The list of sort columns and their sort order (ascending or descending). + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(Map sortClauses) { + return sortClauses.keySet().stream() + .map(this::expandJoinToPath) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Given the set of group by dimensions, extract any entity relationship traversals that require joins. + * This method takes in a {@link SQLAnalyticView} because the sql join path meta data is stored in it. + * + * @param groupByDimensions The list of dimensions we are grouping on. + * @param queriedTable queried analytic view + * @return A set of path elements that capture a relationship traversal. + */ + private Set extractJoinPaths(Set groupByDimensions, + SQLAnalyticView queriedTable) { + return resolveSQLColumns(groupByDimensions, queriedTable).stream() + .filter((dim) -> dim.getJoinPath() != null) + .map(SQLColumn::getJoinPath) + .collect(Collectors.toCollection(LinkedHashSet::new)); + } + + /** + * Translates a filter expression into SQL. + * + * @param expression The filter expression + * @param columnGenerator A function which generates a column reference in SQL from a FilterPredicate. + * @return A SQL expression + */ + private String translateFilterExpression(FilterExpression expression, + Function columnGenerator) { + FilterTranslator filterVisitor = new FilterTranslator(); + + return filterVisitor.apply(expression, columnGenerator); + } + + /** + * Converts a filter predicate into a SQL WHERE/HAVING clause column reference. + * + * @param predicate The predicate to convert + * @return A SQL fragment that references a database column + */ + private String generatePredicateReference(FilterPredicate predicate) { + return generateColumnReference(predicate.getPath(), dictionary); + } + + /** + * Resolve all projected sql column from a queried table. + * + * @param columnProjections projections + * @param queriedTable sql analytic view + * @return projected columns + */ + private Set resolveSQLColumns(Set columnProjections, SQLAnalyticView queriedTable) { + return columnProjections.stream() + .map(colProjection -> queriedTable.getColumn(colProjection.getColumn().getName())) + .collect(Collectors.toSet()); + } + + /** + * Resolve projected sql column as column reference from a queried table. + * If the projection is {@link TimeDimensionProjection}, the correct time grain expression would be used. + * + * @param columnProjection projection + * @param queriedTable sql analytic view + * @return projected columns + */ + private String resolveSQLColumnReference(ColumnProjection columnProjection, SQLAnalyticView queriedTable) { + SQLColumn sqlColumn = queriedTable.getColumn(columnProjection.getColumn().getName()); + + if (columnProjection instanceof TimeDimensionProjection) { + TimeDimension timeDimension = ((TimeDimensionProjection) columnProjection).getTimeDimension(); + TimeDimensionGrain grainInfo = timeDimension.getSupportedGrains().stream() + .filter(g -> g.getGrain().equals(((TimeDimensionProjection) columnProjection).getGrain())) + .findFirst() + .orElseThrow(() -> new IllegalStateException("Requested time grain not supported.")); + + //TODO - We will likely migrate to a templating language when we support parameterized metrics. + return String.format(grainInfo.getExpression(), sqlColumn.getReference()); + } else { + return sqlColumn.getReference(); + } + } + + /** + * Maps an entity class to a physical table of subselect query, if neither {@link javax.persistence.Table} + * nor {@link Subselect} annotation is present on this class, try {@link FromTable} and {@link FromSubquery}. + * + * @param cls The entity class. + * @return The physical SQL table or subselect query. + */ + private static String resolveTableOrSubselect(EntityDictionary dictionary, Class cls) { + if (isSubselect(cls)) { + if (cls.isAnnotationPresent(FromSubquery.class)) { + return dictionary.getAnnotation(cls, FromSubquery.class).sql(); + } else { + return dictionary.getAnnotation(cls, Subselect.class).value(); + } + } else { + javax.persistence.Table table = dictionary.getAnnotation(cls, javax.persistence.Table.class); + + if (table != null) { + return resolveTableAnnotation(table); + } else { + FromTable fromTable = dictionary.getAnnotation(cls, FromTable.class); + + return fromTable != null ? fromTable.name() : dictionary.getJsonAliasFor(cls); + } + } + } + + /** + * Get the full table name from JPA {@link javax.persistence.Table} annotation. + * + * @param table table annotation + * @return catalog.schema.name + */ + private static String resolveTableAnnotation(javax.persistence.Table table) { + StringBuilder fullTableName = new StringBuilder(); + + if (!"".equals(table.catalog())) { + fullTableName.append(table.catalog()).append("."); + } + if (!"".equals(table.schema())) { + fullTableName.append(table.schema()).append("."); + } + fullTableName.append(table.name()); + + return fullTableName.toString(); + } + + /** + * Construct a join on clause based on given constraint expression, replace "%from" with from table alias + * and "%join" with join table alias. + * + * @param joinClause sql join constraint + * @param fromAlias from table alias + * @param joinToAlias join to table alias + * @return sql string that represents a full join condition + */ + private String extractJoinExpression(String joinClause, String fromAlias, String joinToAlias) { + return joinClause.replace("%from", fromAlias).replace("%join", joinToAlias); + } + + /** + * Check whether a class is mapped to a subselect query instead of a physical table. + * + * @param cls The entity class + * @return True if the class has {@link Subselect} annotation + */ + private static boolean isSubselect(Class cls) { + return cls.isAnnotationPresent(Subselect.class) || cls.isAnnotationPresent(FromSubquery.class); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java new file mode 100644 index 0000000000..b4404131a8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -0,0 +1,299 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static com.yahoo.elide.core.filter.FilterPredicate.getPathAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.TimedFunction; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.utils.coerce.CoerceUtil; + +import org.hibernate.jpa.QueryHints; + +import lombok.Getter; +import lombok.extern.slf4j.Slf4j; + +import java.util.ArrayList; +import java.util.Collection; +import java.util.Collections; +import java.util.LinkedHashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.persistence.EntityManager; +import javax.persistence.EntityManagerFactory; +import javax.persistence.EntityTransaction; + +/** + * QueryEngine for SQL backed stores. + */ +@Slf4j +public class SQLQueryEngine implements QueryEngine { + + private final EntityManagerFactory emf; + private final EntityDictionary metadataDictionary; + + @Getter + private Map, Table> tables; + + public SQLQueryEngine(EntityManagerFactory emf, MetaDataStore metaDataStore) { + this.emf = emf; + this.metadataDictionary = metaDataStore.getDictionary(); + + Set tables = metaDataStore.getMetaData(Table.class); + tables.addAll(metaDataStore.getMetaData(AnalyticView.class)); + + this.tables = tables.stream() + .map(table -> table instanceof AnalyticView + ? new SQLAnalyticView(table.getCls(), metadataDictionary) + : new SQLTable(table.getCls(), metadataDictionary)) + .collect(Collectors.toMap(Table::getCls, Function.identity())); + } + + @Override + public Table getTable(Class entityClass) { + return tables.get(entityClass); + } + + @Override + public Iterable executeQuery(Query query) { + EntityManager entityManager = null; + EntityTransaction transaction = null; + try { + entityManager = emf.createEntityManager(); + + // manually begin the transaction + transaction = entityManager.getTransaction(); + if (!transaction.isActive()) { + transaction.begin(); + } + + // Translate the query into SQL. + SQLQuery sql = toSQL(query); + log.debug("SQL Query: " + sql); + + javax.persistence.Query jpaQuery = entityManager.createNativeQuery(sql.toString()); + + Pagination pagination = query.getPagination(); + if (pagination != null) { + jpaQuery.setFirstResult(pagination.getOffset()); + jpaQuery.setMaxResults(pagination.getLimit()); + + if (pagination.isGenerateTotals()) { + + SQLQuery paginationSQL = toPageTotalSQL(sql); + javax.persistence.Query pageTotalQuery = + entityManager.createNativeQuery(paginationSQL.toString()) + .setHint(QueryHints.HINT_READONLY, true); + + //Supply the query parameters to the query + supplyFilterQueryParameters(query, pageTotalQuery); + + //Run the Pagination query and log the time spent. + long total = new TimedFunction<>( + () -> CoerceUtil.coerce(pageTotalQuery.getSingleResult(), Long.class), + "Running Query: " + paginationSQL + ).get(); + + pagination.setPageTotals(total); + } + } + + // Supply the query parameters to the query + supplyFilterQueryParameters(query, jpaQuery); + + // Run the primary query and log the time spent. + List results = new TimedFunction<>( + () -> jpaQuery.setHint(QueryHints.HINT_READONLY, true).getResultList(), + "Running Query: " + sql).get(); + + return new SQLEntityHydrator(results, query, metadataDictionary, entityManager).hydrate(); + } finally { + if (transaction != null && transaction.isActive()) { + transaction.commit(); + } + if (entityManager != null) { + entityManager.close(); + } + } + } + + /** + * Translates the client query into SQL. + * + * @param query the client query. + * @return the SQL query. + */ + protected SQLQuery toSQL(Query query) { + Set groupByDimensions = new LinkedHashSet<>(query.getGroupByDimensions()); + Set timeDimensions = new LinkedHashSet<>(query.getTimeDimensions()); + + // TODO: handle the case of more than one time dimensions + TimeDimensionProjection timeDimension = timeDimensions.stream().findFirst().orElse(null); + + SQLQueryTemplate queryTemplate = query.getMetrics().stream() + .map(invocation -> { + MetricFunction function = invocation.getFunction(); + + if (!(function instanceof SQLMetricFunction)) { + throw new InvalidPredicateException("Non-SQL metric function on " + invocation.getAlias()); + } + + return ((SQLMetricFunction) function).resolve( + invocation.getArgumentMap(), + invocation.getAlias(), + groupByDimensions, + timeDimension); + }) + .reduce(SQLQueryTemplate::merge) + .orElse(new SQLQueryTemplate() { + @Override + public List getMetrics() { + return Collections.emptyList(); + } + + @Override + public Set getNonTimeDimensions() { + return groupByDimensions; + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return timeDimension; + } + }); + + return new SQLQueryConstructor(metadataDictionary).resolveTemplate( + query, + queryTemplate, + query.getSorting(), + query.getWhereFilter(), + query.getHavingFilter()); + } + + + /** + * Given a JPA query, replaces any parameters with their values from client query. + * + * @param query The client query + * @param jpaQuery The JPA query + */ + private void supplyFilterQueryParameters(Query query, + javax.persistence.Query jpaQuery) { + + Collection predicates = new ArrayList<>(); + if (query.getWhereFilter() != null) { + predicates.addAll(query.getWhereFilter().accept(new PredicateExtractionVisitor())); + } + + if (query.getHavingFilter() != null) { + predicates.addAll(query.getHavingFilter().accept(new PredicateExtractionVisitor())); + } + + for (FilterPredicate filterPredicate : predicates) { + if (filterPredicate.getOperator().isParameterized()) { + boolean shouldEscape = filterPredicate.isMatchingOperator(); + filterPredicate.getParameters().forEach(param -> { + jpaQuery.setParameter(param.getName(), shouldEscape ? param.escapeMatching() : param.getValue()); + }); + } + } + } + + /** + * Takes a SQLQuery and creates a new clone that instead returns the total number of records of the original + * query. + * + * @param sql The original query + * @return A new query that returns the total number of records. + */ + private SQLQuery toPageTotalSQL(SQLQuery sql) { + // TODO: refactor this method + String groupByDimensions = + extractSQLDimensions(sql.getClientQuery(), (SQLAnalyticView) sql.getClientQuery().getAnalyticView()) + .stream() + .map(SQLColumn::getReference) + .collect(Collectors.joining(", ")); + + String projectionClause = String.format("COUNT(DISTINCT(%s))", groupByDimensions); + + return SQLQuery.builder() + .clientQuery(sql.getClientQuery()) + .projectionClause(projectionClause) + .fromClause(sql.getFromClause()) + .joinClause(sql.getJoinClause()) + .whereClause(sql.getWhereClause()) + .build(); + } + + /** + * Extract dimension projects in a query to sql dimensions. + * + * @param query requested query + * @param queriedTable queried analytic view + * @return sql dimensions in this query + */ + private List extractSQLDimensions(Query query, SQLAnalyticView queriedTable) { + return query.getDimensions().stream() + .map(projection -> queriedTable.getColumn(projection.getColumn().getName())) + .collect(Collectors.toList()); + } + + /** + * Converts a filter predicate path into a SQL column reference. + * All other code should use this method to generate sql column reference, no matter where the reference is used ( + * select statement, group by clause, where clause, having clause or order by clause). + * + * @param path The predicate path to convert + * @param dictionary dictionary to expand joinTo path + * @return A SQL fragment that references a database column + */ + public static String generateColumnReference(Path path, EntityDictionary dictionary) { + Path.PathElement last = path.lastElement().get(); + Class lastClass = last.getType(); + String fieldName = last.getFieldName(); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(lastClass, JoinTo.class, fieldName); + + if (joinTo == null) { + return getPathAlias(path) + "." + dictionary.getAnnotatedColumnName(lastClass, last.getFieldName()); + } else { + return generateColumnReference(new Path(lastClass, dictionary, joinTo.path()), dictionary); + } + } + + /** + * Get alias for an entity class. + * + * @param entityClass entity class + * @return alias + */ + public static String getClassAlias(Class entityClass) { + return FilterPredicate.getTypeAlias(entityClass); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java new file mode 100644 index 0000000000..c7834b0bc3 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java @@ -0,0 +1,30 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import lombok.Getter; + +import javax.persistence.EntityManagerFactory; + +/** + * Object that constructs {@link QueryEngine} based on given entityDictionary and entityManagerFactory. + */ +public class SQLQueryEngineFactory implements QueryEngineFactory { + @Getter + private EntityManagerFactory emf; + + public SQLQueryEngineFactory(EntityManagerFactory emf) { + this.emf = emf; + } + + @Override + public QueryEngine buildQueryEngine(MetaDataStore metaDataStore) { + return new SQLQueryEngine(emf, metaDataStore); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java new file mode 100644 index 0000000000..9145416051 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromSubquery.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the entity or field is derived from a native SQL subquery. + */ +@Documented +@Target({ElementType.TYPE, ElementType.METHOD, ElementType.FIELD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FromSubquery { + + /** + * The SQL subquery. + * + * @return The SQL subquery. + */ + String sql(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java new file mode 100644 index 0000000000..4715db7639 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/FromTable.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the entity is derived directly from a physical table or view in the database. + */ +@Documented +@Target({ElementType.TYPE}) +@Retention(RetentionPolicy.RUNTIME) +public @interface FromTable { + + /** + * The table or view name. + * + * @return The table or view name. + */ + String name(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java new file mode 100644 index 0000000000..299135aea0 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java @@ -0,0 +1,37 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +/** + * Indicates that the annotated entity field is derived from a join to another table. + * This annotation must be present for relationship to views. + */ +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface JoinTo { + + /** + * Dot separated path through the entity relationship graph to an attribute. + * If the current entity is author, then a path would be "book.publisher.name". + * @return The path + */ + String path() default ""; + + /** + * Join on clause constraint for customizing relationship joins as a plain sql string. Provided in the model. + * Use "%from" and "%join% to represent the two sides of join. + * + * @return join constraint like %from.col1 = %join.col2 + */ + String joinClause() default ""; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java new file mode 100644 index 0000000000..89d5edfbad --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable.resolveSQLDimensions; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Map; + +/** + * SQL extension of {@link AnalyticView} which also contains sql column meta data. + */ +@EqualsAndHashCode(callSuper = true) +@Data +public class SQLAnalyticView extends AnalyticView { + private Map sqlColumns; + + public SQLAnalyticView(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + this.sqlColumns = resolveSQLDimensions(cls, dictionary); + } + + public SQLColumn getColumn(String fieldName) { + return sqlColumns.get(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java new file mode 100644 index 0000000000..2eac1ea9a2 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Getter; + +/** + * SQLColumn contains meta data about underlying physical table. + */ +public class SQLColumn extends Column { + @Getter + private final String reference; + + @Getter + private final Path joinPath; + + protected SQLColumn(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); + this.joinPath = null; + } else { + Path path = new Path(tableClass, dictionary, joinTo.path()); + this.reference = generateColumnReference(path, dictionary); + this.joinPath = path; + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java new file mode 100644 index 0000000000..f0e230cfc7 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java @@ -0,0 +1,51 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.models.Column; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import lombok.Data; +import lombok.EqualsAndHashCode; + +import java.util.Map; +import java.util.function.Function; +import java.util.stream.Collectors; + +/** + * SQL extension of {@link Table} which also contains sql column meta data. + */ +@EqualsAndHashCode(callSuper = true) +@Data +public class SQLTable extends Table { + private Map sqlColumns; + + public SQLTable(Class cls, EntityDictionary dictionary) { + super(cls, dictionary); + this.sqlColumns = resolveSQLDimensions(cls, dictionary); + } + + /** + * Resolve all sql columns of a table. + * + * @param cls table class + * @param dictionary dictionary contains the table class + * @return all resolved sql column metadata + */ + public static Map resolveSQLDimensions(Class cls, EntityDictionary dictionary) { + return dictionary.getAllFields(cls).stream() + .filter(field -> Column.getDataType(cls, field, dictionary) != null) + .filter(field -> !isMetricField(dictionary, cls, field)) + .collect(Collectors.toMap(Function.identity(), field -> new SQLColumn(cls, field, dictionary))); + } + + public SQLColumn getColumn(String fieldName) { + return sqlColumns.get(fieldName); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java new file mode 100644 index 0000000000..f8b633416b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/SQLMetricFunction.java @@ -0,0 +1,62 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric; + +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.request.Argument; + +import java.util.Collections; +import java.util.List; +import java.util.Map; +import java.util.Set; + +/** + * SQL extension of {@link MetricFunction} which would be invoked as sql and can construct sql templates. + */ +public class SQLMetricFunction extends MetricFunction { + public SQLMetricFunction(String name, String longName, String description, String expression, + Set arguments) { + super(name, longName, description, expression, arguments); + } + + /** + * Construct a sql query template for a physical table with provided information. + * Table name would be filled in when convert the template into real query. + * + * @param arguments arguments provided in the request + * @param alias result alias + * @param dimensions groupBy dimensions + * @param timeDimension aggregated time dimension + * @return SELECT function(arguments, fields) AS alias GROUP BY dimensions, timeDimension + */ + public SQLQueryTemplate resolve(Map arguments, + String alias, + Set dimensions, + TimeDimensionProjection timeDimension) { + MetricFunctionInvocation invoked = invoke(arguments, alias); + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return Collections.singletonList(invoked); + } + + @Override + public Set getNonTimeDimensions() { + return dimensions; + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return timeDimension; + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java new file mode 100644 index 0000000000..4ff7ddeb81 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlAvg.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Average of a field. + */ +public class SqlAvg extends SQLMetricFunction { + public SqlAvg() { + super("avg", "average", "sql average function", "AVG(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java new file mode 100644 index 0000000000..7a0918e927 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMax.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Max of a field. + */ +public class SqlMax extends SQLMetricFunction { + public SqlMax() { + super("max", "max", "sql max function", "MAX(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java new file mode 100644 index 0000000000..221559e5fd --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlMin.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Min of a field. + */ +public class SqlMin extends SQLMetricFunction { + public SqlMin() { + super("min", "min", "sql min function", "MIN(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java new file mode 100644 index 0000000000..9319f817bb --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metric/functions/SqlSum.java @@ -0,0 +1,19 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions; + +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Collections; + +/** + * Sum of a field. + */ +public class SqlSum extends SQLMetricFunction { + public SqlSum() { + super("sum", "sum", "sql sum function", "SUM(%s)", Collections.emptySet()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java new file mode 100644 index 0000000000..efc06caecf --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java @@ -0,0 +1,111 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.query; + +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import com.google.common.collect.Sets; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; +import java.util.Set; + +/** + * SQLQueryTemplate contains projections information about a sql query. + */ +public interface SQLQueryTemplate { + /** + * Get all invoked metrics in this query. + * + * @return invoked metrics + */ + List getMetrics(); + + /** + * Get all non-time dimensions in this query. + * + * @return non-time dimensions + */ + Set getNonTimeDimensions(); + + /** + * Get aggregated time dimension for this query. + * + * @return time dimension + */ + TimeDimensionProjection getTimeDimension(); + + /** + * Get all GROUP BY dimensions in this query, include time and non-time dimensions. + * + * @return all GROUP BY dimensions + */ + default Set getGroupByDimensions() { + return getTimeDimension() == null + ? getNonTimeDimensions() + : Sets.union(getNonTimeDimensions(), Collections.singleton(getTimeDimension())); + } + + /** + * Get a copy of this query with a requested time grain. + * + * @param timeGrain requested time grain + * @return a copied query template + */ + default SQLQueryTemplate toTimeGrain(TimeGrain timeGrain) { + SQLQueryTemplate wrapped = this; + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return wrapped.getMetrics(); + } + + @Override + public Set getNonTimeDimensions() { + return wrapped.getNonTimeDimensions(); + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return wrapped.getTimeDimension().toTimeGrain(timeGrain); + } + }; + } + + /** + * Merge with other query. + * + * @param second other query template + * @return merged query template + */ + default SQLQueryTemplate merge(SQLQueryTemplate second) { + SQLQueryTemplate first = this; + // TODO: validate dimension + List merged = new ArrayList<>(first.getMetrics()); + merged.addAll(second.getMetrics()); + + return new SQLQueryTemplate() { + @Override + public List getMetrics() { + return merged; + } + + @Override + public Set getNonTimeDimensions() { + return first.getNonTimeDimensions(); + } + + @Override + public TimeDimensionProjection getTimeDimension() { + return first.getTimeDimension(); + } + }; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java new file mode 100644 index 0000000000..c45102a3ac --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java @@ -0,0 +1,26 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.time; + +import java.time.Period; + +/** + * {@link TimeGrain} is a set of concrete {@link TimeGrain} implementations which support "natural" time buckets. + */ +public enum TimeGrain { + + DAY(Period.ofDays(1)), + WEEK(Period.ofWeeks(1)), + MONTH(Period.ofMonths(1)), + YEAR(Period.ofYears(1)) + ; + + private final Period period; + + TimeGrain(final Period period) { + this.period = period; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java new file mode 100644 index 0000000000..6078874b1e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java @@ -0,0 +1,164 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.request.Argument; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.ArrayList; +import java.util.Date; +import java.util.List; + +public class EntityProjectionTranslatorTest extends SQLUnitTest { + private static EntityProjection basicProjection = EntityProjection.builder() + .type(PlayerStats.class) + .attribute(Attribute.builder() + .type(long.class) + .name("lowScore") + .build()) + .attribute(Attribute.builder() + .type(String.class) + .name("overallRating") + .build()) + .relationship(Relationship.builder() + .name("country") + .projection(EntityProjection.builder() + .type(Country.class) + .attribute(Attribute.builder() + .type(String.class) + .name("name") + .build()) + .build()) + .build()) + .build(); + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + } + + @Test + public void testBasicTranslation() { + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + basicProjection, + dictionary + ); + + Query query = translator.getQuery(); + + assertEquals(playerStatsTable, query.getAnalyticView()); + assertEquals(1, query.getMetrics().size()); + assertEquals("lowScore", query.getMetrics().get(0).getAlias()); + assertEquals(2, query.getGroupByDimensions().size()); + + List dimensions = new ArrayList<>(query.getGroupByDimensions()); + assertEquals("overallRating", dimensions.get(0).getColumn().getName()); + assertEquals("country", dimensions.get(1).getColumn().getName()); + } + + @Test + public void testWherePromotion() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("overallRating==Good,lowScore<45", + PlayerStats.class, false); + + EntityProjection projection = basicProjection.copyOf() + .filterExpression(originalFilter) + .build(); + + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + ); + + Query query = translator.getQuery(); + + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + assertEquals(whereFilter, query.getWhereFilter()); + assertEquals(havingFilter, query.getHavingFilter()); + } + + @Test + public void testInvalidQueriedTable() { + EntityProjection projection = EntityProjection.builder() + .type(Country.class) + .build(); + + assertThrows(InvalidOperationException.class, () -> new EntityProjectionTranslator( + new Table(Country.class, dictionary), + projection, + dictionary + )); + } + + @Test + public void testTimeDimension() { + EntityProjection projection = basicProjection.copyOf() + .attribute(Attribute.builder() + .type(Date.class) + .name("recordedDate") + .build()) + .build(); + + EntityProjectionTranslator translator = new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + ); + + Query query = translator.getQuery(); + + List timeDimensions = new ArrayList<>(query.getTimeDimensions()); + assertEquals(1, timeDimensions.size()); + assertEquals("recordedDate", timeDimensions.get(0).getAlias()); + assertEquals(TimeGrain.DAY, timeDimensions.get(0).getGrain()); + } + + @Test + public void testUnsupportedTimeGrain() { + EntityProjection projection = basicProjection.copyOf() + .attribute(Attribute.builder() + .type(Date.class) + .name("recordedDate") + .argument(Argument.builder() + .name("grain") + .value("year") + .build()) + .build()) + .build(); + + assertThrows(InvalidOperationException.class, () -> new EntityProjectionTranslator( + playerStatsTable, + projection, + dictionary + )); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java new file mode 100644 index 0000000000..e0b7f2ba45 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java @@ -0,0 +1,204 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidOperationException; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; +import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.query.Query; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.Arrays; +import java.util.Collections; +import java.util.HashSet; +import java.util.Map; +import java.util.Set; +import java.util.TreeMap; + +public class QueryValidatorTest extends SQLUnitTest { + @BeforeAll + public static void init() { + SQLUnitTest.init(); + } + + @Test + public void testNoMetricQuery() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + QueryValidator validator = new QueryValidator(query, Collections.singleton("overallRating"), dictionary); + + UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); + assertEquals("Query with no metric can't sort on nested field.", exception.getMessage()); + } + + @Test + public void testSortingOnId() { + Map sortMap = new TreeMap<>(); + sortMap.put("id", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("id"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("id", "overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Sorting on id field is not permitted'", exception.getMessage()); + } + + @Test + public void testSortingOnNotQueriedDimension() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Can't sort on country as it is not present in query'", exception.getMessage()); + } + + @Test + public void testSortingOnNotQueriedMetric() { + Map sortMap = new TreeMap<>(); + sortMap.put("highScore", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals("Invalid operation: 'Can't sort on highScore as it is not present in query'", exception.getMessage()); + } + + @Test + public void testSortingOnNestedDimensionField() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.continent.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + Set allFields = new HashSet<>(Arrays.asList("country", "lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); + assertEquals("Currently sorting on double nested fields is not supported", exception.getMessage()); + } + + @Test + public void testHavingFilterPromotionUngroupedDimension() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("countryIsoCode==USA,lowScore<45", + PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Dimension field countryIsoCode must be grouped before filtering in having clause.'", + exception.getMessage()); + } + + @Test + public void testHavingFilterNoAggregatedMetric() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("lowScore<45", PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("highScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Metric field lowScore must be aggregated before filtering in having clause.'", + exception.getMessage()); + } + + @Test + public void testHavingFilterOnDimensionTable() throws ParseException { + FilterExpression originalFilter = filterParser.parseFilterExpression("country.isoCode==USA,lowScore<45", + PlayerStats.class, false); + SplitFilterExpressionVisitor visitor = new SplitFilterExpressionVisitor(playerStatsTable); + FilterConstraints constraints = originalFilter.accept(visitor); + FilterExpression whereFilter = constraints.getWhereExpression(); + FilterExpression havingFilter = constraints.getHavingExpression(); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .whereFilter(whereFilter) + .havingFilter(havingFilter) + .build(); + + Set allFields = new HashSet<>(Collections.singletonList("lowScore")); + QueryValidator validator = new QueryValidator(query, allFields, dictionary); + + InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); + assertEquals( + "Invalid operation: 'Can't filter on relationship field [PlayerStats].country/[Country].isoCode in HAVING clause when querying table PlayerStats.'", + exception.getMessage()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java new file mode 100644 index 0000000000..0a557a9b2f --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Continent.java @@ -0,0 +1,28 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.Table; + +@Data +@Entity +@Include(rootLevel = true) +@Table(name = "continents") +@Cardinality(size = CardinalitySize.SMALL) +public class Continent { + + @Id + private String id; + + private String name; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java new file mode 100644 index 0000000000..4e43e7b0e0 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Country.java @@ -0,0 +1,73 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; +import javax.persistence.Table; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Data +@Entity +@Include(rootLevel = true) +@Table(name = "countries") +@Cardinality(size = CardinalitySize.SMALL) +public class Country { + + private String id; + + private String isoCode; + + private String name; + + private Continent continent; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } + + @ManyToOne + @JoinColumn(name = "continent_id") + public Continent getContinent() { + return continent; + } + + public void setContinent(Continent continent) { + this.continent = continent; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java new file mode 100644 index 0000000000..0f983ff1ef --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -0,0 +1,80 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.ToOne; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Data; + +import javax.persistence.Column; +import javax.persistence.Table; + +/** + * A view version of table countries. + */ +@Data +@Include +@Table(name = "countries") +public class CountryView { + @Column(name = "id") + private String countryId; + + private String isoCode; + + private String name; + + private CountryViewNested nestedView; + + @ToOne + @JoinTo( + joinClause = "%from.id = %join.id" + ) + public CountryViewNested getNestedView() { + return nestedView; + } + + @JoinTo(path = "nestedView.isoCode") + private String nestedViewIsoCode; + + private Country nestedRelationship; + + @ToOne + @Column(name = "id") + public Country getNestedRelationship() { + return nestedRelationship; + } + + @JoinTo(path = "nestedRelationship.isoCode") + private String nestedRelationshipIsoCode; + + public String getCountryId() { + return countryId; + } + + public void setCountryId(final String countryId) { + this.countryId = countryId; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java new file mode 100644 index 0000000000..c77ef81a0f --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java @@ -0,0 +1,54 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import lombok.Data; + +import javax.persistence.Id; +import javax.persistence.Table; + +/** + * A nested view for testing. + */ +@Data +@Include +@Table(name = "countries") +public class CountryViewNested { + private String id; + + private String isoCode; + + private String name; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java new file mode 100644 index 0000000000..c8b0c8c718 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/Player.java @@ -0,0 +1,34 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; +import javax.persistence.Table; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Entity +@Include(rootLevel = true) +@Table(name = "players") +@Cardinality(size = CardinalitySize.MEDIUM) +@Data +public class Player { + + @Id + private long id; + + @FriendlyName + private String name; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java new file mode 100644 index 0000000000..e5157c49fe --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java @@ -0,0 +1,235 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.EqualsAndHashCode; +import lombok.Setter; +import lombok.ToString; + +import java.util.Date; +import javax.persistence.Column; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "playerStats") +public class PlayerStats { + + public static final String DAY_FORMAT = "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-dd'), 'yyyy-MM-dd')"; + public static final String MONTH_FORMAT = "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-01'), 'yyyy-MM-dd')"; + + /** + * PK. + */ + private String id; + + /** + * A metric. + */ + private long highScore; + + /** + * A metric. + */ + private long lowScore; + + /** + * A degenerate dimension. + */ + private String overallRating; + + /** + * A table dimension. + */ + private Country country; + + /** + * A subselect dimension. + */ + private SubCountry subCountry; + + @Setter + private String countryViewIsoCode; + + /** + * A dimension field joined to this table. + */ + private String countryIsoCode; + + /** + * A dimension field joined to this table. + */ + private String subCountryIsoCode; + + /** + * A table dimension. + */ + private Player player; + + /** + * A table dimension. + */ + private Player player2; + + private String playerName; + + private String player2Name; + + private Date recordedDate; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + @MetricAggregation(function = SqlMax.class) + @Meta(longName = "awesome score", description = "very awesome score") + public long getHighScore() { + return highScore; + } + + public void setHighScore(final long highScore) { + this.highScore = highScore; + } + + @MetricAggregation(function = SqlMin.class) + public long getLowScore() { + return lowScore; + } + + public void setLowScore(final long lowScore) { + this.lowScore = lowScore; + } + + @FriendlyName + @Cardinality(size = CardinalitySize.MEDIUM) + public String getOverallRating() { + return overallRating; + } + + public void setOverallRating(final String overallRating) { + this.overallRating = overallRating; + } + + @ManyToOne + @JoinColumn(name = "country_id") + public Country getCountry() { + return country; + } + + public void setCountry(final Country country) { + this.country = country; + } + + @ManyToOne + @JoinColumn(name = "sub_country_id") + public SubCountry getSubCountry() { + return subCountry; + } + + public void setSubCountry(final SubCountry subCountry) { + this.subCountry = subCountry; + } + + @ManyToOne + @JoinColumn(name = "player_id") + public Player getPlayer() { + return player; + } + + public void setPlayer(final Player player) { + this.player = player; + } + + /** + * DO NOT put {@link Cardinality} annotation on this field. See + * + * @return the date of the player session. + */ + @Temporal(grains = { + @TimeGrainDefinition(grain = TimeGrain.DAY, expression = DAY_FORMAT), + @TimeGrainDefinition(grain = TimeGrain.MONTH, expression = MONTH_FORMAT) + }, timeZone = "UTC") + public Date getRecordedDate() { + return recordedDate; + } + + public void setRecordedDate(final Date recordedDate) { + this.recordedDate = recordedDate; + } + + @JoinTo(path = "country.isoCode") + public String getCountryIsoCode() { + return countryIsoCode; + } + + public void setCountryIsoCode(String isoCode) { + this.countryIsoCode = isoCode; + } + + + @JoinTo(path = "subCountry.isoCode") + @Column(updatable = false, insertable = false) // subselect field should be read-only + public String getSubCountryIsoCode() { + return subCountryIsoCode; + } + + public void setSubCountryIsoCode(String isoCode) { + this.subCountryIsoCode = isoCode; + } + + @JoinColumn(name = "player2_id") + @ManyToOne + public Player getPlayer2() { + return player2; + } + + public void setPlayer2(Player player2) { + this.player2 = player2; + } + + @JoinTo(path = "player.name") + public String getPlayerName() { + return playerName; + } + + public void setPlayerName(String playerName) { + this.playerName = playerName; + } + + @JoinTo(path = "player2.name") + public String getPlayer2Name() { + return player2Name; + } + + public void setPlayer2Name(String player2Name) { + this.player2Name = player2Name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java new file mode 100644 index 0000000000..22562b204a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java @@ -0,0 +1,46 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import lombok.Data; + +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.OneToOne; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Data +@FromSubquery(sql = "SELECT stats.highScore, stats.player_id, c.name as countryName FROM playerStats AS stats LEFT JOIN countries AS c ON stats.country_id = c.id WHERE stats.overallRating = 'Great'") +public class PlayerStatsView { + + /** + * PK. + */ + @Id + private String id; + + /** + * A metric. + */ + @MetricAggregation(function = SqlMax.class) + private long highScore; + + /** + * A degenerate dimension. + */ + private String countryName; + + @OneToOne + @JoinColumn(name = "player_id") + private Player player; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java new file mode 100644 index 0000000000..7541c38775 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -0,0 +1,223 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.ToOne; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import lombok.EqualsAndHashCode; +import lombok.Setter; +import lombok.ToString; + +import java.util.Date; +import javax.persistence.Column; +import javax.persistence.Id; +import javax.persistence.JoinColumn; +import javax.persistence.ManyToOne; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "playerStats") +public class PlayerStatsWithView { + + /** + * PK. + */ + private String id; + + /** + * A metric. + */ + private long highScore; + + /** + * A metric. + */ + private long lowScore; + + /** + * A degenerate dimension. + */ + private String overallRating; + + /** + * A table dimension. + */ + private Country country; + + /** + * A subselect dimension. + */ + private SubCountry subCountry; + + private CountryView countryView; + + @Setter + private String countryViewIsoCode; + + @Setter + private String countryViewViewIsoCode; + + @Setter + private String countryViewRelationshipIsoCode; + + /** + * A dimension field joined to this table. + */ + private String countryIsoCode; + + /** + * A dimension field joined to this table. + */ + private String subCountryIsoCode; + + /** + * A table dimension. + */ + private Player player; + + private Date recordedDate; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + @MetricAggregation(function = SqlMax.class) + @Meta(longName = "awesome score", description = "very awesome score") + public long getHighScore() { + return highScore; + } + + public void setHighScore(final long highScore) { + this.highScore = highScore; + } + + @MetricAggregation(function = SqlMin.class) + public long getLowScore() { + return lowScore; + } + + public void setLowScore(final long lowScore) { + this.lowScore = lowScore; + } + + @FriendlyName + @Cardinality(size = CardinalitySize.MEDIUM) + public String getOverallRating() { + return overallRating; + } + + public void setOverallRating(final String overallRating) { + this.overallRating = overallRating; + } + + @ManyToOne + @JoinColumn(name = "country_id") + public Country getCountry() { + return country; + } + + public void setCountry(final Country country) { + this.country = country; + } + + @ManyToOne + @JoinColumn(name = "sub_country_id") + public SubCountry getSubCountry() { + return subCountry; + } + + public void setSubCountry(final SubCountry subCountry) { + this.subCountry = subCountry; + } + + @ManyToOne + @JoinColumn(name = "player_id") + public Player getPlayer() { + return player; + } + + public void setPlayer(final Player player) { + this.player = player; + } + + /** + * DO NOT put {@link Cardinality} annotation on this field. + * + * @return the date of the player session. + */ + @Temporal(grains = { @TimeGrainDefinition(grain = TimeGrain.DAY, expression = "") }, timeZone = "UTC") + public Date getRecordedDate() { + return recordedDate; + } + + public void setRecordedDate(final Date recordedDate) { + this.recordedDate = recordedDate; + } + + @JoinTo(path = "country.isoCode") + public String getCountryIsoCode() { + return countryIsoCode; + } + + public void setCountryIsoCode(String isoCode) { + this.countryIsoCode = isoCode; + } + + + @JoinTo(path = "subCountry.isoCode") + @Column(updatable = false, insertable = false) // subselect field should be read-only + public String getSubCountryIsoCode() { + return subCountryIsoCode; + } + + public void setSubCountryIsoCode(String isoCode) { + this.subCountryIsoCode = isoCode; + } + + @ToOne + @JoinTo(joinClause = "%from.country_id = %join.id") + public CountryView getCountryView() { + return countryView; + } + + @JoinTo(path = "countryView.isoCode") + public String getCountryViewIsoCode() { + return countryViewIsoCode; + } + + @JoinTo(path = "countryView.nestedView.isoCode") + public String getCountryViewViewIsoCode() { + return countryViewViewIsoCode; + } + + @JoinTo(path = "countryView.nestedRelationship.isoCode") + public String getCountryViewRelationshipIsoCode() { + return countryViewRelationshipIsoCode; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java new file mode 100644 index 0000000000..f6c14c611e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/SubCountry.java @@ -0,0 +1,61 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; + +import org.hibernate.annotations.Subselect; + +import lombok.Data; + +import javax.persistence.Entity; +import javax.persistence.Id; + +/** + * A root level entity for testing AggregationDataStore with @Subselect annotation. + */ +@Data +@Entity +@Include(rootLevel = true) +@Subselect(value = "select * from countries") +@Cardinality(size = CardinalitySize.SMALL) +public class SubCountry { + + private String id; + + private String isoCode; + + private String name; + + @Id + public String getId() { + return id; + } + + public void setId(final String id) { + this.id = id; + } + + public String getIsoCode() { + return isoCode; + } + + public void setIsoCode(final String isoCode) { + this.isoCode = isoCode; + } + + @FriendlyName + public String getName() { + return name; + } + + public void setName(final String name) { + this.name = name; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java new file mode 100644 index 0000000000..fa45f179d8 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/VideoGame.java @@ -0,0 +1,79 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.example; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.annotation.MetricComputation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlSum; + +import javax.persistence.Column; +import javax.persistence.Id; + +/** + * A root level entity for testing AggregationDataStore. + */ +@Include(rootLevel = true) +@FromTable(name = "videoGames") +public class VideoGame { + + @Id + private Long id; + + @Column(name = "game_rounds") + @MetricAggregation(function = SqlSum.class) + Long sessions; + + @MetricAggregation(function = SqlSum.class) + Long timeSpent; + + @MetricComputation(expression = "timeSpent / sessions") + private Float timeSpentPerSession; + + @MetricComputation(expression = "timeSpentPerSession / 100") + private Float timeSpentPerGame; + + public Long getId() { + return id; + } + + public void setId(final Long id) { + this.id = id; + } + + public Long getSessions() { + return sessions; + } + + public void setSessions(final Long sessions) { + this.sessions = sessions; + } + + public Long getTimeSpent() { + return timeSpent; + } + + public void setTimeSpent(final Long timeSpent) { + this.timeSpent = timeSpent; + } + + public Float getTimeSpentPerSession() { + return timeSpentPerSession; + } + + public void setTimeSpentPerSession(final Float timeSpentPerSession) { + this.timeSpentPerSession = timeSpentPerSession; + } + + public Float getTimeSpentPerGame() { + return timeSpentPerGame; + } + + public void setTimeSpentPerGame(final Float timeSpentPerGame) { + this.timeSpentPerGame = timeSpentPerGame; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java new file mode 100644 index 0000000000..2ee7bad495 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/FilterConstraintsTest.java @@ -0,0 +1,70 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; + +import org.junit.jupiter.api.Test; + +import java.util.Collections; + +public class FilterConstraintsTest { + + private static final FilterPredicate WHERE_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, String.class, "id"), + Operator.IN, + Collections.singletonList("foo") + ); + private static final FilterPredicate HAVING_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, long.class, "highScore"), + Operator.GT, + Collections.singletonList(99) + ); + + @Test + public void testPureHaving() { + assertTrue(FilterConstraints.pureHaving(HAVING_PREDICATE).isPureHaving()); + assertFalse(FilterConstraints.pureHaving(HAVING_PREDICATE).isPureWhere()); + assertEquals( + "playerStats.highScore GT [99]", + FilterConstraints.pureHaving(HAVING_PREDICATE).getHavingExpression().toString() + ); + assertNull(FilterConstraints.pureHaving(HAVING_PREDICATE).getWhereExpression()); + } + + @Test + public void testPureWhere() { + assertTrue(FilterConstraints.pureWhere(WHERE_PREDICATE).isPureWhere()); + assertFalse(FilterConstraints.pureWhere(WHERE_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.id IN [foo]", + FilterConstraints.pureWhere(WHERE_PREDICATE).getWhereExpression().toString() + ); + assertNull(FilterConstraints.pureWhere(WHERE_PREDICATE).getHavingExpression()); + } + + @Test + public void testWithWhereAndHaving() { + assertFalse(FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).isPureWhere()); + assertFalse(FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.id IN [foo]", + FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + FilterConstraints.withWhereAndHaving(WHERE_PREDICATE, HAVING_PREDICATE).getHavingExpression().toString() + ); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java new file mode 100644 index 0000000000..50ce0dbf4d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/filter/visitor/SplitFilterExpressionVisitorTest.java @@ -0,0 +1,176 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.filter.visitor; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertFalse; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertTrue; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpressionVisitor; +import com.yahoo.elide.core.filter.expression.NotFilterExpression; +import com.yahoo.elide.core.filter.expression.OrFilterExpression; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.Collections; + +public class SplitFilterExpressionVisitorTest { + + private static final FilterPredicate WHERE_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, String.class, "id"), + Operator.IN, + Collections.singletonList("foo") + ); + private static final FilterPredicate HAVING_PREDICATE = new FilterPredicate( + new Path.PathElement(PlayerStats.class, long.class, "highScore"), + Operator.GT, + Collections.singletonList(99) + ); + + private static FilterExpressionVisitor splitFilterExpressionVisitor; + + @BeforeAll + public static void setupEntityDictionary() { + EntityDictionary entityDictionary = new EntityDictionary(Collections.emptyMap()); + entityDictionary.bindEntity(PlayerStats.class); + entityDictionary.bindEntity(Country.class); + entityDictionary.bindEntity(SubCountry.class); + entityDictionary.bindEntity(Player.class); + Table table = new Table(PlayerStats.class, entityDictionary); + splitFilterExpressionVisitor = new SplitFilterExpressionVisitor(table); + } + + @Test + public void testVisitPredicate() { + // predicate should be a WHERE + assertTrue(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).isPureWhere()); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).getWhereExpression().toString() + ); + assertFalse(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).isPureHaving()); + assertNull(splitFilterExpressionVisitor.visitPredicate(WHERE_PREDICATE).getHavingExpression()); + + // predicate should be a HAVING + assertTrue(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).isPureHaving()); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).getHavingExpression().toString() + ); + assertFalse(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).isPureWhere()); + assertNull(splitFilterExpressionVisitor.visitPredicate(HAVING_PREDICATE).getWhereExpression()); + } + + @Test + public void testVisitAndExpression() { + // pure-W AND pure-W + AndFilterExpression filterExpression = new AndFilterExpression(WHERE_PREDICATE, WHERE_PREDICATE); + assertEquals( + "(playerStats.id IN [foo] AND playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertNull(splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression()); + + // pure-H AND pure-W + filterExpression = new AndFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression().toString() + ); + + // pure-W AND pure-H + filterExpression = new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getWhereExpression().toString() + ); + assertEquals( + "playerStats.highScore GT [99]", + splitFilterExpressionVisitor.visitAndExpression(filterExpression).getHavingExpression().toString() + ); + + // non-pure case - H1 AND W1 AND H2 + AndFilterExpression and1 = new AndFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + AndFilterExpression and2 = new AndFilterExpression(and1, HAVING_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(and2).getWhereExpression().toString() + ); + assertEquals( + "(playerStats.highScore GT [99] AND playerStats.highScore GT [99])", + splitFilterExpressionVisitor.visitAndExpression(and2).getHavingExpression().toString() + ); + + // non-pure case - (H1 OR H2) AND W1 + OrFilterExpression or = new OrFilterExpression(HAVING_PREDICATE, HAVING_PREDICATE); + AndFilterExpression and = new AndFilterExpression(or, WHERE_PREDICATE); + assertEquals( + "playerStats.id IN [foo]", + splitFilterExpressionVisitor.visitAndExpression(and).getWhereExpression().toString() + ); + assertEquals( + "(playerStats.highScore GT [99] OR playerStats.highScore GT [99])", + splitFilterExpressionVisitor.visitAndExpression(and).getHavingExpression().toString() + ); + } + + @Test + public void testVisitOrExpression() { + // pure-W OR pure-W + OrFilterExpression filterExpression = new OrFilterExpression(WHERE_PREDICATE, WHERE_PREDICATE); + assertEquals( + "(playerStats.id IN [foo] OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(filterExpression).getWhereExpression().toString() + ); + assertNull(splitFilterExpressionVisitor.visitOrExpression(filterExpression).getHavingExpression()); + + // H1 OR W1 + OrFilterExpression or = new OrFilterExpression(HAVING_PREDICATE, WHERE_PREDICATE); + assertNull(splitFilterExpressionVisitor.visitOrExpression(or).getWhereExpression()); + assertEquals( + "(playerStats.highScore GT [99] OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(or).getHavingExpression().toString() + ); + + // (W1 AND H1) OR W2 + AndFilterExpression and = new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE); + or = new OrFilterExpression(and, WHERE_PREDICATE); + assertNull(splitFilterExpressionVisitor.visitOrExpression(or).getWhereExpression()); + assertEquals( + "((playerStats.id IN [foo] AND playerStats.highScore GT [99]) OR playerStats.id IN [foo])", + splitFilterExpressionVisitor.visitOrExpression(or).getHavingExpression().toString() + ); + } + + @Test + public void testVisitNotExpression() { + NotFilterExpression notExpression = new NotFilterExpression( + new AndFilterExpression(WHERE_PREDICATE, HAVING_PREDICATE) + ); + assertNull(splitFilterExpressionVisitor.visitNotExpression(notExpression).getWhereExpression()); + assertEquals( + "(playerStats.id NOT [foo] OR playerStats.highScore LE [99])", + splitFilterExpressionVisitor.visitNotExpression(notExpression).getHavingExpression().toString() + ); + + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java new file mode 100644 index 0000000000..486dde0269 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.framework; + +import com.yahoo.elide.core.DataStore; +import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.datastores.jpa.JpaDataStore; +import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; +import com.yahoo.elide.datastores.multiplex.MultiplexManager; + +public class AggregationDataStoreTestHarness implements DataStoreTestHarness { + private SQLQueryEngineFactory queryEngineFactory; + + public AggregationDataStoreTestHarness(SQLQueryEngineFactory queryEngineFactory) { + this.queryEngineFactory = queryEngineFactory; + } + + @Override + public DataStore getDataStore() { + MetaDataStore metaDataStore = new MetaDataStore(); + + AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + + DataStore jpaStore = new JpaDataStore( + () -> queryEngineFactory.getEmf().createEntityManager(), + NonJtaTransaction::new + ); + + // meta data store needs to be put at first to populate meta data models + return new MultiplexManager(jpaStore, metaDataStore, aggregationDataStore); + } + + public void cleanseTestData() { + + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java new file mode 100644 index 0000000000..0ad213230b --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java @@ -0,0 +1,99 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.framework; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.example.Continent; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.CountryView; +import com.yahoo.elide.datastores.aggregation.example.CountryViewNested; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; +import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import java.util.Collections; +import java.util.HashMap; +import javax.persistence.EntityManagerFactory; +import javax.persistence.Persistence; + +public abstract class SQLUnitTest { + protected static EntityManagerFactory emf; + protected static AnalyticView playerStatsTable; + protected static EntityDictionary dictionary; + protected static RSQLFilterDialect filterParser; + protected static MetaDataStore metaDataStore = new MetaDataStore(); + + protected static final Country HONG_KONG = new Country(); + protected static final Country USA = new Country(); + protected static final Continent ASIA = new Continent(); + protected static final Continent NA = new Continent(); + + protected static QueryEngine engine; + + public static void init() { + emf = Persistence.createEntityManagerFactory("aggregationStore"); + dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(PlayerStatsWithView.class); + dictionary.bindEntity(PlayerStatsView.class); + dictionary.bindEntity(PlayerStats.class); + dictionary.bindEntity(Country.class); + dictionary.bindEntity(SubCountry.class); + dictionary.bindEntity(Player.class); + dictionary.bindEntity(CountryView.class); + dictionary.bindEntity(CountryViewNested.class); + dictionary.bindEntity(Continent.class); + filterParser = new RSQLFilterDialect(dictionary); + + playerStatsTable = new SQLAnalyticView(PlayerStats.class, dictionary); + + metaDataStore.populateEntityDictionary(dictionary); + + engine = new SQLQueryEngine(emf, metaDataStore); + + ASIA.setName("Asia"); + ASIA.setId("1"); + + NA.setName("North America"); + NA.setId("2"); + + HONG_KONG.setIsoCode("HKG"); + HONG_KONG.setName("Hong Kong"); + HONG_KONG.setId("344"); + HONG_KONG.setContinent(ASIA); + + USA.setIsoCode("USA"); + USA.setName("United States"); + USA.setId("840"); + USA.setContinent(NA); + } + + public static ColumnProjection toProjection(Dimension dimension) { + return ColumnProjection.toProjection(dimension, dimension.getName()); + } + + public static TimeDimensionProjection toProjection(TimeDimension dimension, TimeGrain grain) { + return ColumnProjection.toProjection(dimension, grain, dimension.getName()); + } + + public static MetricFunctionInvocation invoke(Metric metric) { + return metric.getMetricFunction().invoke(Collections.emptySet(), metric.getName()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java new file mode 100644 index 0000000000..aa439d571e --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -0,0 +1,947 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.integration; + +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.argument; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.arguments; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.document; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.field; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.selection; +import static com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL.selections; +import static io.restassured.RestAssured.given; +import static org.hamcrest.CoreMatchers.equalTo; +import static org.hamcrest.CoreMatchers.hasItem; +import static org.hamcrest.CoreMatchers.hasItems; +import static org.junit.jupiter.api.Assertions.assertEquals; + +import com.yahoo.elide.core.HttpStatus; +import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.framework.AggregationDataStoreTestHarness; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.initialization.IntegrationTest; + +import com.fasterxml.jackson.databind.JsonNode; +import com.fasterxml.jackson.databind.ObjectMapper; +import com.fasterxml.jackson.databind.node.ArrayNode; +import com.fasterxml.jackson.databind.node.JsonNodeFactory; +import com.fasterxml.jackson.databind.node.ObjectNode; +import org.junit.jupiter.api.Disabled; +import org.junit.jupiter.api.Test; + +import io.restassured.response.ValidatableResponse; + +import java.io.IOException; +import java.util.Map; +import javax.persistence.EntityManagerFactory; +import javax.persistence.Persistence; +import javax.ws.rs.core.MediaType; + +/** + * Integration tests for {@link AggregationDataStore}. + */ +public class AggregationDataStoreIntegrationTest extends IntegrationTest { + SQLQueryEngineFactory queryEngineFactory; + + private static final ObjectMapper JSON_MAPPER = new ObjectMapper(); + + @Override + protected DataStoreTestHarness createHarness() { + EntityManagerFactory emf = Persistence.createEntityManagerFactory("aggregationStore"); + queryEngineFactory = new SQLQueryEngineFactory(emf); + return new AggregationDataStoreTestHarness(queryEngineFactory); + } + + @Test + public void basicAggregationTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + selections( + field("highScore"), + field("overallRating"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 1234), + field("overallRating", "Good"), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 2412), + field("overallRating", "Great"), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 1000), + field("overallRating", "Good"), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void noMetricQueryTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStatsWithView", + arguments( + argument("sort", "\"countryViewRelationshipIsoCode\"") + ), + selections( + field( + "country", + selections( + field("name"), + field("isoCode") + ) + ), + field("countryViewRelationshipIsoCode") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStatsWithView", + selections( + field( + "country", + selections( + field("name", "Hong Kong"), + field("isoCode", "HKG") + ) + ), + field("countryViewRelationshipIsoCode", "HKG") + ), + selections( + field( + "country", + selections( + field("name", "United States"), + field("isoCode", "USA") + ) + ), + field("countryViewRelationshipIsoCode", "USA") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void whereFilterTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"overallRating==\\\"Good\\\"\"") + ), + selections( + field("highScore"), + field("overallRating") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 1234), + field("overallRating", "Good") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void havingFilterTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test the case that a where clause is promoted into having clause. + * @throws Exception exception + */ + @Test + public void wherePromotionTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"overallRating==\\\"Good\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 72), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Han") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test the case that a where clause, which requires dimension join, is promoted into having clause. + * @throws Exception exception + */ + @Test + public void havingClauseJoinTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"countryIsoCode==\\\"USA\\\",lowScore<\\\"45\\\"\""), + argument("sort", "\"lowScore\"") + ), + selections( + field("lowScore"), + field("countryIsoCode"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("countryIsoCode", "USA"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 241), + field("countryIsoCode", "USA"), + field( + "player", + selections( + field("name", "Jane Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + /** + * Test invalid where promotion on a dimension field that is not grouped. + * @throws Exception exception + */ + @Test + public void ungroupedHavingDimensionTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"countryIsoCode==\\\"USA\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Dimension field countryIsoCode must be grouped before filtering in having clause.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + /** + * Test invalid having clause on a metric field that is not aggregated. + * @throws Exception exception + */ + @Test + public void nonAggregatedHavingMetricTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"highScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Metric field highScore must be aggregated before filtering in having clause.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + /** + * Test invalid where promotion on a different class than the queried class. + * @throws Exception exception + */ + @Test + public void invalidHavingClauseClassTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("filter", "\"country.isoCode==\\\"USA\\\",lowScore<\\\"45\\\"\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String errorMessage = "\"Exception while fetching data (/playerStats) : Invalid operation: " + + "'Can't filter on relationship field [PlayerStats].country/[Country].isoCode in HAVING clause " + + "when querying table PlayerStats.'\""; + + runQueryWithExpectedError(graphQLRequest, errorMessage); + } + + @Test + public void dimensionSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"overallRating\"") + ), + selections( + field("lowScore"), + field("overallRating") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 35), + field("overallRating", "Good") + ), + selections( + field("lowScore", 241), + field("overallRating", "Great") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void metricSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"-highScore\"") + ), + selections( + field("highScore"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("highScore", 2412), + field( + "country", + selections( + field("name", "United States") + ) + ) + ), + selections( + field("highScore", 1000), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void multipleColumnsSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"overallRating,player.name\"") + ), + selections( + field("lowScore"), + field("overallRating"), + field( + "player", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStats", + selections( + field("lowScore", 72), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Han") + ) + ) + ), + selections( + field("lowScore", 35), + field("overallRating", "Good"), + field( + "player", + selections( + field("name", "Jon Doe") + ) + ) + ), + selections( + field("lowScore", 241), + field("overallRating", "Great"), + field( + "player", + selections( + field("name", "Jane Doe") + ) + ) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void idSortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"id\"") + ), + selections( + field("lowScore"), + field("id") + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Sorting on id field is not permitted'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void nestedDimensionNotInQuerySortingTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"-country.name,lowScore\"") + ), + selections( + field("lowScore") + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on country as it is not present in query'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void sortingOnMetricNotInQueryTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"highScore\"") + ), + selections( + field("lowScore"), + field( + "country", + selections( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on highScore as it is not present in query'\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + public void sortingMultipleLevelNesting() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStats", + arguments( + argument("sort", "\"country.continent.name\"") + ), + selections( + field("lowScore"), + field( + "country", + selections( + field("name"), + field( + "continent", + selections( + field("name") + ) + ) + ) + ) + ) + ) + ) + ).toQuery(); + + String expected = "\"Exception while fetching data (/playerStats) : Currently sorting on double nested fields is not supported\""; + + runQueryWithExpectedError(graphQLRequest, expected); + } + + @Test + @Disabled + //FIXME Needs metric computation support for test case to be valid. + public void aggregationComputedMetricTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "videoGame", + selections( + field("timeSpent"), + field("sessions"), + field("timeSpentPerSession"), + field("timeSpentPerGame") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "videoGame", + selections( + field("timeSpent", 1400), + field("sessions", 70), + field("timeSpentPerSession", 20), + field("timeSpentPerGame", 14) + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void basicViewAggregationTest() throws Exception { + String graphQLRequest = document( + selection( + field( + "playerStatsWithView", + selections( + field("highScore"), + field( + "country", + selections( + field("name") + ) + ), + field("countryViewIsoCode") + ) + ) + ) + ).toQuery(); + + String expected = document( + selections( + field( + "playerStatsWithView", + selections( + field("highScore", 1000), + field( + "country", + selections( + field("name", "Hong Kong") + ) + ), + field("countryViewIsoCode", "HKG") + ), + selections( + field("highScore", 2412), + field( + "country", + selections( + field("name", "United States") + ) + ), + field("countryViewIsoCode", "USA") + ) + ) + ) + ).toResponse(); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + + @Test + public void jsonApiAggregationTest() { + given() + .accept("application/vnd.api+json") + .get("/playerStats") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.id", hasItems("0", "1", "2")) + .body("data.attributes.highScore", hasItems(1000, 1234, 2412)) + .body("data.relationships.country.data.id", hasItems("840", "344")); + } + + @Test + public void metaDataTest() { + given() + .accept("application/vnd.api+json") + .get("/table/country") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.cardinality", equalTo("SMALL")) + .body("data.relationships.columns.data.id", hasItems("country.id", "country.name", "country.isoCode")); + + given() + .accept("application/vnd.api+json") + .get("/analyticView/playerStats") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.cardinality", equalTo("LARGE")) + .body( + "data.relationships.dimensions.data.id", + hasItems( + "playerStats.id", + "playerStats.player", + "playerStats.country", + "playerStats.subCountry", + "playerStats.recordedDate", + "playerStats.overallRating", + "playerStats.countryIsoCode", + "playerStats.subCountryIsoCode")) + .body("data.relationships.metrics.data.id", hasItems("playerStats.lowScore", "playerStats.highScore")); + + given() + .accept("application/vnd.api+json") + .get("/dimension/playerStats.player") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("player")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("player")); + + given() + .accept("application/vnd.api+json") + .get("/metric/playerStats.lowScore?include=metricFunction") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("lowScore")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("p_bigint")) + .body("data.relationships.metricFunction.data.id", equalTo("playerStats.lowScore[min]")) + .body("included.id", hasItem("playerStats.lowScore[min]")) + .body("included.attributes.description", hasItem("sql min function")) + .body("included.attributes.expression", hasItem("MIN(lowScore)")) + .body("included.attributes.longName", hasItem("min")); + + given() + .accept("application/vnd.api+json") + .get("/timeDimension/playerStats.recordedDate?include=supportedGrains") + .then() + .statusCode(HttpStatus.SC_OK) + .body("data.attributes.name", equalTo("recordedDate")) + .body("data.attributes.tableName", equalTo("playerStats")) + .body("data.relationships.dataType.data.id", equalTo("date")) + .body( + "data.relationships.supportedGrains.data.id", + hasItems("playerStats.recordedDate.day", "playerStats.recordedDate.month")) + .body("included.id", hasItems("playerStats.recordedDate.day", "playerStats.recordedDate.month")) + .body("included.attributes.grain", hasItems("DAY", "MONTH")) + .body( + "included.attributes.expression", + hasItems( + "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-dd'), 'yyyy-MM-dd')", + "PARSEDATETIME(FORMATDATETIME(%s, 'yyyy-MM-01'), 'yyyy-MM-dd')")); + } + + private void create(String query, Map variables) throws IOException { + runQuery(toJsonQuery(query, variables)); + } + + private void runQueryWithExpectedResult( + String graphQLQuery, + Map variables, + String expected + ) throws IOException { + compareJsonObject(runQuery(graphQLQuery, variables), expected); + } + + private void runQueryWithExpectedResult(String graphQLQuery, String expected) throws IOException { + runQueryWithExpectedResult(graphQLQuery, null, expected); + } + + private void runQueryWithExpectedError( + String graphQLQuery, + Map variables, + String errorMessage + ) throws IOException { + compareErrorMessage(runQuery(graphQLQuery, variables), errorMessage); + } + + private void runQueryWithExpectedError(String graphQLQuery, String errorMessage) throws IOException { + runQueryWithExpectedError(graphQLQuery, null, errorMessage); + } + + private void compareJsonObject(ValidatableResponse response, String expected) throws IOException { + JsonNode responseNode = JSON_MAPPER.readTree(response.extract().body().asString()); + JsonNode expectedNode = JSON_MAPPER.readTree(expected); + assertEquals(expectedNode, responseNode); + } + + private void compareErrorMessage(ValidatableResponse response, String expected) throws IOException { + JsonNode responseNode = JSON_MAPPER.readTree(response.extract().body().asString()); + assertEquals(expected, responseNode.get("errors").get(0).get("message").toString()); + } + + private ValidatableResponse runQuery(String query, Map variables) throws IOException { + return runQuery(toJsonQuery(query, variables)); + } + + private ValidatableResponse runQuery(String query) { + ValidatableResponse res = given() + .contentType(MediaType.APPLICATION_JSON) + .accept(MediaType.APPLICATION_JSON) + .body(query) + .post("/graphQL") + .then(); + + return res; + } + + private String toJsonArray(JsonNode... nodes) throws IOException { + ArrayNode arrayNode = JsonNodeFactory.instance.arrayNode(); + for (JsonNode node : nodes) { + arrayNode.add(node); + } + return JSON_MAPPER.writeValueAsString(arrayNode); + } + + private String toJsonQuery(String query, Map variables) throws IOException { + return JSON_MAPPER.writeValueAsString(toJsonNode(query, variables)); + } + + private JsonNode toJsonNode(String query) { + return toJsonNode(query, null); + } + + private JsonNode toJsonNode(String query, Map variables) { + ObjectNode graphqlNode = JsonNodeFactory.instance.objectNode(); + graphqlNode.put("query", query); + if (variables != null) { + graphqlNode.set("variables", JSON_MAPPER.valueToTree(variables)); + } + return graphqlNode; + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java new file mode 100644 index 0000000000..673112b9ea --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStoreTest.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata; + +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.example.Country; +import com.yahoo.elide.datastores.aggregation.example.CountryView; +import com.yahoo.elide.datastores.aggregation.example.CountryViewNested; +import com.yahoo.elide.datastores.aggregation.example.Player; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.HashMap; + +public class MetaDataStoreTest { + private static MetaDataStore dataStore = new MetaDataStore(); + + @BeforeAll + public static void setup() { + EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary.bindEntity(PlayerStatsWithView.class); + dictionary.bindEntity(PlayerStatsView.class); + dictionary.bindEntity(PlayerStats.class); + dictionary.bindEntity(Country.class); + dictionary.bindEntity(SubCountry.class); + dictionary.bindEntity(Player.class); + dictionary.bindEntity(CountryView.class); + dictionary.bindEntity(CountryViewNested.class); + + dataStore.populateEntityDictionary(dictionary); + } + + @Test + public void testSetup() { + assertNotNull(dataStore); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java new file mode 100644 index 0000000000..6ba6d1f548 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -0,0 +1,681 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.filter.FilterPredicate; +import com.yahoo.elide.core.filter.Operator; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import com.google.common.collect.Lists; +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.sql.Timestamp; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class QueryEngineTest extends SQLUnitTest { + private static AnalyticView playerStatsViewTable; + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + + playerStatsViewTable = new SQLAnalyticView(PlayerStatsView.class, dictionary); + } + + /** + * Test loading all three records from the table. + */ + @Test + public void testFullTableLoad() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(35); + stats1.setHighScore(1234); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(72); + stats2.setHighScore(1000); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test group by a degenerate dimension with a filter applied. + * + * @throws Exception exception + */ + @Test + public void testDegenerateDimensionFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(filterParser.parseFilterExpression("overallRating==Great", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setLowScore(241); + stats1.setOverallRating("Great"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats1, results.get(0)); + } + + /** + * Test filtering on a dimension attribute. + * + * @throws Exception exception + */ + @Test + public void testFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .whereFilter(filterParser.parseFilterExpression("country.name=='United States'", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountry(USA); + + assertEquals(1, results.size()); + assertEquals(usa0, results.get(0)); + + // test relationship hydration + PlayerStats actualStats1 = (PlayerStats) results.get(0); + assertNotNull(actualStats1.getCountry()); + } + + /** + * Test filtering on an attribute that's not present in the query. + * + * @throws Exception exception + */ + @Test + public void testSubqueryFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", + PlayerStatsView.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test a view which filters on "stats.overallRating = 'Great'". + */ + @Test + public void testSubqueryLoad() { + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test sorting by dimension attribute which is not present in the query. + */ + @Test + public void testSortJoin() { + Map sortMap = new TreeMap<>(); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(72); + stats0.setOverallRating("Good"); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(241); + stats1.setOverallRating("Great"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(35); + stats2.setOverallRating("Good"); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test pagination. + */ + @Test + public void testPagination() { + Pagination pagination = Pagination.fromOffsetAndLimit(1, 0, true); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .pagination(pagination) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + //Jon Doe,1234,72,Good,840,2019-07-12 00:00:00 + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setLowScore(35); + stats1.setOverallRating("Good"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(results.size(), 1, "Number of records returned does not match"); + assertEquals(results.get(0), stats1, "Returned record does not match"); + assertEquals(pagination.getPageTotals(), 3, "Page totals does not match"); + } + + /** + * Test having clause integrates with group by clause. + * + * @throws Exception exception + */ + @Test + public void testHavingClause() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .havingFilter(filterParser.parseFilterExpression("highScore < 2400", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + // Only "Good" rating would have total high score less than 2400 + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + assertEquals(1, results.size()); + assertEquals(stats1, results.get(0)); + } + + /** + * Test having clause integrates with group by clause and join. + * + * @throws Exception exception + */ + @Test + public void testHavingClauseJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) + .havingFilter(filterParser.parseFilterExpression("countryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setOverallRating("Great"); + stats0.setCountryIsoCode("USA"); + stats0.setHighScore(2412); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setOverallRating("Good"); + stats1.setCountryIsoCode("USA"); + stats1.setHighScore(1234); + + assertEquals(2, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + } + + /** + * Test group by, having, dimension, metric at the same time. + * + * @throws Exception exception + */ + @Test + public void testEdgeCasesQuery() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsViewTable.getDimension("countryName"))) + .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", + PlayerStatsView.class, false)) + .havingFilter(filterParser.parseFilterExpression("highScore > 300", + PlayerStatsView.class, false)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + stats2.setCountryName("United States"); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test sorting by two different columns-one metric and one dimension. + */ + @Test + public void testSortByMultipleColumns() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.desc); + sortMap.put("player.name", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setOverallRating("Great"); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(72); + stats1.setOverallRating("Good"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(35); + stats2.setOverallRating("Good"); + stats2.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + /** + * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. + */ + @Test + public void testRelationshipHydration() { + Map sortMap = new TreeMap<>(); + sortMap.put("country.name", Sorting.SortOrder.desc); + sortMap.put("overallRating", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(241); + usa0.setOverallRating("Great"); + usa0.setCountry(USA); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(35); + usa1.setOverallRating("Good"); + usa1.setCountry(USA); + + PlayerStats hk2 = new PlayerStats(); + hk2.setId("2"); + hk2.setLowScore(72); + hk2.setOverallRating("Good"); + hk2.setCountry(HONG_KONG); + + assertEquals(3, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + assertEquals(hk2, results.get(2)); + + // test join + PlayerStats actualStats1 = (PlayerStats) results.get(0); + assertNotNull(actualStats1.getCountry()); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + */ + @Test + public void testJoinToGroupBy() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setHighScore(2412); + stats1.setCountryIsoCode("USA"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setHighScore(1000); + stats2.setCountryIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .whereFilter(filterParser.parseFilterExpression("countryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Great"); + stats2.setHighScore(2412); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + */ + @Test + public void testJoinToSort() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryIsoCode", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setCountry(HONG_KONG); + stats1.setHighScore(1000); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Good"); + stats2.setCountry(USA); + stats2.setHighScore(1234); + + PlayerStats stats3 = new PlayerStats(); + stats3.setId("2"); + stats3.setOverallRating("Great"); + stats3.setCountry(USA); + stats3.setHighScore(2412); + + assertEquals(3, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + assertEquals(stats3, results.get(2)); + } + + /** + * Test month grain query. + */ + @Test + public void testTotalScoreByMonth() { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.MONTH)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-01 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats0, results.get(0)); + } + + /** + * Test filter by time dimension. + */ + @Test + public void testFilterByTemporalDimension() { + FilterPredicate predicate = new FilterPredicate( + new Path(PlayerStats.class, dictionary, "recordedDate"), + Operator.IN, + Lists.newArrayList(Timestamp.valueOf("2019-07-11 00:00:00"))); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(predicate) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setHighScore(2412); + stats0.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(1, results.size()); + assertEquals(stats0, results.get(0)); + } + + @Test + public void testSortAggregatedMetric() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setOverallRating("Great"); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(35); + stats1.setOverallRating("Good"); + + assertEquals(2, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + } + + @Test + public void testAmbiguousFields() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("playerName"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("player2Name"))) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(35); + stats0.setPlayerName("Jon Doe"); + stats0.setPlayer2Name("Jane Doe"); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(72); + stats1.setPlayerName("Han"); + stats1.setPlayer2Name("Jon Doe"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("2"); + stats2.setLowScore(241); + stats2.setPlayerName("Jane Doe"); + stats2.setPlayer2Name("Han"); + + assertEquals(3, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); + assertEquals(stats2, results.get(2)); + } + + //TODO - Add Invalid Request Tests +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java new file mode 100644 index 0000000000..b0f536c9db --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -0,0 +1,267 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; + +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStats; +import com.yahoo.elide.datastores.aggregation.example.SubCountry; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.time.TimeGrain; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.sql.Timestamp; +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class SubselectTest extends SQLUnitTest { + private static final SubCountry SUB_HONG_KONG = new SubCountry(); + private static final SubCountry SUB_USA = new SubCountry(); + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + + SUB_HONG_KONG.setIsoCode("HKG"); + SUB_HONG_KONG.setName("Hong Kong"); + SUB_HONG_KONG.setId("344"); + + SUB_USA.setIsoCode("USA"); + SUB_USA.setName("United States"); + SUB_USA.setId("840"); + } + + /** + * Test filtering on a dimension attribute. + * + * @throws Exception exception + */ + @Test + public void testFilterJoin() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(filterParser.parseFilterExpression("subCountry.name=='United States'", + PlayerStats.class, false)) + .build(); + + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setHighScore(1234); + usa0.setOverallRating("Good"); + usa0.setCountry(USA); + usa0.setSubCountry(SUB_USA); + usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(241); + usa1.setHighScore(2412); + usa1.setOverallRating("Great"); + usa1.setCountry(USA); + usa1.setSubCountry(SUB_USA); + usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + + // test join + PlayerStats actualStats0 = (PlayerStats) results.get(0); + assertNotNull(actualStats0.getSubCountry()); + assertNotNull(actualStats0.getCountry()); + } + + /** + * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. + */ + @Test + public void testRelationshipHydration() { + Map sortMap = new TreeMap<>(); + sortMap.put("subCountry.name", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats usa0 = new PlayerStats(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setHighScore(1234); + usa0.setOverallRating("Good"); + usa0.setCountry(USA); + usa0.setSubCountry(SUB_USA); + usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); + + PlayerStats usa1 = new PlayerStats(); + usa1.setId("1"); + usa1.setLowScore(241); + usa1.setHighScore(2412); + usa1.setOverallRating("Great"); + usa1.setCountry(USA); + usa1.setSubCountry(SUB_USA); + usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + + PlayerStats hk2 = new PlayerStats(); + hk2.setId("2"); + hk2.setLowScore(72); + hk2.setHighScore(1000); + hk2.setOverallRating("Good"); + hk2.setCountry(HONG_KONG); + hk2.setSubCountry(SUB_HONG_KONG); + hk2.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); + + assertEquals(3, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(usa1, results.get(1)); + assertEquals(hk2, results.get(2)); + + // test join + PlayerStats actualStats0 = (PlayerStats) results.get(0); + assertNotNull(actualStats0.getSubCountry()); + assertNotNull(actualStats0.getCountry()); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToGroupBy() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountryIsoCode"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setHighScore(2412); + stats1.setSubCountryIsoCode("USA"); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setHighScore(1000); + stats2.setSubCountryIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToFilter() throws Exception { + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .whereFilter(filterParser.parseFilterExpression("subCountryIsoCode==USA", + PlayerStats.class, false)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setHighScore(1234); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Great"); + stats2.setHighScore(2412); + + assertEquals(2, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + } + + /** + * Test grouping by a dimension with a JoinTo annotation. + * + * @throws Exception exception + */ + @Test + public void testJoinToSort() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("subCountryIsoCode", Sorting.SortOrder.asc); + + Query query = Query.builder() + .analyticView(playerStatsTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setOverallRating("Good"); + stats1.setSubCountry(SUB_HONG_KONG); + stats1.setHighScore(1000); + + PlayerStats stats2 = new PlayerStats(); + stats2.setId("1"); + stats2.setOverallRating("Good"); + stats2.setSubCountry(SUB_USA); + stats2.setHighScore(1234); + + PlayerStats stats3 = new PlayerStats(); + stats3.setId("2"); + stats3.setOverallRating("Great"); + stats3.setSubCountry(SUB_USA); + stats3.setHighScore(2412); + + assertEquals(3, results.size()); + assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(1)); + assertEquals(stats3, results.get(2)); + } + + //TODO - Add Invalid Request Tests +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java new file mode 100644 index 0000000000..779029104a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -0,0 +1,261 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.datastores.aggregation.queryengines.sql; + +import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNull; +import static org.junit.jupiter.api.Assertions.assertThrows; + +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; +import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; + +import org.junit.jupiter.api.BeforeAll; +import org.junit.jupiter.api.Test; + +import java.util.List; +import java.util.Map; +import java.util.TreeMap; +import java.util.stream.Collectors; +import java.util.stream.StreamSupport; + +public class ViewTest extends SQLUnitTest { + protected static AnalyticView playerStatsWithViewSchema; + + @BeforeAll + public static void init() { + SQLUnitTest.init(); + playerStatsWithViewSchema = new SQLAnalyticView(PlayerStatsWithView.class, dictionary); + } + + @Test + public void testViewRelationFailure() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryView"))) + .sorting(new Sorting(sortMap)) + .build(); + + assertThrows(InvalidPredicateException.class, () -> engine.executeQuery(query)); + } + + @Test + public void testViewAttribute() { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testNestedViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewViewIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewViewIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testNestedRelationshipAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingNestedViewAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.nestedView.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } + + @Test + public void testSortingNestedRelationshipAttribute() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryView.nestedRelationship.isoCode", Sorting.SortOrder.desc); + + Query query = Query.builder() + .analyticView(playerStatsWithViewSchema) + .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) + .groupByDimension( + toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + .sorting(new Sorting(sortMap)) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsWithView usa0 = new PlayerStatsWithView(); + usa0.setId("0"); + usa0.setLowScore(35); + usa0.setCountryViewRelationshipIsoCode("USA"); + + PlayerStatsWithView hk1 = new PlayerStatsWithView(); + hk1.setId("1"); + hk1.setLowScore(72); + hk1.setCountryViewRelationshipIsoCode("HKG"); + + assertEquals(2, results.size()); + assertEquals(usa0, results.get(0)); + assertEquals(hk1, results.get(1)); + + // the join would not happen for a view join + PlayerStatsWithView actualStats1 = (PlayerStatsWithView) results.get(0); + assertNull(actualStats1.getCountry()); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml b/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml new file mode 100644 index 0000000000..2844a0b784 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/META-INF/persistence.xml @@ -0,0 +1,28 @@ + + + + + + com.yahoo.elide.datastores.aggregation.example.Country + com.yahoo.elide.datastores.aggregation.example.SubCountry + com.yahoo.elide.datastores.aggregation.example.Player + com.yahoo.elide.datastores.aggregation.example.Continent + + + + + + + + + + + + diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv new file mode 100644 index 0000000000..b9f40f2ced --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/continent.csv @@ -0,0 +1,3 @@ +id,name +1,Asia +2,North America diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv new file mode 100644 index 0000000000..ae0eeebc02 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/country.csv @@ -0,0 +1,3 @@ +id,isoCode,name,continent_id +344,HKG,Hong Kong,1 +840,USA,United States,2 diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql b/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql new file mode 100644 index 0000000000..1667f6f061 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/create_tables.sql @@ -0,0 +1,37 @@ +CREATE TABLE IF NOT EXISTS playerStats + ( + highScore BIGINT, + lowScore BIGINT, + overallRating VARCHAR(255), + country_id VARCHAR(255), + sub_country_id VARCHAR(255), + player_id BIGINT, + player2_id BIGINT, + recordedDate DATETIME + ) AS SELECT * FROM CSVREAD('classpath:player_stats.csv'); + +CREATE TABLE IF NOT EXISTS countries + ( + id VARCHAR(255), + isoCode VARCHAR(255), + name VARCHAR(255), + continent_id VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:country.csv'); + +CREATE TABLE IF NOT EXISTS players + ( + id BIGINT, + name VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:player.csv'); + +CREATE TABLE IF NOT EXISTS videoGames + ( + game_rounds BIGINT, + timeSpent BIGINT + ) AS SELECT * FROM CSVREAD('classpath:video_games.csv'); + +CREATE TABLE IF NOT EXISTS continents + ( + id BIGINT, + name VARCHAR(255) + ) AS SELECT * FROM CSVREAD('classpath:continent.csv'); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv new file mode 100644 index 0000000000..68b871ea30 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/player.csv @@ -0,0 +1,4 @@ +id,name +1,Jon Doe +2,Jane Doe +3,Han diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv new file mode 100644 index 0000000000..0778a7aa53 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/player_stats.csv @@ -0,0 +1,4 @@ +highScore,lowScore,overallRating,country_id,sub_country_id,player_id,player2_id,recordedDate +1234,35,Good,840,840,1,2,2019-07-12 00:00:00 +2412,241,Great,840,840,2,3,2019-07-11 00:00:00 +1000,72,Good,344,344,3,1,2019-07-13 00:00:00 diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv b/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv new file mode 100644 index 0000000000..407c57c551 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/video_games.csv @@ -0,0 +1,5 @@ +rounds,timeSpent +10,50 +20,150 +30,1000 +10,200 diff --git a/elide-datastore/elide-datastore-hibernate/pom.xml b/elide-datastore/elide-datastore-hibernate/pom.xml index 05d175ceb6..c50501afc0 100644 --- a/elide-datastore/elide-datastore-hibernate/pom.xml +++ b/elide-datastore/elide-datastore-hibernate/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -58,7 +58,7 @@ com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java index f436b1c0d0..5609d4369e 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java @@ -41,6 +41,7 @@ import java.util.HashMap; import java.util.List; import java.util.Map; +import java.util.function.Function; import java.util.stream.Collectors; /** @@ -52,6 +53,11 @@ public class FilterTranslator implements FilterOperation { private static Map operatorGenerators; private static Map, String>, JPQLPredicateGenerator> predicateOverrides; + public static final Function GENERATE_HQL_COLUMN_NO_ALIAS = FilterPredicate::getFieldPath; + + public static final Function GENERATE_HQL_COLUMN_WITH_ALIAS = + (predicate) -> FilterPredicate.getPathAlias(predicate.getPath()) + "." + predicate.getField(); + static { predicateOverrides = new HashMap<>(); @@ -204,8 +210,8 @@ public static void registerJPQLGenerator(Operator op, * @return Returns null if no generator is registered. */ public static JPQLPredicateGenerator lookupJPQLGenerator(Operator op, - Class entityClass, - String fieldName) { + Class entityClass, + String fieldName) { return predicateOverrides.get(Triple.of(op, entityClass, fieldName)); } @@ -225,22 +231,17 @@ public static JPQLPredicateGenerator lookupJPQLGenerator(Operator op) { */ @Override public String apply(FilterPredicate filterPredicate) { - return apply(filterPredicate, false); + return apply(filterPredicate, GENERATE_HQL_COLUMN_NO_ALIAS); } /** * Transforms a filter predicate into a JPQL query fragment. * @param filterPredicate The predicate to transform. - * @param prefixWithAlias Whether or not to append the entity type to the predicate. - * This is useful for table aliases referenced in JPQL for some kinds of joins. + * @param columnGenerator Function which supplies a HQL fragment which represents the column in the predicate. * @return The hql query fragment. */ - protected String apply(FilterPredicate filterPredicate, boolean prefixWithAlias) { - String fieldPath = filterPredicate.getFieldPath(); - - if (prefixWithAlias) { - fieldPath = filterPredicate.getAlias() + "." + filterPredicate.getField(); - } + protected String apply(FilterPredicate filterPredicate, Function columnGenerator) { + String fieldPath = columnGenerator.apply(filterPredicate); Path.PathElement last = filterPredicate.getPath().lastElement().get(); @@ -275,24 +276,46 @@ private static String leastClause(List params) { .collect(Collectors.joining(COMMA))); } + /** + * Translates the filterExpression to a JPQL filter fragment. + * @param filterExpression The filterExpression to translate + * @param prefixWithAlias If true, use the default alias provider to append the predicates with an alias. + * Otherwise, don't append aliases. + * @return A JPQL filter fragment. + */ public String apply(FilterExpression filterExpression, boolean prefixWithAlias) { - JPQLQueryVisitor visitor = new JPQLQueryVisitor(prefixWithAlias); - return "WHERE " + filterExpression.accept(visitor); + Function columnGenerator = GENERATE_HQL_COLUMN_NO_ALIAS; + if (prefixWithAlias) { + columnGenerator = GENERATE_HQL_COLUMN_WITH_ALIAS; + } + + return apply(filterExpression, columnGenerator); + } + + /** + * Translates the filterExpression to a JPQL filter fragment. + * @param filterExpression The filterExpression to translate + * @param columnGenerator Generates a HQL fragment that represents a column in the predicate + * @return A JPQL filter fragment. + */ + public String apply(FilterExpression filterExpression, Function columnGenerator) { + JPQLQueryVisitor visitor = new JPQLQueryVisitor(columnGenerator); + return filterExpression.accept(visitor); } /** * Filter expression visitor which builds an JPQL query. */ public class JPQLQueryVisitor implements FilterExpressionVisitor { - private boolean prefixWithAlias; + private Function columnGenerator; - public JPQLQueryVisitor(boolean prefixWithAlias) { - this.prefixWithAlias = prefixWithAlias; + public JPQLQueryVisitor(Function columnGenerator) { + this.columnGenerator = columnGenerator; } @Override public String visitPredicate(FilterPredicate filterPredicate) { - return apply(filterPredicate, prefixWithAlias); + return apply(filterPredicate, columnGenerator); } @Override diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java index cc31048e5c..194e897bcb 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java @@ -5,6 +5,9 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; +import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.RelationshipType; @@ -49,6 +52,7 @@ public abstract class AbstractHQLQueryBuilder { protected static final String FETCH = " FETCH "; protected static final String SELECT = "SELECT "; protected static final String AS = " AS "; + protected static final String WHERE = " WHERE "; protected static final boolean USE_ALIAS = true; protected static final boolean NO_ALIAS = false; @@ -154,23 +158,22 @@ private String extractJoinClause(FilterPredicate predicate, Set alreadyJ for (Path.PathElement pathElement : predicate.getPath().getPathElements()) { String fieldName = pathElement.getFieldName(); Class typeClass = dictionary.lookupEntityClass(pathElement.getType()); - String typeAlias = FilterPredicate.getTypeAlias(typeClass); + String typeAlias = getTypeAlias(typeClass); - //Nothing left to join. + // Nothing left to join. if (! dictionary.isRelation(pathElement.getType(), fieldName)) { return joinClause.toString(); } - String alias = typeAlias + UNDERSCORE + fieldName; + String alias = previousAlias == null + ? appendAlias(typeAlias, fieldName) + : appendAlias(previousAlias, fieldName); - String joinFragment; + String joinFragment = previousAlias == null + ? LEFT + JOIN + typeAlias + PERIOD + fieldName + SPACE + alias + SPACE + : LEFT + JOIN + previousAlias + PERIOD + fieldName + SPACE + alias + SPACE; //This is the first path element - if (previousAlias == null) { - joinFragment = LEFT + JOIN + typeAlias + PERIOD + fieldName + SPACE + alias + SPACE; - } else { - joinFragment = LEFT + JOIN + previousAlias + PERIOD + fieldName + SPACE + alias + SPACE; - } if (!alreadyJoined.contains(joinFragment)) { joinClause.append(joinFragment); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java index 181d47272e..9978721da6 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java @@ -44,7 +44,7 @@ public Query build() { Collection predicates = filterExpression.get().accept(extractor); //Build the WHERE clause - String filterClause = new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); + String filterClause = WHERE + new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); //Build the JOIN clause String joinClause = getJoinClauseFromFilters(filterExpression.get()) diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java index de95cead5d..80be6043e3 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java @@ -67,10 +67,10 @@ public Query build() { predicates = filterExpression.get().accept(extractor); //Build the WHERE clause - filterClause = new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); + filterClause = WHERE + new FilterTranslator().apply(filterExpression.get(), USE_ALIAS); //Build the JOIN clause - joinClause = getJoinClauseFromFilters(filterExpression.get()); + joinClause = getJoinClauseFromFilters(filterExpression.get()); } else { predicates = new HashSet<>(); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java index 1d04bfbb83..75a9ba681f 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java @@ -63,7 +63,7 @@ public Query build() { + JOIN + parentAlias + PERIOD + relationshipName + SPACE + childAlias + joinClause - + SPACE + + WHERE + filterClause + " AND " + parentAlias + "=:" + parentAlias + SPACE diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java index 9957ce526e..fb999db7d2 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java @@ -130,7 +130,7 @@ public Query build() { + parentAlias + SPACE + joinClause - + SPACE + + WHERE + filterClause); //Fill in the query parameters diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java index 9cc2ebe808..570318a49d 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/core/filter/FilterTranslatorTest.java @@ -65,6 +65,7 @@ public void testHQLQueryVisitor() throws Exception { FilterTranslator filterOp = new FilterTranslator(); String query = filterOp.apply(not, false); + query = query.trim().replaceAll(" +", " "); String p1Params = p1.getParameters().stream() .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); @@ -72,7 +73,7 @@ public void testHQLQueryVisitor() throws Exception { .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); String p3Params = p3.getParameters().stream() .map(FilterPredicate.FilterParameter::getPlaceholder).collect(Collectors.joining(", ")); - String expected = "WHERE NOT (((name IN (" + p2Params + ") OR genre IN (" + p3Params + ")) " + String expected = "NOT (((name IN (" + p2Params + ") OR genre IN (" + p3Params + ")) " + "AND (authors IS NOT EMPTY AND authors.name IN (" + p1Params + "))))"; assertEquals(expected, query); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java index ea9c148aab..8555f9dcbc 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java @@ -99,8 +99,8 @@ public void testFilterJoinClause() { String actual = getJoinClauseFromFilters(andExpression); String expected = " LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.chapters example_Book_chapters " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher "; + + "LEFT JOIN example_Author_books.chapters example_Author_books_chapters " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher "; assertEquals(expected, actual); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java index b7d9fa890c..e6d37f41f9 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java @@ -54,8 +54,9 @@ public void testRootFetch() { TestQueryWrapper query = (TestQueryWrapper) builder.build(); - String expected = "SELECT example_Book FROM example.Book AS example_Book "; + String expected = "SELECT example_Book FROM example.Book AS example_Book"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -72,8 +73,9 @@ public void testRootFetchWithSorting() { .withPossibleSorting(Optional.of(new Sorting(sorting))) .build(); - String expected = "SELECT example_Book FROM example.Book AS example_Book order by example_Book.title asc"; + String expected = "SELECT example_Book FROM example.Book AS example_Book order by example_Book.title asc"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -111,14 +113,15 @@ public void testRootFetchWithJoinFilter() { String expected = - "SELECT example_Author FROM example.Author AS example_Author " - + "LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.chapters example_Book_chapters " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher " - + "WHERE (example_Book_chapters.title IN (:books_chapters_title_XXX, :books_chapters_title_XXX) " - + "OR example_Book_publisher.name IN (:books_publisher_name_XXX)) "; + "SELECT example_Author FROM example.Author AS example_Author " + + "LEFT JOIN example_Author.books example_Author_books " + + "LEFT JOIN example_Author_books.chapters example_Author_books_chapters " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher " + + "WHERE (example_Author_books_chapters.title IN (:books_chapters_title_XXX, :books_chapters_title_XXX) " + + "OR example_Author_books_publisher.name IN (:books_publisher_name_XXX))"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_publisher_name_\\w\\w\\w\\w+", ":books_publisher_name_XXX"); @@ -145,10 +148,11 @@ public void testRootFetchWithSortingAndFilters() { .build(); String expected = - "SELECT example_Book FROM example.Book AS example_Book " - + "WHERE example_Book.id IN (:id_XXX) order by example_Book.title asc"; + "SELECT example_Book FROM example.Book AS example_Book " + + "WHERE example_Book.id IN (:id_XXX) order by example_Book.title asc"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); assertEquals(expected, actual); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java index 8e174be8b7..3b15499dad 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java @@ -57,10 +57,11 @@ public void testRootFetch() { TestQueryWrapper query = (TestQueryWrapper) builder.build(); String expected = - "SELECT COUNT(DISTINCT example_Book) " - + "FROM example.Book AS example_Book "; + "SELECT COUNT(DISTINCT example_Book) " + + "FROM example.Book AS example_Book"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -116,14 +117,16 @@ public void testRootFetchWithJoinFilter() { .build(); String expected = - "SELECT COUNT(DISTINCT example_Author) FROM example.Author AS example_Author " - + "LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.chapters example_Book_chapters " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher " - + "WHERE (example_Book_chapters.title IN (:books_chapters_title_XXX, :books_chapters_title_XXX) " - + "OR example_Book_publisher.name IN (:books_publisher_name_XXX))"; + "SELECT COUNT(DISTINCT example_Author) FROM example.Author AS example_Author " + + "LEFT JOIN example_Author.books example_Author_books " + + "LEFT JOIN example_Author_books.chapters example_Author_books_chapters " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher " + + "WHERE (example_Author_books_chapters.title IN " + + "(:books_chapters_title_XXX, :books_chapters_title_XXX) " + + "OR example_Author_books_publisher.name IN (:books_publisher_name_XXX))"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_chapters_title_\\w\\w\\w\\w+", ":books_chapters_title_XXX"); actual = actual.replaceFirst(":books_publisher_name_\\w\\w\\w\\w+", ":books_publisher_name_XXX"); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java index 0d72f7d61f..5717fcb641 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java @@ -103,6 +103,7 @@ public void testSubCollectionFetchWithSorting() { + "JOIN example_Author__fetch.books example_Book " + "WHERE example_Author__fetch=:example_Author__fetch order by example_Book.title asc"; String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } @@ -187,11 +188,12 @@ public void testSubCollectionFetchWithSortingAndFilters() { String expected = "SELECT example_Book FROM example.Author example_Author__fetch " + "JOIN example_Author__fetch.books example_Book " - + "LEFT JOIN example_Book.publisher example_Book_publisher " - + "WHERE example_Book_publisher.name IN (:publisher_name_XXX) AND example_Author__fetch=:example_Author__fetch order by example_Book.title asc"; + + "LEFT JOIN example_Book.publisher example_Book_publisher " + + "WHERE example_Book_publisher.name IN (:publisher_name_XXX) AND example_Author__fetch=:example_Author__fetch order by example_Book.title asc"; String actual = query.getQueryText(); actual = actual.replaceFirst(":publisher_name_\\w+", ":publisher_name_XXX"); + actual = actual.trim().replaceAll(" +", " "); assertEquals(expected, actual); } diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java index 7f15941f6f..c2e78c9198 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java @@ -73,12 +73,13 @@ public void testSubCollectionPageTotals() { .build(); String actual = query.getQueryText(); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); String expected = - "SELECT COUNT(DISTINCT example_Author_books) " - + "FROM example.Author AS example_Author " - + "JOIN example_Author.books example_Author_books " + "SELECT COUNT(DISTINCT example_Author_books) " + + "FROM example.Author AS example_Author " + + "JOIN example_Author.books example_Author_books " + "WHERE example_Author.id IN (:id_XXX)"; assertEquals(expected, actual); @@ -139,15 +140,16 @@ public void testSubCollectionPageTotalsWithJoinFilter() { .build(); String expected = - "SELECT COUNT(DISTINCT example_Author_books) " - + "FROM example.Author AS example_Author " - + "LEFT JOIN example_Author.books example_Author_books " - + "LEFT JOIN example_Author_books.publisher example_Book_publisher " - + "WHERE (example_Book_publisher.name IN (:books_publisher_name_XXX) " + "SELECT COUNT(DISTINCT example_Author_books) " + + "FROM example.Author AS example_Author " + + "LEFT JOIN example_Author.books example_Author_books " + + "LEFT JOIN example_Author_books.publisher example_Author_books_publisher " + + "WHERE (example_Author_books_publisher.name IN (:books_publisher_name_XXX) " + "AND example_Author.id IN (:id_XXX))"; String actual = query.getQueryText(); actual = actual.replaceFirst(":books_publisher_name_\\w+", ":books_publisher_name_XXX"); + actual = actual.trim().replaceAll(" +", " "); actual = actual.replaceFirst(":id_\\w+", ":id_XXX"); assertEquals(expected, actual); diff --git a/elide-datastore/elide-datastore-hibernate3/pom.xml b/elide-datastore/elide-datastore-hibernate3/pom.xml index 3bc37c34ec..94e16bc4d2 100644 --- a/elide-datastore/elide-datastore-hibernate3/pom.xml +++ b/elide-datastore/elide-datastore-hibernate3/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,19 +52,19 @@ com.yahoo.elide elide-datastore-hibernate - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java index bab6d350fa..b7acfb4b6a 100644 --- a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate3.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate3.porting.SessionWrapper; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -114,17 +116,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection The projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -139,9 +142,9 @@ public Object loadObject(Class entityClass, idExpression = new FalsePredicate(idPath); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, sessionWrapper) @@ -156,23 +159,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (isScrollEnabled) { @@ -185,14 +189,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (filteredVal instanceof AbstractPersistentCollection) { @@ -201,31 +206,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let Hibernate manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { diff --git a/elide-datastore/elide-datastore-hibernate5/pom.xml b/elide-datastore/elide-datastore-hibernate5/pom.xml index db5e531c71..445b673b68 100644 --- a/elide-datastore/elide-datastore-hibernate5/pom.xml +++ b/elide-datastore/elide-datastore-hibernate5/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -51,19 +51,19 @@ com.yahoo.elide elide-datastore-hibernate - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java index a8d5c1aa28..f0e08aeac5 100644 --- a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate5.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate5.porting.SessionWrapper; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -119,17 +121,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection The projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -144,9 +147,9 @@ public Object loadObject(Class entityClass, idExpression = new FalsePredicate(idPath); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, sessionWrapper) @@ -161,23 +164,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); @@ -191,14 +195,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (filteredVal instanceof AbstractPersistentCollection) { @@ -207,31 +212,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let Hibernate manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } final QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, sessionWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { @@ -264,7 +268,7 @@ private Long getTotalRecords(Class entityClass, } /** - * Returns the total record count for a entity relationship + * Returns the total record count for a entity relationship. * @param relationship The relationship * @param filterExpression optional security and request filters * @param dictionary the entity dictionary diff --git a/elide-datastore/elide-datastore-inmemorydb/pom.xml b/elide-datastore/elide-datastore-inmemorydb/pom.xml index 14f3f7e52e..6f75d107d5 100644 --- a/elide-datastore/elide-datastore-inmemorydb/pom.xml +++ b/elide-datastore/elide-datastore-inmemorydb/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -51,7 +51,7 @@ com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java b/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java index a7dff3038b..d50da6b506 100644 --- a/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java +++ b/elide-datastore/elide-datastore-inmemorydb/src/test/java/com/yahoo/elide/datastores/inmemory/HashMapDataStoreTest.java @@ -17,6 +17,7 @@ import com.yahoo.elide.example.beans.FirstBean; import com.yahoo.elide.example.beans.NonEntity; import com.yahoo.elide.example.beans.SecondBean; +import com.yahoo.elide.request.EntityProjection; import com.google.common.collect.ImmutableSet; import org.apache.commons.collections4.IterableUtils; @@ -25,7 +26,6 @@ import java.util.HashMap; import java.util.HashSet; -import java.util.Optional; import java.util.Set; /** @@ -33,10 +33,11 @@ */ public class HashMapDataStoreTest { private InMemoryDataStore inMemoryDataStore; + private EntityDictionary entityDictionary; @BeforeEach public void setup() { - final EntityDictionary entityDictionary = new EntityDictionary(new HashMap<>()); + entityDictionary = new EntityDictionary(new HashMap<>()); inMemoryDataStore = new InMemoryDataStore(FirstBean.class.getPackage()); inMemoryDataStore.populateEntityDictionary(entityDictionary); } @@ -56,13 +57,19 @@ public void testValidCommit() throws Exception { object.id = "0"; object.name = "Test"; try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.createObject(object, null); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.commit(null); } try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); assertTrue(beans.iterator().hasNext()); FirstBean bean = (FirstBean) IterableUtils.first(beans); @@ -97,8 +104,9 @@ public void testCanGenerateIdsAfterDataCommitted() throws Exception { // and a meaningful ID is assigned Set names = new HashSet<>(); try (DataStoreTransaction t = inMemoryDataStore.beginTransaction()) { - for (Object objBean : t.loadObjects(FirstBean.class, - Optional.empty(), Optional.empty(), Optional.empty(), null)) { + for (Object objBean : t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null)) { FirstBean bean = (FirstBean) objBean; names.add(bean.name); assertFalse(bean.id == null); diff --git a/elide-datastore/elide-datastore-jpa/pom.xml b/elide-datastore/elide-datastore-jpa/pom.xml index 8feaabf57f..d8822da994 100644 --- a/elide-datastore/elide-datastore-jpa/pom.xml +++ b/elide-datastore/elide-datastore-jpa/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,7 +52,7 @@ com.yahoo.elide elide-datastore-hibernate - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -75,7 +75,11 @@ com.yahoo.elide elide-integration-tests +<<<<<<< HEAD 4.5.14-SNAPSHOT +======= + 5.0.0-pr6-SNAPSHOT +>>>>>>> ef111d6e... Create AggregationDataStore module (#845) test-jar test diff --git a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java index f6f4080c8e..36cf668f14 100644 --- a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java +++ b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java @@ -25,6 +25,8 @@ import com.yahoo.elide.datastores.jpa.porting.EntityManagerWrapper; import com.yahoo.elide.datastores.jpa.porting.QueryWrapper; import com.yahoo.elide.datastores.jpa.transaction.checker.PersistentCollectionChecker; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.extern.slf4j.Slf4j; @@ -136,17 +138,18 @@ public void createObject(Object entity, RequestScope scope) { /** * load a single record with id and filter. * - * @param entityClass class of query object + * @param projection the projection to query * @param id id of the query object - * @param filterExpression FilterExpression contains the predicates * @param scope Request scope associated with specific request */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { + Class entityClass = projection.getType(); + FilterExpression filterExpression = projection.getFilterExpression(); + try { EntityDictionary dictionary = scope.getDictionary(); Class idType = dictionary.getIdType(entityClass); @@ -161,9 +164,9 @@ public Object loadObject(Class entityClass, idExpression = new FilterPredicate(idPath, Operator.FALSE, Collections.emptyList()); } - FilterExpression joinedExpression = filterExpression - .map(fe -> (FilterExpression) new AndFilterExpression(fe, idExpression)) - .orElse(idExpression); + FilterExpression joinedExpression = (filterExpression != null) + ? new AndFilterExpression(filterExpression, idExpression) + : idExpression; QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, dictionary, emWrapper) @@ -178,23 +181,24 @@ public Object loadObject(Class entityClass, @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(entityClass, filterExpression, scope.getDictionary())); - } - }); + Class entityClass = projection.getType(); + Pagination pagination = projection.getPagination(); + FilterExpression filterExpression = projection.getFilterExpression(); + Sorting sorting = projection.getSorting(); + + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(entityClass, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } QueryWrapper query = (QueryWrapper) new RootCollectionFetchQueryBuilder(entityClass, scope.getDictionary(), emWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); return query.getQuery().getResultList(); @@ -204,14 +208,15 @@ public Iterable loadObjects( public Object getRelation( DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { + FilterExpression filterExpression = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + EntityDictionary dictionary = scope.getDictionary(); - Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relationName, scope); + Object val = com.yahoo.elide.core.PersistentResource.getValue(entity, relation.getName(), scope); if (val instanceof Collection) { Collection filteredVal = (Collection) val; if (IS_PERSISTENT_COLLECTION.test(filteredVal)) { @@ -220,31 +225,30 @@ public Object getRelation( * If there is no filtering or sorting required in the data store, and the pagination is default, * return the proxy and let the ORM manage the SQL generation. */ - if (! filterExpression.isPresent() && ! sorting.isPresent() - && (! pagination.isPresent() || (pagination.isPresent() && pagination.get().isDefaultInstance()))) { + if (filterExpression == null && sorting == null + && (pagination == null || (pagination.isDefaultInstance()))) { return val; } - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); RelationshipImpl relationship = new RelationshipImpl( dictionary.lookupEntityClass(entity.getClass()), relationClass, - relationName, + relation.getName(), entity, filteredVal); - pagination.ifPresent(p -> { - if (p.isGenerateTotals()) { - p.setPageTotals(getTotalRecords(relationship, filterExpression, dictionary)); - } - }); + if (pagination != null && pagination.isGenerateTotals()) { + pagination.setPageTotals(getTotalRecords(relationship, + Optional.ofNullable(filterExpression), scope.getDictionary())); + } QueryWrapper query = (QueryWrapper) new SubCollectionFetchQueryBuilder(relationship, dictionary, emWrapper) - .withPossibleFilterExpression(filterExpression) - .withPossibleSorting(sorting) - .withPossiblePagination(pagination) + .withPossibleFilterExpression(Optional.ofNullable(filterExpression)) + .withPossibleSorting(Optional.ofNullable(sorting)) + .withPossiblePagination(Optional.ofNullable(pagination)) .build(); if (query != null) { diff --git a/elide-datastore/elide-datastore-multiplex/pom.xml b/elide-datastore/elide-datastore-multiplex/pom.xml index c2edb1fd2a..b4a7c68677 100644 --- a/elide-datastore/elide-datastore-multiplex/pom.xml +++ b/elide-datastore/elide-datastore-multiplex/pom.xml @@ -10,7 +10,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -47,19 +47,19 @@ com.yahoo.elide elide-datastore-inmemorydb - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test com.yahoo.elide elide-datastore-hibernate5 - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java index 322d734247..f9f39803d8 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java @@ -9,6 +9,9 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; +import lombok.AccessLevel; +import lombok.Setter; + import java.util.Arrays; import java.util.List; import java.util.concurrent.ConcurrentHashMap; @@ -28,10 +31,12 @@ *
  • Attempt to reverse DB1 commit fails * */ -public class MultiplexManager implements DataStore { +public final class MultiplexManager implements DataStore { protected final List dataStores; protected final ConcurrentHashMap, DataStore> dataStoreMap = new ConcurrentHashMap<>(); + + @Setter(AccessLevel.PROTECTED) private EntityDictionary dictionary; /** @@ -55,7 +60,13 @@ public void populateEntityDictionary(EntityDictionary dictionary) { this.dataStoreMap.put(cls, dataStore); // bind to multiplex dictionary dictionary.bindEntity(cls); - dictionary.bindInitializer(subordinateDictionary::initializeEntity, cls); + // copy attribute arguments + subordinateDictionary.getAttributes(cls).forEach( + attribute -> dictionary.addArgumentsToAttribute( + cls, + attribute, + subordinateDictionary.getAttributeArguments(cls, attribute)) + ); } } } diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java index 6efb11fbad..b6685a055b 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java @@ -17,6 +17,9 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import java.io.IOException; @@ -69,27 +72,18 @@ public void createObject(Object entity, RequestScope scope) { getTransaction(entity).createObject(entity, scope); } - @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - return getTransaction(entityClass).loadObject(entityClass, id, filterExpression, scope); + return getTransaction(projection.getType()).loadObject(projection, id, scope); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - return getTransaction(entityClass).loadObjects(entityClass, - filterExpression, - sorting, - pagination, - scope); + return getTransaction(projection.getType()).loadObjects(projection, scope); } @Override @@ -153,36 +147,44 @@ protected DataStoreTransaction getRelationTransaction(Object object, String rela @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filter, - Optional sorting, - Optional pagination, + Relationship relation, RequestScope scope) { - relationTx = getRelationTransaction(entity, relationName); + + FilterExpression filter = relation.getProjection().getFilterExpression(); + Sorting sorting = relation.getProjection().getSorting(); + Pagination pagination = relation.getProjection().getPagination(); + + relationTx = getRelationTransaction(entity, relation.getName()); DataStoreTransaction entityTransaction = getTransaction(entity.getClass()); EntityDictionary dictionary = scope.getDictionary(); - Class relationClass = dictionary.getParameterizedType(entity, relationName); + Class relationClass = dictionary.getParameterizedType(entity, relation.getName()); String idFieldName = dictionary.getIdFieldName(relationClass); // If different transactions, check if bridgeable and try to bridge if (entityTransaction != relationTx && relationTx instanceof BridgeableTransaction) { BridgeableTransaction bridgeableTx = (BridgeableTransaction) relationTx; - RelationshipType relationType = dictionary.getRelationshipType(entity.getClass(), relationName); - Serializable id = filter.map(fe -> extractId(fe, idFieldName, relationClass)).orElse(null); + RelationshipType relationType = dictionary.getRelationshipType(entity.getClass(), relation.getName()); + Serializable id = (filter != null) ? extractId(filter, idFieldName, relationClass) : null; if (relationType.isToMany()) { return id == null ? bridgeableTx.bridgeableLoadObjects( - this, entity, relationName, filter, sorting, pagination, scope) - : bridgeableTx.bridgeableLoadObject(this, entity, relationName, id, filter, scope); + this, entity, relation.getName(), + Optional.ofNullable(filter), + Optional.ofNullable(relation.getProjection().getSorting()), + Optional.ofNullable(relation.getProjection().getPagination()), + scope) + : bridgeableTx.bridgeableLoadObject(this, entity, relation.getName(), + id, Optional.ofNullable(filter), scope); } - return bridgeableTx.bridgeableLoadObject(this, entity, relationName, id, filter, scope); + return bridgeableTx.bridgeableLoadObject(this, entity, relation.getName(), id, + Optional.ofNullable(filter), scope); } // Otherwise, rely on existing underlying transaction to call correctly into relationTx - return entityTransaction.getRelation(relationTx, entity, relationName, filter, sorting, pagination, scope); + return entityTransaction.getRelation(relationTx, entity, relation, scope); } @Override @@ -206,16 +208,15 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } @Override - public Object getAttribute(Object entity, - String attributeName, RequestScope scope) { + public Object getAttribute(Object entity, Attribute attribute, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - return transaction.getAttribute(entity, attributeName, scope); + return transaction.getAttribute(entity, attribute, scope); } @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { + public void setAttribute(Object entity, Attribute attribute, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - transaction.setAttribute(entity, attributeName, attributeValue, scope); + transaction.setAttribute(entity, attribute, scope); } @Override diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java index d20e03097d..5f70e1bf02 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexWriteTransaction.java @@ -10,9 +10,8 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.core.exceptions.TransactionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import java.io.IOException; import java.io.Serializable; @@ -23,7 +22,6 @@ import java.util.IdentityHashMap; import java.util.List; import java.util.Map.Entry; -import java.util.Optional; import javax.ws.rs.WebApplicationException; import javax.ws.rs.core.MultivaluedHashMap; @@ -170,36 +168,28 @@ private Object cloneObject(Object object) { } @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - DataStoreTransaction transaction = getTransaction(entityClass); - return hold(transaction, transaction.loadObject(entityClass, id, filterExpression, scope)); + DataStoreTransaction transaction = getTransaction(projection.getType()); + return hold(transaction, transaction.loadObject(projection, id, scope)); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { - DataStoreTransaction transaction = getTransaction(entityClass); - return hold(transaction, transaction.loadObjects(entityClass, filterExpression, sorting, pagination, scope)); + DataStoreTransaction transaction = getTransaction(projection.getType()); + return hold(transaction, transaction.loadObjects(projection, scope)); } @Override public Object getRelation(DataStoreTransaction relationTx, Object entity, - String relationName, - Optional filter, - Optional sorting, - Optional pagination, + Relationship relationship, RequestScope scope) { DataStoreTransaction transaction = getTransaction(entity.getClass()); - Object relation = super.getRelation(relationTx, entity, relationName, - filter, sorting, pagination, scope); + Object relation = super.getRelation(relationTx, entity, relationship, scope); if (relation instanceof Iterable) { return hold(transaction, (Iterable) relation); diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java index 33526c0b50..3b1d054fda 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/MultiplexManagerTest.java @@ -18,6 +18,7 @@ import com.yahoo.elide.datastores.inmemory.InMemoryDataStore; import com.yahoo.elide.example.beans.FirstBean; import com.yahoo.elide.example.other.OtherBean; +import com.yahoo.elide.request.EntityProjection; import com.google.common.collect.Lists; import org.apache.commons.collections4.IterableUtils; @@ -30,7 +31,6 @@ import java.io.IOException; import java.util.ArrayList; import java.util.HashMap; -import java.util.Optional; /** * MultiplexManager tests. @@ -39,10 +39,11 @@ @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class MultiplexManagerTest { private MultiplexManager multiplexManager; + private EntityDictionary entityDictionary; @BeforeAll public void setup() { - final EntityDictionary entityDictionary = new EntityDictionary(new HashMap<>()); + entityDictionary = new EntityDictionary(new HashMap<>()); final InMemoryDataStore inMemoryDataStore1 = new InMemoryDataStore(FirstBean.class.getPackage()); final InMemoryDataStore inMemoryDataStore2 = new InMemoryDataStore(OtherBean.class.getPackage()); multiplexManager = new MultiplexManager(inMemoryDataStore1, inMemoryDataStore2); @@ -62,15 +63,21 @@ public void testValidCommit() throws IOException { object.id = null; object.name = "Test"; try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null) + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null) .iterator().hasNext()); t.createObject(object, null); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null) + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null) .iterator().hasNext()); t.commit(null); } try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); assertTrue(beans.iterator().hasNext()); FirstBean bean = (FirstBean) IterableUtils.first(beans); @@ -90,19 +97,25 @@ public void partialCommitFailure() throws IOException { assertEquals(ds2, multiplexManager.getSubManager(OtherBean.class)); try (DataStoreTransaction t = ds1.beginTransaction()) { - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); FirstBean firstBean = FirstBean.class.newInstance(); firstBean.name = "name"; t.createObject(firstBean, null); //t.save(firstBean); - assertFalse(t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().hasNext()); + assertFalse(t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().hasNext()); t.commit(null); } catch (InstantiationException | IllegalAccessException e) { log.error("", e); } try (DataStoreTransaction t = multiplexManager.beginTransaction()) { - FirstBean firstBean = (FirstBean) t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null).iterator().next(); + FirstBean firstBean = (FirstBean) t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null).iterator().next(); firstBean.name = "update"; t.save(firstBean, null); OtherBean otherBean = OtherBean.class.newInstance(); @@ -119,7 +132,9 @@ public void partialCommitFailure() throws IOException { } // verify state try (DataStoreTransaction t = ds1.beginTransaction()) { - Iterable beans = t.loadObjects(FirstBean.class, Optional.empty(), Optional.empty(), Optional.empty(), null); + Iterable beans = t.loadObjects(EntityProjection.builder() + .type(FirstBean.class) + .build(), null); assertNotNull(beans); ArrayList list = Lists.newArrayList(beans.iterator()); assertEquals(list.size(), 1); diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java index e1d2414a84..1712965e8a 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java @@ -10,14 +10,11 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.TransactionException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.utils.ClassScanner; import java.io.IOException; import java.io.Serializable; -import java.util.Optional; import javax.persistence.Entity; @@ -68,19 +65,15 @@ public void createObject(Object entity, RequestScope scope) { } @Override - public Object loadObject(Class entityClass, - Serializable id, - Optional filterExpression, - RequestScope scope) { + public Object loadObject(EntityProjection projection, + Serializable id, + RequestScope scope) { throw new TransactionException(null); } @Override public Iterable loadObjects( - Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + EntityProjection projection, RequestScope scope) { throw new TransactionException(null); } diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java index 19d6335329..5cb16dc6e7 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java @@ -9,7 +9,6 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; -import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.InPredicate; @@ -26,6 +25,8 @@ import com.yahoo.elide.datastores.multiplex.MultiplexTransaction; import com.yahoo.elide.example.beans.HibernateUser; import com.yahoo.elide.example.hbase.beans.RedisActions; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; @@ -61,19 +62,18 @@ public DataStoreTransaction beginReadTransaction() { public class ExampleRedisTransaction implements DataStoreTransaction, BridgeableTransaction { @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { - if (entityClass != RedisActions.class) { - log.debug("Tried to load unexpected object from redis: {}", entityClass); + if (projection.getType() != RedisActions.class) { + log.debug("Tried to load unexpected object from redis: {}", projection.getType()); throw new RuntimeException("Tried to load unexpected object from redis!"); } String key = RedisActions.class.getCanonicalName(); - if (filterExpression.isPresent()) { - FilterExpression fe = filterExpression.get(); + FilterExpression fe = projection.getFilterExpression(); + if (fe != null) { RedisFilter filter = fe.accept(new FilterExpressionParser()); if ("user_id".equals(filter.getFieldName())) { Iterable values = fetchValues(key, @@ -89,26 +89,23 @@ public Object loadObject(Class entityClass, } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { - if (entityClass != RedisActions.class) { - log.debug("Tried to load unexpected object from redis: {}", entityClass); + if (projection.getType() != RedisActions.class) { + log.debug("Tried to load unexpected object from redis: {}", projection.getType()); throw new RuntimeException("Tried to load unexpected object from redis!"); } String key = RedisActions.class.getCanonicalName(); - return filterExpression + return Optional.ofNullable(projection.getFilterExpression()) .map(fe -> { RedisFilter filter = fe.accept(new FilterExpressionParser()); if ("user_id".equals(filter.getFieldName())) { return fetchValues(key, v -> v.startsWith("user" + filter.getValues().get(0) + ":")); } - log.error("Received bad filter: {} for type: {}", filter, entityClass); + log.error("Received bad filter: {} for type: {}", filter, projection.getType()); throw new UnsupportedOperationException("Cannot filter object of that type"); }) .orElseGet(() -> fetchValues(key, unused -> true)); @@ -133,11 +130,6 @@ private RedisActions deserializeAction(Map.Entry entry) { return action; } - @Override - public Object getAttribute(Object entity, String attributeName, RequestScope scope) { - return PersistentResource.getValue(entity, attributeName, scope); - } - // ---- Bridgeable Interfaces ---- @Override @@ -147,9 +139,9 @@ public Object bridgeableLoadObject(MultiplexTransaction muxTx, Object parent, St Class entityClass = dictionary.getParameterizedType(parent, relationName); HibernateUser user = (HibernateUser) parent; if ("specialAction".equals(relationName)) { - return muxTx.loadObject(entityClass, + return muxTx.loadObject( + EntityProjection.builder().type(entityClass).build(), String.valueOf(user.getSpecialActionId()), - Optional.empty(), scope); } else if ("redisActions".equals(relationName)) { FilterExpression updatedExpression = new InPredicate( @@ -157,9 +149,11 @@ public Object bridgeableLoadObject(MultiplexTransaction muxTx, Object parent, St String.valueOf(((HibernateUser) parent).getId()) ); - return muxTx.loadObject(entityClass, + return muxTx.loadObject(EntityProjection.builder() + .type(entityClass) + .filterExpression(updatedExpression) + .build(), String.valueOf(lookupId), - Optional.of(updatedExpression), scope); } } @@ -176,11 +170,12 @@ public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, Object new Path.PathElement(entityClass, String.class, "user_id"), String.valueOf(((HibernateUser) parent).getId()) ); - return muxTx.loadObjects(entityClass, - Optional.of(filterExpression), - sorting, - pagination, - scope); + return muxTx.loadObjects(EntityProjection.builder() + .type(entityClass) + .filterExpression(filterExpression) + .sorting(sorting.orElse(null)) + .pagination(pagination.orElse(null)) + .build(), scope); } log.error("Tried to bridge from parent: {} to relation name: {}", parent, relationName); throw new RuntimeException("Unsupported bridging attempted!"); @@ -190,11 +185,7 @@ public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, Object @Override public Object getRelation(DataStoreTransaction relationTx, - Object entity, - String relationName, - Optional filterExpression, - Optional sorting, - Optional pagination, RequestScope scope) { + Object entity, Relationship relationship, RequestScope scope) { throw new UnsupportedOperationException("No redis relationships currently supported."); } @@ -208,11 +199,6 @@ public void updateToOneRelation(DataStoreTransaction relationTx, Object entity, } - @Override - public void setAttribute(Object entity, String attributeName, Object attributeValue, RequestScope scope) { - - } - @Override public void close() throws IOException { diff --git a/elide-datastore/elide-datastore-noop/pom.xml b/elide-datastore/elide-datastore-noop/pom.xml index 4265be7bf2..f51f1d9618 100644 --- a/elide-datastore/elide-datastore-noop/pom.xml +++ b/elide-datastore/elide-datastore-noop/pom.xml @@ -6,7 +6,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java index 3bb0862a96..8e51831449 100644 --- a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java +++ b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java @@ -8,16 +8,13 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import lombok.extern.slf4j.Slf4j; import java.io.IOException; import java.io.Serializable; import java.util.Collections; -import java.util.Optional; /** * Noop transaction. Specifically, this transaction does not perform any actions (i.e. no operation). @@ -74,24 +71,22 @@ public void createObject(Object entity, RequestScope scope) { /** * No-op transaction, do nothing. - * @param entityClass the type of class to load + * @param projection the projection to query * @param id - the ID of the object to load. - * @param filterExpression - security filters that can be evaluated in the data store. * @param scope - the current request scope. It is optional for the data store to attempt evaluation. * @return a new persistent resource with a new instance of {@code entityClass} */ @Override - public Object loadObject(Class entityClass, + public Object loadObject(EntityProjection projection, Serializable id, - Optional filterExpression, RequestScope scope) { // Loads are supported but empty object (with specified id) is returned. // NOTE: This is primarily useful for enabling objects of solely computed properties to be fetched. Object entity; try { - entity = entityClass.newInstance(); + entity = projection.getType().newInstance(); } catch (IllegalAccessException | InstantiationException e) { - log.error("Could not load object {} through NoopStore", entityClass, e); + log.error("Could not load object {} through NoopStore", projection.getType(), e); throw new RuntimeException(e); } @@ -105,22 +100,15 @@ public Object loadObject(Class entityClass, /** * No-op transaction, do nothing. - * @param entityClass - the class to load - * @param filterExpression - filters that can be evaluated in the data store. - * It is optional for the data store to attempt evaluation. - * @param sorting - sorting which can be pushed down to the data store. - * @param pagination - pagination which can be pushed down to the data store. + * @param projection - the projection to load * @param scope - contains request level metadata. * @return a {@link Collections#singletonList} with a new persistent resource with id 1 */ @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope scope) { // Default behavior: load object 1 and return as an array - return Collections.singletonList(this.loadObject(entityClass, 1L, filterExpression, scope)); + return Collections.singletonList(this.loadObject(projection, 1L, scope)); } /** diff --git a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java index 024b6a6a3c..2baaf27ca3 100644 --- a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java +++ b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/beans/NoopBean.java @@ -10,7 +10,7 @@ import javax.persistence.Id; /** - * Simple bean intended to not be persisted + * Simple bean intended to not be persisted. */ @Include(type = "theNoopBean") public class NoopBean { diff --git a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java index 67d1f43f95..26aed1626f 100644 --- a/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java +++ b/elide-datastore/elide-datastore-noop/src/test/java/com/yahoo/elide/datastores/noop/NoopTransactionTest.java @@ -15,6 +15,7 @@ import com.yahoo.elide.core.ObjectEntityCache; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.jsonapi.JsonApiMapper; +import com.yahoo.elide.request.EntityProjection; import com.fasterxml.jackson.databind.ObjectMapper; import org.apache.commons.collections4.IterableUtils; @@ -23,17 +24,17 @@ import org.junit.jupiter.api.TestInstance; import java.util.HashMap; -import java.util.Optional; @TestInstance(TestInstance.Lifecycle.PER_CLASS) public class NoopTransactionTest { - DataStoreTransaction tx = new NoopTransaction(); - NoopBean bean = new NoopBean(); - RequestScope requestScope; + private DataStoreTransaction tx = new NoopTransaction(); + private NoopBean bean = new NoopBean(); + private RequestScope requestScope; + private EntityDictionary dictionary; @BeforeAll public void setup() { - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(NoopBean.class); requestScope = mock(RequestScope.class); JsonApiMapper mapper = mock(JsonApiMapper.class); @@ -79,13 +80,17 @@ public void testCreateObject() throws Exception { public void testLoadObject() throws Exception { // Should return bean with id set - NoopBean bean = (NoopBean) tx.loadObject(NoopBean.class, 1, Optional.empty(), requestScope); - assertEquals((Long) 1L, bean.getId()); + NoopBean bean = (NoopBean) tx.loadObject(EntityProjection.builder() + .type(NoopBean.class) + .build(), 1, requestScope); + assertEquals(bean.getId(), (Long) 1L); } @Test public void testLoadObjects() throws Exception { - Iterable iterable = (Iterable) tx.loadObjects(NoopBean.class, Optional.empty(), Optional.empty(), Optional.empty(), requestScope); + Iterable iterable = (Iterable) tx.loadObjects(EntityProjection.builder() + .type(NoopBean.class) + .build(), requestScope); NoopBean bean = IterableUtils.first(iterable); assertEquals((Long) 1L, bean.getId()); } diff --git a/elide-datastore/elide-datastore-search/pom.xml b/elide-datastore/elide-datastore-search/pom.xml index 6c13cce53d..3d6b1467d9 100644 --- a/elide-datastore/elide-datastore-search/pom.xml +++ b/elide-datastore/elide-datastore-search/pom.xml @@ -12,7 +12,7 @@ com.yahoo.elide elide-datastore-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -57,7 +57,7 @@ com.yahoo.elide elide-integration-tests - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test-jar test @@ -65,7 +65,7 @@ com.yahoo.elide elide-datastore-jpa - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java index dc69d282c9..5a0dcfb7f0 100644 --- a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java +++ b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java @@ -23,6 +23,7 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.EntityProjection; import org.apache.lucene.search.Query; import org.apache.lucene.search.Sort; @@ -69,26 +70,25 @@ public SearchDataTransaction(DataStoreTransaction tx, } @Override - public Iterable loadObjects(Class entityClass, - Optional filterExpression, - Optional sorting, - Optional pagination, + public Iterable loadObjects(EntityProjection projection, RequestScope requestScope) { - if (!filterExpression.isPresent()) { - return super.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + if (projection.getFilterExpression() == null) { + return super.loadObjects(projection, requestScope); } - boolean canSearch = (canSearch(entityClass, filterExpression.get()) != NONE); + boolean canSearch = (canSearch(projection.getType(), projection.getFilterExpression()) != NONE); - if (mustSort(sorting, entityClass)) { - canSearch = canSearch && canSort(sorting.get(), entityClass); + if (mustSort(Optional.ofNullable(projection.getSorting()), projection.getType())) { + canSearch = canSearch && canSort(projection.getSorting(), projection.getType()); } if (canSearch) { - return search(entityClass, filterExpression.get(), sorting, pagination); + return search(projection.getType(), projection.getFilterExpression(), + Optional.ofNullable(projection.getSorting()), + Optional.ofNullable(projection.getPagination())); } - return super.loadObjects(entityClass, filterExpression, sorting, pagination, requestScope); + return super.loadObjects(projection, requestScope); } /** diff --git a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java index 8e30ae5e5f..0d22fb5bf9 100644 --- a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java +++ b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java @@ -25,6 +25,7 @@ import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.search.models.Item; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.utils.coerce.CoerceUtil; import com.yahoo.elide.utils.coerce.converters.ISO8601DateSerde; @@ -40,7 +41,6 @@ import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.stream.Collectors; import java.util.stream.StreamSupport; @@ -54,9 +54,10 @@ public class DataStoreLoadTest { private SearchDataStore searchStore; private DataStoreTransaction wrappedTransaction; private RequestScope mockScope; + private EntityDictionary dictionary; public DataStoreLoadTest() { - EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); + dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(Item.class); filterParser = new RSQLFilterDialect(dictionary); @@ -99,12 +100,16 @@ public void testEqualityPredicate() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name==drum", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects( + EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); /* This query should hit the underlying store */ - verify(wrappedTransaction, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, times(1)).loadObjects(any(), any()); } @Test @@ -115,10 +120,13 @@ public void testEscapedPrefixPredicate() throws Exception { /* Verify that '-' is escaped before we run the query */ FilterExpression filter = filterParser.parseFilterExpression("name==-lucen*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(6L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -127,10 +135,13 @@ public void testEmptyResult() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==+lucen*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -142,10 +153,13 @@ public void testPrefixPredicateWithInMemoryFiltering() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name==dru*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -156,10 +170,13 @@ public void testPrefixPredicatePhrase() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name=='snare\\ dru*'", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -170,10 +187,13 @@ public void testTabCharacter() throws Exception { //Case sensitive query against case insensitive index must lowercase FilterExpression filter = filterParser.parseFilterExpression("name=='*est\tTa*'", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(7L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -184,10 +204,13 @@ public void testContainsPredicate() throws Exception { //Case insensitive query against case insensitive index FilterExpression filter = filterParser.parseFilterExpression("name==*DrU*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L, 3L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -197,10 +220,13 @@ public void testPredicateConjunction() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==drum*;description==brass*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -210,12 +236,15 @@ public void testNonIndexedPredicate() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("price==123;description==brass*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList()); /* This query should hit the underlying store */ - verify(wrappedTransaction, times(1)).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, times(1)).loadObjects(any(), any()); } @Test @@ -225,10 +254,13 @@ public void testPredicateDisjunction() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==drum*,description==ride*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.empty(), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(1L, 2L, 3L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -243,10 +275,13 @@ public void testSortingAscending() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .build(), mockScope); assertListContains(loaded, Lists.newArrayList(4L, 5L, 2L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -261,10 +296,14 @@ public void testSortingDescending() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.empty(), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(2L, 5L, 4L)); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -281,12 +320,16 @@ public void testPaginationPageOne() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.of(pagination), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .pagination(pagination) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(2L)); - assertEquals(3, pagination.getPageTotals()); - - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + assertEquals(pagination.getPageTotals(), 3); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test @@ -303,11 +346,16 @@ public void testPaginationPageTwo() throws Exception { FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); - Iterable loaded = testTransaction.loadObjects(Item.class, Optional.of(filter), Optional.of(sorting), Optional.of(pagination), mockScope); + Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() + .type(Item.class) + .filterExpression(filter) + .sorting(sorting) + .pagination(pagination) + .build(), mockScope); assertListMatches(loaded, Lists.newArrayList(5L)); - assertEquals(3, pagination.getPageTotals()); - verify(wrappedTransaction, never()).loadObjects(any(), any(), any(), any(), any()); + assertEquals(pagination.getPageTotals(), 3); + verify(wrappedTransaction, never()).loadObjects(any(), any()); } @Test diff --git a/elide-datastore/pom.xml b/elide-datastore/pom.xml index 465b27f65c..bdfe31c7d5 100644 --- a/elide-datastore/pom.xml +++ b/elide-datastore/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -44,6 +44,7 @@ + elide-datastore-aggregation elide-datastore-hibernate elide-datastore-hibernate5 elide-datastore-hibernate3 @@ -59,7 +60,7 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.hibernate diff --git a/elide-example-models/pom.xml b/elide-example-models/pom.xml index a2796e8437..ed892206c9 100644 --- a/elide-example-models/pom.xml +++ b/elide-example-models/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/elide-blog-example-resteasy/pom.xml b/elide-example/elide-blog-example-resteasy/pom.xml index 40c3017d15..80bd25d560 100644 --- a/elide-example/elide-blog-example-resteasy/pom.xml +++ b/elide-example/elide-blog-example-resteasy/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -52,12 +52,12 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate5 - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/elide-blog-example/pom.xml b/elide-example/elide-blog-example/pom.xml index fc92db6f0c..551cc5c777 100644 --- a/elide-example/elide-blog-example/pom.xml +++ b/elide-example/elide-blog-example/pom.xml @@ -10,7 +10,7 @@ Elide Example: Hibernate5 API with Security Elide example using javax.persistence, MySQL and Elide Security com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT https://github.com/yahoo/elide @@ -59,17 +59,17 @@ com.yahoo.elide elide-annotations - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-standalone - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-test-helpers - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.antlr diff --git a/elide-example/elide-hibernate3-mysql-example/pom.xml b/elide-example/elide-hibernate3-mysql-example/pom.xml index f6901a8f27..9b7edb4b6e 100644 --- a/elide-example/elide-hibernate3-mysql-example/pom.xml +++ b/elide-example/elide-hibernate3-mysql-example/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -25,12 +25,12 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-hibernate3 - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-example/pom.xml b/elide-example/pom.xml index 93a5df5c20..a6033691a7 100644 --- a/elide-example/pom.xml +++ b/elide-example/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -48,7 +48,7 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.apache.logging.log4j diff --git a/elide-graphql/pom.xml b/elide-graphql/pom.xml index e7220c8867..6880a83fce 100644 --- a/elide-graphql/pom.xml +++ b/elide-graphql/pom.xml @@ -11,7 +11,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -44,12 +44,12 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-test-helpers - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.fasterxml.jackson.core diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java index e48dcbf01e..67e88cf9ea 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Entity.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.request.EntityProjection; import lombok.AllArgsConstructor; import lombok.Getter; @@ -37,11 +38,14 @@ public class Entity { * Class constructor. * @param parentResource parent entity * @param data entity data - * @param entityClass binding entity class + * @param entityClass entity class * @param requestScope the request context object */ - public Entity(Optional parentResource, Map data, - Class entityClass, RequestScope requestScope) { + public Entity( + Optional parentResource, + Map data, + Class entityClass, + RequestScope requestScope) { this.parentResource = parentResource; this.data = data; this.entityClass = entityClass; @@ -90,24 +94,33 @@ private void setRelationships() { this.relationships = new LinkedHashSet<>(); EntityDictionary dictionary = this.requestScope.getDictionary(); - for (Map.Entry entry : this.data.entrySet()) { - if (dictionary.isRelation(this.entityClass, entry.getKey())) { - Set entitySet = new LinkedHashSet<>(); - Class loadClass = dictionary.getParameterizedType(this.entityClass, entry.getKey()); - Boolean isToOne = dictionary.getRelationshipType(this.entityClass, entry.getKey()).isToOne(); - if (isToOne) { - entitySet.add(new Entity(Optional.of(this), - ((Map) entry.getValue()), - loadClass, - this.requestScope)); - } else { - for (Map row : (List>) entry.getValue()) { - entitySet.add(new Entity(Optional.of(this), row, loadClass, this.requestScope)); + this.data.entrySet().stream() + .filter(entry -> dictionary.isRelation(this.entityClass, entry.getKey())) + .forEach(entry -> { + String relationshipName = entry.getKey(); + Class relationshipClass = + dictionary.getParameterizedType(this.entityClass, relationshipName); + + Set relationshipEntities = new LinkedHashSet<>(); + + // if the relationship is ToOne, entry.getValue() should be a single map + if (dictionary.getRelationshipType(this.entityClass, relationshipName).isToOne()) { + relationshipEntities.add(new Entity( + Optional.of(this), + ((Map) entry.getValue()), + relationshipClass, + this.requestScope)); + } else { + for (Map row : (List>) entry.getValue()) { + relationshipEntities.add(new Entity( + Optional.of(this), + row, + relationshipClass, + this.requestScope)); + } } - } - this.relationships.add(new Relationship(entry.getKey(), entitySet)); - } - } + this.relationships.add(new Relationship(relationshipName, relationshipEntities)); + }); } } @@ -173,8 +186,16 @@ public void setId() { * @return {@link PersistentResource} object */ public PersistentResource toPersistentResource() { - return this.data == null ? null : PersistentResource.loadRecord(this.entityClass, - getId().orElse(null), - this.requestScope); + return this.data == null + ? null + : PersistentResource.loadRecord(getProjection(), getId().orElse(null), this.requestScope); + } + + /** + * Get a projection for this entity class. Used for querying inserted entities. + * @return {@link EntityProjection} object + */ + public EntityProjection getProjection() { + return EntityProjection.builder().type(entityClass).build(); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java index cacb21a007..8693fc4446 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/Environment.java @@ -42,7 +42,7 @@ public class Environment { public Environment(DataFetchingEnvironment environment) { Map args = environment.getArguments(); - requestScope = (GraphQLRequestScope) environment.getContext(); + requestScope = environment.getContext(); filters = Optional.ofNullable((String) args.get(ModelBuilder.ARGUMENT_FILTER)); offset = Optional.ofNullable((String) args.get(ModelBuilder.ARGUMENT_AFTER)); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java index a00fd0b338..8d5773744c 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLConversionUtils.java @@ -6,6 +6,7 @@ package com.yahoo.elide.graphql; +import static graphql.schema.GraphQLArgument.newArgument; import static graphql.schema.GraphQLEnumType.newEnum; import static graphql.schema.GraphQLFieldDefinition.newFieldDefinition; import static graphql.schema.GraphQLInputObjectField.newInputObjectField; @@ -19,6 +20,7 @@ import graphql.Scalars; import graphql.schema.DataFetcher; +import graphql.schema.GraphQLArgument; import graphql.schema.GraphQLEnumType; import graphql.schema.GraphQLFieldDefinition; import graphql.schema.GraphQLInputObjectField; @@ -34,7 +36,9 @@ import java.util.Collection; import java.util.Date; import java.util.HashMap; +import java.util.List; import java.util.Map; +import java.util.stream.Collectors; /** * Contains methods that convert from a class to a GraphQL input or query type. @@ -443,6 +447,42 @@ public GraphQLInputObjectType classToInputObject(Class clazz) { return object; } + /** + * Build an Argument list object for the given attribute. + * @param entityClass The Entity class to which this attribute belongs to. + * @param attribute The name of the attribute. + * @param fetcher The data fetcher to associated with the newly created GraphQL Query Type. + * @return Newly created GraphQLArgument Collection for the given attribute. + */ + public List attributeArgumentToQueryObject(Class entityClass, + String attribute, + DataFetcher fetcher) { + return attributeArgumentToQueryObject(entityClass, attribute, fetcher, entityDictionary); + } + + /** + * Build an Argument list object for the given attribute + * @param entityClass The Entity class to which this attribute belongs to. + * @param attribute The name of the attribute. + * @param fetcher The data fetcher to associated with the newly created GraphQL Query Type + * @param dictionary The dictionary that contains the runtime type information for the parent class. + * @return Newly created GraphQLArgument Collection for the given attribute + */ + public List attributeArgumentToQueryObject(Class entityClass, + String attribute, + DataFetcher fetcher, + EntityDictionary dictionary) { + return dictionary.getAttributeArguments(entityClass, attribute) + .stream() + .map(argumentType -> newArgument() + .name(argumentType.getName()) + .type(fetchScalarOrObjectInput(argumentType.getType())) + .build()) + .collect(Collectors.toList()); + + } + + private GraphQLOutputType fetchScalarOrObjectOutput(Class conversionClass, DataFetcher fetcher) { /* If class is enum, provide enum type */ diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java index 35a9b87ba1..c6048d487a 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLEndpoint.java @@ -8,7 +8,6 @@ import com.yahoo.elide.Elide; import com.yahoo.elide.ElideResponse; import com.yahoo.elide.resources.DefaultOpaqueUserFunction; - import lombok.extern.slf4j.Slf4j; import java.util.function.Function; @@ -60,7 +59,6 @@ public GraphQLEndpoint( public Response post( @Context SecurityContext securityContext, String graphQLDocument) { - ElideResponse response = runner.run(graphQLDocument, getUser.apply(securityContext)); return Response.status(response.getResponseCode()).entity(response.getBody()).build(); } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java index 6f31328804..fcaa83d3b6 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/GraphQLRequestScope.java @@ -8,13 +8,12 @@ import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.security.User; import lombok.Getter; - import java.util.HashMap; import java.util.Map; - import javax.ws.rs.core.MultivaluedHashMap; /** @@ -23,13 +22,23 @@ public class GraphQLRequestScope extends RequestScope { @Getter private final Map totalRecordCounts = new HashMap<>(); - public GraphQLRequestScope(DataStoreTransaction transaction, - User user, - ElideSettings elideSettings) { + @Getter + private final GraphQLProjectionInfo projectionInfo; + + public GraphQLRequestScope( + DataStoreTransaction transaction, + User user, + ElideSettings elideSettings, + GraphQLProjectionInfo projectionInfo + ) { // TODO: We're going to break out the two request scopes. `RequestScope` should become an interface and // we should have a GraphQLRequestScope and a JSONAPIRequestScope. // TODO: What should mutate multiple entity value be? There is a problem with this setting in practice. // Namely, we don't filter or paginate in the data store. super("/", null, transaction, user, new MultivaluedHashMap<>(), elideSettings); + this.projectionInfo = projectionInfo; + + // Entity Projection is retrieved from projectionInfo. + this.setEntityProjection(null); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java new file mode 100644 index 0000000000..4e1b69f130 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/KeyWord.java @@ -0,0 +1,47 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql; + +import lombok.Getter; + +/** + * Key words used in graphql parsing. + */ +public enum KeyWord { + NODE("node"), + EDGES("edges"), + PAGE_INFO("pageInfo"), + PAGE_INFO_HAS_NEXT_PAGE("hasNextPage"), + PAGE_INFO_START_CURSOR("startCursor"), + PAGE_INFO_END_CURSOR("endCursor"), + PAGE_INFO_TOTAL_RECORDS("totalRecords"), + TYPENAME("__typename"), + SCHEMA("__schema"), + TYPE("__type"), + UNKNOWN("unknown"); + + @Getter + private String name; + + KeyWord(String name) { + this.name = name; + } + + public boolean equals(String name) { + return this.name.equals(name); + } + + public static KeyWord byName(String value) { + for (KeyWord keyWord : KeyWord.values()) { + if (keyWord.equals(value)) { + return keyWord; + } + } + + return UNKNOWN; + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java index 06947b5409..640aa98c6a 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java @@ -242,7 +242,6 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { .name("_node__" + entityName); String id = dictionary.getIdFieldName(entityClass); - /* our id types are DeferredId objects (not Scalars.GraphQLID) */ builder.field(newFieldDefinition() .name(id) @@ -255,9 +254,10 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { continue; } - log.debug("Building query attribute {} {} for entity {}", + log.debug("Building query attribute {} {} with arguments {} for entity {}", attribute, attributeClass.getName(), + dictionary.getAttributeArguments(attributeClass, attribute).toString(), entityClass.getName()); GraphQLType attributeType = @@ -269,6 +269,7 @@ private GraphQLObjectType buildQueryObject(Class entityClass) { builder.field(newFieldDefinition() .name(attribute) + .argument(generator.attributeArgumentToQueryObject(entityClass, attribute, dataFetcher)) .dataFetcher(dataFetcher) .type((GraphQLOutputType) attributeType) ); @@ -357,9 +358,11 @@ private GraphQLInputType buildInputObjectStub(Class clazz) { builder.name(entityName + ARGUMENT_INPUT); String id = dictionary.getIdFieldName(clazz); - builder.field(newInputObjectField() - .name(id) - .type(Scalars.GraphQLID)); + if (id != null) { + builder.field(newInputObjectField() + .name(id) + .type(Scalars.GraphQLID)); + } for (String attribute : dictionary.getAttributes(clazz)) { Class attributeClass = dictionary.getType(clazz, attribute); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java index 627a88233a..44713a8257 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java @@ -8,18 +8,14 @@ import static com.yahoo.elide.graphql.ModelBuilder.ARGUMENT_OPERATION; -import com.yahoo.elide.ElideSettings; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; -import com.yahoo.elide.core.exceptions.InvalidPredicateException; import com.yahoo.elide.core.exceptions.InvalidValueException; -import com.yahoo.elide.core.filter.dialect.ParseException; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.graphql.containers.ConnectionContainer; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.google.common.collect.Sets; @@ -30,10 +26,8 @@ import graphql.schema.DataFetchingEnvironment; import graphql.schema.GraphQLType; import lombok.extern.slf4j.Slf4j; - import java.util.ArrayDeque; import java.util.ArrayList; -import java.util.Arrays; import java.util.Collections; import java.util.LinkedHashSet; import java.util.List; @@ -43,21 +37,15 @@ import java.util.Queue; import java.util.Set; import java.util.stream.Collectors; - +import javax.validation.constraints.NotNull; import javax.ws.rs.BadRequestException; -import javax.ws.rs.core.MultivaluedHashMap; -import javax.ws.rs.core.MultivaluedMap; /** * Invoked by GraphQL Java to fetch/mutate data from Elide. */ @Slf4j public class PersistentResourceFetcher implements DataFetcher { - private final ElideSettings settings; - - public PersistentResourceFetcher(ElideSettings settings) { - this.settings = settings; - } + public PersistentResourceFetcher() { } /** * Override graphql-java's {@link DataFetcher} get method to execute @@ -157,27 +145,18 @@ private Object fetchObjects(Environment context) { /** * Fetches a root-level entity. - * @param context Context for request * @param requestScope Request scope - * @param entityClass Entity class + * @param projection constructed entityProjection for a class * @param ids List of ids (can be NULL) - * @param sort Sort by ASC/DESC - * @param offset Pagination offset argument - * @param first Pagination first argument - * @param filters Filter params - * @param generateTotals True if page totals should be generated for this type, false otherwise * @return {@link PersistentResource} object(s) */ - public ConnectionContainer fetchObject(Environment context, RequestScope requestScope, Class entityClass, - Optional> ids, Optional sort, - Optional offset, Optional first, - Optional filters, boolean generateTotals) { + public ConnectionContainer fetchObject( + RequestScope requestScope, + EntityProjection projection, + Optional> ids + ) { EntityDictionary dictionary = requestScope.getDictionary(); - String typeName = dictionary.getJsonAliasFor(entityClass); - - Optional pagination = buildPagination(first, offset, generateTotals); - Optional sorting = buildSorting(sort); - Optional filter = buildFilter(typeName, filters, requestScope); + String typeName = dictionary.getJsonAliasFor(projection.getType()); /* fetching a collection */ Set records = ids.map((idList) -> { @@ -186,67 +165,53 @@ public ConnectionContainer fetchObject(Environment context, RequestScope request throw new BadRequestException("Empty list passed to ids"); } - return PersistentResource.loadRecords(entityClass, idList, - filter, sorting, pagination, requestScope); - }).orElseGet(() -> PersistentResource.loadRecords( - entityClass, /* Empty list of IDs */ new ArrayList<>(), filter, sorting, pagination, requestScope - )); + return PersistentResource.loadRecords(projection, idList, requestScope); + }).orElseGet(() -> PersistentResource.loadRecords(projection, new ArrayList<>(), requestScope)); - return new ConnectionContainer(records, pagination, typeName); + return new ConnectionContainer(records, Optional.ofNullable(projection.getPagination()), typeName); } /** * Fetches a relationship for a top-level entity. * - * @param context Request context * @param parentResource Parent object - * @param fieldName Field type + * @param relationship constructed relationship object with entityProjection * @param ids List of ids - * @param offset Pagination offset - * @param first Pagination first - * @param filters Filter string - * @param generateTotals True if page totals should be generated for this type, false otherwise * @return persistence resource object(s) */ - public Object fetchRelationship(Environment context, - PersistentResource parentResource, - String fieldName, - Optional> ids, - Optional offset, - Optional first, - Optional sort, - Optional filters, - boolean generateTotals) { + public Object fetchRelationship( + PersistentResource parentResource, + @NotNull Relationship relationship, + Optional> ids + ) { EntityDictionary dictionary = parentResource.getRequestScope().getDictionary(); - Class entityClass = dictionary.getParameterizedType(parentResource.getObject(), fieldName); - String typeName = dictionary.getJsonAliasFor(entityClass); - - Optional pagination = buildPagination(first, offset, generateTotals); - Optional sorting = buildSorting(sort); - Optional filter = buildFilter(typeName, filters, parentResource.getRequestScope()); + Class relationshipClass = dictionary.getParameterizedType(parentResource.getObject(), relationship.getName()); + String relationshipType = dictionary.getJsonAliasFor(relationshipClass); - Set relations; + Set relationResources; if (ids.isPresent()) { - relations = parentResource.getRelation(fieldName, ids.get(), filter, sorting, pagination); + relationResources = parentResource.getRelation(ids.get(), relationship); } else { - relations = parentResource.getRelationCheckedFiltered(fieldName, - filter, sorting, pagination); + relationResources = parentResource.getRelationCheckedFiltered(relationship); } - return new ConnectionContainer(relations, pagination, typeName); + return new ConnectionContainer( + relationResources, + Optional.ofNullable(relationship.getProjection().getPagination()), + relationshipType); } private ConnectionContainer upsertObjects(Environment context) { return upsertOrUpdateObjects( context, - (entityObject) -> upsertObject(context, entityObject), + (entityObject) -> upsertObject(entityObject), RelationshipOp.UPSERT); } private ConnectionContainer updateObjects(Environment context) { return upsertOrUpdateObjects( context, - (entityObject) -> updateObject(context, entityObject), + (entityObject) -> updateObject(entityObject), RelationshipOp.UPDATE); } @@ -256,9 +221,12 @@ private ConnectionContainer updateObjects(Environment context) { * @param updateFunc controls the behavior of how the update (or upsert) is performed. * @return Connection object. */ - private ConnectionContainer upsertOrUpdateObjects(Environment context, - Executor updateFunc, - RelationshipOp operation) { + + private ConnectionContainer upsertOrUpdateObjects( + Environment context, + Executor updateFunc, + RelationshipOp operation + ) { /* sanity check for id and data argument w UPSERT/UPDATE */ if (context.ids.isPresent()) { throw new BadRequestException(operation + " must not include ids"); @@ -273,14 +241,18 @@ private ConnectionContainer upsertOrUpdateObjects(Environment context, if (context.isRoot()) { entityClass = dictionary.getEntityClass(context.field.getName()); } else { - entityClass = dictionary.getParameterizedType(context.parentResource.getResourceClass(), + assert context.parentResource != null; + entityClass = dictionary.getParameterizedType( + context.parentResource.getResourceClass(), context.field.getName()); } /* form entities */ Optional parentEntity; if (!context.isRoot()) { - parentEntity = Optional.of(new Entity(Optional.empty(), + assert context.parentResource != null; + parentEntity = Optional.of(new Entity( + Optional.empty(), null, context.parentResource.getResourceClass(), context.requestScope)); @@ -303,6 +275,7 @@ private ConnectionContainer upsertOrUpdateObjects(Environment context, PersistentResource childResource = entity.toPersistentResource(); if (!context.isRoot()) { /* add relation between parent and nested entity */ + assert context.parentResource != null; context.parentResource.addRelation(context.field.getName(), childResource); } } @@ -373,53 +346,45 @@ private PersistentResource updateRelationship(Entity entity) { /** * updates or creates existing/new entities - * @param context request context * @param entity Resource entity * @return {@link PersistentResource} object */ - private PersistentResource upsertObject(Environment context, Entity entity) { + private PersistentResource upsertObject(Entity entity) { Set attributes = entity.getAttributes(); Optional id = entity.getId(); RequestScope requestScope = entity.getRequestScope(); - PersistentResource upsertedResource; - PersistentResource parentResource; - if (!entity.getParentResource().isPresent()) { - parentResource = null; - } else { - parentResource = entity.getParentResource().get().toPersistentResource(); - } + PersistentResource upsertedResource; + + PersistentResource parentResource = !entity.getParentResource().isPresent() + ? null + : entity.getParentResource().get().toPersistentResource(); if (!id.isPresent()) { entity.setId(); id = entity.getId(); - upsertedResource = PersistentResource.createObject(parentResource, - entity.getEntityClass(), - requestScope, - id); + + upsertedResource = PersistentResource.createObject( + parentResource, entity.getEntityClass(), requestScope, id); } else { try { - Set loadedResource = fetchObject(context, requestScope, entity.getEntityClass(), - Optional.of(Arrays.asList(id.get())), - Optional.empty(), - Optional.empty(), - Optional.empty(), - Optional.empty(), - false).getPersistentResources(); - upsertedResource = IterableUtils.first(loadedResource); - - //The ID doesn't exist yet. Let's create the object. - } catch (InvalidObjectIdentifierException | InvalidValueException e) { - upsertedResource = PersistentResource.createObject(parentResource, - entity.getEntityClass(), + Set loadedResource = fetchObject( requestScope, - id); + entity.getProjection(), + Optional.of(Collections.singletonList(id.get())) + ).getPersistentResources(); + upsertedResource = loadedResource.iterator().next(); + + // The ID doesn't exist yet. Let's create the object. + } catch (InvalidObjectIdentifierException | InvalidValueException e) { + upsertedResource = PersistentResource.createObject( + parentResource, entity.getEntityClass(), requestScope, id); } } return updateAttributes(upsertedResource, entity, attributes); } - private PersistentResource updateObject(Environment context, Entity entity) { + private PersistentResource updateObject(Entity entity) { Set attributes = entity.getAttributes(); Optional id = entity.getId(); RequestScope requestScope = entity.getRequestScope(); @@ -427,15 +392,14 @@ private PersistentResource updateObject(Environment context, Entity entity) { if (!id.isPresent()) { throw new BadRequestException("UPDATE data objects must include ids"); + } else { + Set loadedResource = fetchObject( + requestScope, + entity.getProjection(), + Optional.of(Collections.singletonList(id.get())) + ).getPersistentResources(); + updatedResource = loadedResource.iterator().next(); } - Set loadedResource = fetchObject(context, requestScope, entity.getEntityClass(), - Optional.of(Arrays.asList(id.get())), - Optional.empty(), - Optional.empty(), - Optional.empty(), - Optional.empty(), - false).getPersistentResources(); - updatedResource = IterableUtils.first(loadedResource); return updateAttributes(updatedResource, entity, attributes); } @@ -552,34 +516,4 @@ private ConnectionContainer replaceObjects(Environment context) { } return upsertedObjects; } - - private Optional buildPagination(Optional first, - Optional offset, - boolean generateTotals) { - return Pagination.fromOffsetAndFirst(first, offset, generateTotals, settings); - } - - private Optional buildSorting(Optional sort) { - return sort.map(Sorting::parseSortRule); - } - - private Optional buildFilter(String typeName, - Optional filter, - RequestScope requestScope) { - // TODO: Refactor FilterDialect interfaces to accept string or List instead of (or in addition to?) - // query params. - return filter.map(filterStr -> { - MultivaluedMap queryParams = new MultivaluedHashMap() { - { - put("filter[" + typeName + "]", Arrays.asList(filterStr)); - } - }; - try { - return requestScope.getFilterDialect().parseTypedExpression(typeName, queryParams).get(typeName); - } catch (ParseException e) { - log.debug("Filter parse exception caught", e); - throw new InvalidPredicateException("Could not parse filter for type: " + typeName); - } - }); - } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java index 6c783159be..ca51898fb1 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java @@ -15,6 +15,8 @@ import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; import com.yahoo.elide.core.exceptions.TransactionException; +import com.yahoo.elide.graphql.parser.GraphQLEntityProjectionMaker; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.security.User; import com.fasterxml.jackson.core.JsonProcessingException; @@ -64,7 +66,7 @@ public class QueryRunner { public QueryRunner(Elide elide) { this.elide = elide; - PersistentResourceFetcher fetcher = new PersistentResourceFetcher(elide.getElideSettings()); + PersistentResourceFetcher fetcher = new PersistentResourceFetcher(); ModelBuilder builder = new ModelBuilder(elide.getElideSettings().getDictionary(), fetcher); this.api = new GraphQL(builder.build()); @@ -142,8 +144,6 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa boolean isVerbose = false; try (DataStoreTransaction tx = elide.getDataStore().beginTransaction()) { final User user = tx.accessUser(principal); - GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, user, elide.getElideSettings()); - isVerbose = requestScope.getPermissionExecutor().isVerbose(); if (!jsonDocument.has(QUERY)) { return ElideResponse.builder() @@ -151,9 +151,21 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa .body("A `query` key is required.") .build(); } - String query = jsonDocument.get(QUERY).asText(); + // get variables from request for constructing entityProjections + Map variables = new HashMap<>(); + if (jsonDocument.has(VARIABLES) && !jsonDocument.get(VARIABLES).isNull()) { + variables = mapper.convertValue(jsonDocument.get(VARIABLES), Map.class); + } + + GraphQLProjectionInfo projectionInfo = + new GraphQLEntityProjectionMaker(elide.getElideSettings(), variables).make(query); + GraphQLRequestScope requestScope = + new GraphQLRequestScope(tx, user, elide.getElideSettings(), projectionInfo); + + isVerbose = requestScope.getPermissionExecutor().isVerbose(); + // Logging all queries. It is recommended to put any private information that shouldn't be logged into // the "variables" section of your query. Variable values are not logged. log.info("Processing GraphQL query:\n{}", query); @@ -166,10 +178,7 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa executionInput.operationName(jsonDocument.get(OPERATION_NAME).asText()); } - if (jsonDocument.has(VARIABLES) && !jsonDocument.get(VARIABLES).isNull()) { - Map variables = mapper.convertValue(jsonDocument.get(VARIABLES), Map.class); - executionInput.variables(variables); - } + executionInput.variables(variables); ExecutionResult result = api.execute(executionInput); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java index 5524f21c4a..05885b4ef4 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java @@ -8,6 +8,7 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; +import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; import lombok.AllArgsConstructor; @@ -29,19 +30,16 @@ public class ConnectionContainer implements GraphQLContainer { // Refers to the type of persistentResources @Getter private final String typeName; - public static final String EDGES_KEYWORD = "edges"; - public static final String PAGE_INFO_KEYWORD = "pageInfo"; - @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { String fieldName = context.field.getName(); - switch (fieldName) { - case EDGES_KEYWORD: + switch (KeyWord.byName(fieldName)) { + case EDGES: return getPersistentResources().stream() .map(EdgesContainer::new) .collect(Collectors.toList()); - case PAGE_INFO_KEYWORD: + case PAGE_INFO: return new PageInfoContainer(this); default: break; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java index 64a1a0d270..1b11c8df4b 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/EdgesContainer.java @@ -5,6 +5,8 @@ */ package com.yahoo.elide.graphql.containers; +import static com.yahoo.elide.graphql.KeyWord.NODE; + import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; @@ -21,14 +23,12 @@ public class EdgesContainer implements PersistentResourceContainer, GraphQLContainer { @Getter private final PersistentResource persistentResource; - private static final String NODE_KEYWORD = "node"; - @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { String fieldName = context.field.getName(); // TODO: Cursor - if (NODE_KEYWORD.equals(fieldName)) { + if (NODE.equals(fieldName)) { return new NodeContainer(context.parentResource); } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java index 5e66afc576..f246851e54 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/NodeContainer.java @@ -5,13 +5,13 @@ */ package com.yahoo.elide.graphql.containers; -import static com.yahoo.elide.graphql.containers.RootContainer.requestContainsPageInfo; - import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.graphql.DeferredId; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.Relationship; import lombok.AllArgsConstructor; import lombok.Getter; @@ -37,7 +37,9 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche String idFieldName = dictionary.getIdFieldName(parentClass); if (dictionary.isAttribute(parentClass, fieldName)) { /* fetch attribute properties */ - Object attribute = context.parentResource.getAttribute(fieldName); + Attribute requested = context.requestScope.getProjectionInfo() + .getAttributeMap().getOrDefault(context.field.getSourceLocation(), null); + Object attribute = context.parentResource.getAttribute(requested); if (attribute instanceof Map) { return ((Map) attribute).entrySet().stream() .map(MapEntryContainer::new) @@ -46,10 +48,18 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche return attribute; } if (dictionary.isRelation(parentClass, fieldName)) { /* fetch relationship properties */ - boolean generateTotals = requestContainsPageInfo(context.field); - return fetcher.fetchRelationship(context, context.parentResource, - fieldName, context.ids, context.offset, context.first, context.sort, context.filters, - generateTotals); + // get the relationship from constructed projections + Relationship relationship = context.requestScope + .getProjectionInfo() + .getRelationshipMap() + .getOrDefault(context.field.getSourceLocation(), null); + + if (relationship == null) { + throw new BadRequestException( + "Relationship doesn't have projection " + context.parentResource.getType() + "." + fieldName); + } + + return fetcher.fetchRelationship(context.parentResource, relationship, context.ids); } if (Objects.equals(idFieldName, fieldName)) { return new DeferredId(context.parentResource); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java index bbbfac9074..4e0bd8b7c3 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java @@ -8,14 +8,13 @@ import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; +import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; import lombok.Getter; - import java.util.List; import java.util.Optional; import java.util.stream.Collectors; - import javax.ws.rs.BadRequestException; /** @@ -24,12 +23,6 @@ public class PageInfoContainer implements GraphQLContainer { @Getter private final ConnectionContainer connectionContainer; - // Page info keywords - private static final String PAGE_INFO_HAS_NEXT_PAGE_KEYWORD = "hasNextPage"; - private static final String PAGE_INFO_START_CURSOR_KEYWORD = "startCursor"; - private static final String PAGE_INFO_END_CURSOR_KEYWORD = "endCursor"; - private static final String PAGE_INFO_TOTAL_RECORDS_KEYWORD = "totalRecords"; - public PageInfoContainer(ConnectionContainer connectionContainer) { this.connectionContainer = connectionContainer; } @@ -46,17 +39,17 @@ public Object processFetch(Environment context, PersistentResourceFetcher fetche .collect(Collectors.toList()); return pagination.map(pageValue -> { - switch (fieldName) { - case PAGE_INFO_HAS_NEXT_PAGE_KEYWORD: { + switch (KeyWord.byName(fieldName)) { + case PAGE_INFO_HAS_NEXT_PAGE: { int numResults = ids.size(); int nextOffset = numResults + pageValue.getOffset(); return nextOffset < pageValue.getPageTotals(); } - case PAGE_INFO_START_CURSOR_KEYWORD: + case PAGE_INFO_START_CURSOR: return pageValue.getOffset(); - case PAGE_INFO_END_CURSOR_KEYWORD: + case PAGE_INFO_END_CURSOR: return pageValue.getOffset() + ids.size(); - case PAGE_INFO_TOTAL_RECORDS_KEYWORD: + case PAGE_INFO_TOTAL_RECORDS: return pageValue.getPageTotals(); default: break; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java index da2780858c..6449e334e7 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/RootContainer.java @@ -5,28 +5,24 @@ */ package com.yahoo.elide.graphql.containers; -import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.PersistentResourceFetcher; -import graphql.language.Field; - /** * Root container for GraphQL requests. */ public class RootContainer implements GraphQLContainer { @Override public Object processFetch(Environment context, PersistentResourceFetcher fetcher) { - EntityDictionary dictionary = context.requestScope.getDictionary(); - Class entityClass = dictionary.getEntityClass(context.field.getName()); - boolean generateTotals = requestContainsPageInfo(context.field); - return fetcher.fetchObject(context, context.requestScope, entityClass, context.ids, - context.sort, context.offset, context.first, context.filters, generateTotals); - } + String entityName = context.field.getName(); + String aliasName = context.field.getAlias(); - public static boolean requestContainsPageInfo(Field field) { - return field.getSelectionSet().getSelections().stream() - .anyMatch(f -> f instanceof Field - && ConnectionContainer.PAGE_INFO_KEYWORD.equals(((Field) f).getName())); + return fetcher.fetchObject( + context.requestScope, + context.requestScope + .getProjectionInfo() + .getProjection(aliasName, entityName), // root-level projection + context.ids + ); } } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java new file mode 100644 index 0000000000..cb00878555 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/FragmentResolver.java @@ -0,0 +1,139 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; + +import graphql.language.Document; +import graphql.language.Field; +import graphql.language.FragmentDefinition; +import graphql.language.FragmentSpread; +import graphql.language.Selection; +import graphql.language.SelectionSet; +import java.util.ArrayList; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.function.Function; +import java.util.stream.Collectors; +import javax.ws.rs.BadRequestException; + +/** + * Class that fetch {@link FragmentDefinition}s from graphQL {@link Document} and store them for future reference. + */ +public class FragmentResolver { + private final Map fragmentMap = new HashMap<>(); + + public boolean contains(String fragmentName) { + return fragmentMap.containsKey(fragmentName); + } + + public FragmentDefinition get(String fragmentName) { + return fragmentMap.get(fragmentName); + } + + /** + * Fetch fragments from documents. Only fragment definitions would be processed. + * + * @param document graphql document + */ + public void addFragments(Document document) { + addFragments(document.getDefinitions().stream() + .filter(definition -> definition instanceof FragmentDefinition) + .map(definition -> (FragmentDefinition) definition) + .collect(Collectors.toList())); + } + + /** + * Make sure there is not fragment loop in in-coming definitions and store those fragments. + * + * @param fragments fragments to add + */ + private void addFragments(List fragments) { + final Map newFragments = fragments.stream() + .collect(Collectors.toMap(FragmentDefinition::getName, Function.identity())); + + // make sure there is no fragment loop and undefined fragments in fragment definitions + final Set fragmentNames = new HashSet<>(); + fragments.forEach(fragmentDefinition -> validateFragment(newFragments, fragmentDefinition, fragmentNames)); + + this.fragmentMap.putAll(newFragments); + } + + /** + * Recursive DFS to validate that there is not reference loop in a fragment and there is not un-defined + * fragments. + * + * @param fragmentDefinition fragment to be checked + * @param fragmentNames fragment names appear in the current check path + */ + private static void validateFragment( + Map fragmentMap, + FragmentDefinition fragmentDefinition, + Set fragmentNames + ) { + String fragmentName = fragmentDefinition.getName(); + if (fragmentNames.contains(fragmentName)) { + throw new InvalidEntityBodyException("There is a fragment definition loop in: {" + + String.join(",", fragmentNames) + "} with " + fragmentName + " duplicated."); + } + + fragmentNames.add(fragmentName); + + getNestedFragments(fragmentDefinition.getSelectionSet()).stream() + .map(FragmentSpread::getName) + .distinct() + .forEach(name -> { + if (!fragmentMap.containsKey(name)) { + throw new InvalidEntityBodyException(String.format("Unknown fragment {%s}.", name)); + } + validateFragment(fragmentMap, fragmentMap.get(name), fragmentNames); + }); + + fragmentNames.remove(fragmentName); + } + + /** + * Get nested fragments from a selection set, skip other graphQL {@link Field}s. + * This is only for fragment loop validation. + * + * @param selectionSet graphql selection set + * @return nested fragments in the selection set + */ + private static List getNestedFragments(SelectionSet selectionSet) { + return selectionSet.getSelections().stream() + .map(FragmentResolver::getNestedFragments) + .reduce(new ArrayList<>(), (a, b) -> { + a.addAll(b); + return a; + }); + } + + /** + * Get nested fragments from a field, skip other graphQL {@link Field}s. + * This is only for fragment loop validation. + * + * @param selection graphql selection + * @return nested fragments in the selection set of this selection + */ + private static List getNestedFragments(Selection selection) { + if (selection instanceof Field) { + return ((Field) selection).getSelectionSet() == null + || ((Field) selection).getSelectionSet().getSelections().isEmpty() + ? new ArrayList<>() + : getNestedFragments(((Field) selection).getSelectionSet()); + } else if (selection instanceof FragmentSpread) { + return Collections.singletonList((FragmentSpread) selection); + } else { + // TODO: support inline fragment + throw new BadRequestException("Unsupported graphQL selection type: " + selection.getClass()); + } + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java new file mode 100644 index 0000000000..e0a48a3af6 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java @@ -0,0 +1,557 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import static com.yahoo.elide.graphql.KeyWord.EDGES; +import static com.yahoo.elide.graphql.KeyWord.NODE; +import static com.yahoo.elide.graphql.KeyWord.PAGE_INFO; +import static com.yahoo.elide.graphql.KeyWord.PAGE_INFO_TOTAL_RECORDS; +import static com.yahoo.elide.graphql.KeyWord.SCHEMA; +import static com.yahoo.elide.graphql.KeyWord.TYPE; +import static com.yahoo.elide.graphql.KeyWord.TYPENAME; + +import com.yahoo.elide.ElideSettings; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.RelationshipType; +import com.yahoo.elide.core.exceptions.InvalidEntityBodyException; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.core.filter.dialect.MultipleFilterDialect; +import com.yahoo.elide.core.filter.dialect.ParseException; +import com.yahoo.elide.core.filter.expression.AndFilterExpression; +import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.graphql.ModelBuilder; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.EntityProjection.EntityProjectionBuilder; +import com.yahoo.elide.request.Relationship; + +import graphql.language.Argument; +import graphql.language.Document; +import graphql.language.Field; +import graphql.language.FragmentDefinition; +import graphql.language.FragmentSpread; +import graphql.language.OperationDefinition; +import graphql.language.Selection; +import graphql.language.SelectionSet; +import graphql.language.SourceLocation; +import graphql.parser.Parser; +import lombok.extern.slf4j.Slf4j; +import java.math.BigInteger; +import java.util.Collections; +import java.util.HashMap; +import java.util.List; +import java.util.Map; +import java.util.Optional; +import java.util.stream.Collectors; + +import javax.ws.rs.BadRequestException; +import javax.ws.rs.core.MultivaluedHashMap; + +/** + * This class converts a GraphQL query string into an Elide {@link EntityProjection} using + * {@link #make(String)} method. + */ +@Slf4j +public class GraphQLEntityProjectionMaker { + private final ElideSettings elideSettings; + private final EntityDictionary entityDictionary; + private final MultipleFilterDialect filterDialect; + + private final VariableResolver variableResolver; + private final FragmentResolver fragmentResolver; + + private final Map relationshipMap = new HashMap<>(); + private final Map rootProjections = new HashMap<>(); + private final Map attributeMap = new HashMap<>(); + + /** + * Constructor. + * + * @param elideSettings settings of current Elide instance + * @param variables variables provided in the request + */ + public GraphQLEntityProjectionMaker(ElideSettings elideSettings, Map variables) { + this.elideSettings = elideSettings; + this.entityDictionary = elideSettings.getDictionary(); + this.filterDialect = new MultipleFilterDialect( + elideSettings.getJoinFilterDialects(), + elideSettings.getSubqueryFilterDialects()); + + this.variableResolver = new VariableResolver(variables); + this.fragmentResolver = new FragmentResolver(); + } + + /** + * Constructor. + * + * @param elideSettings settings of current Elide instance + */ + public GraphQLEntityProjectionMaker(ElideSettings elideSettings) { + this(elideSettings, new HashMap<>()); + } + + /** + * Convert a GraphQL query string into a collection of Elide {@link EntityProjection}s. + * + * @param query GraphQL query + * @return all projections in the query + */ + public GraphQLProjectionInfo make(String query) { + Parser parser = new Parser(); + Document parsedDocument; + try { + parsedDocument = parser.parseDocument(query); + } catch (Exception e) { + throw new InvalidEntityBodyException("Can't parse query: " + query); + } + + // resolve fragment definitions + fragmentResolver.addFragments(parsedDocument); + + // resolve operation definitions + parsedDocument.getDefinitions().forEach(definition -> { + if (definition instanceof OperationDefinition) { + // Operations would be converted into EntityProjection tree + OperationDefinition operationDefinition = (OperationDefinition) definition; + if (operationDefinition.getOperation() == OperationDefinition.Operation.SUBSCRIPTION) { + // TODO: support SUBSCRIPTION + return; + } + + // resolve variable definitions in this operation + variableResolver.newScope(operationDefinition); + + addRootProjection(operationDefinition.getSelectionSet()); + } else if (!(definition instanceof FragmentDefinition)) { + throw new InvalidEntityBodyException( + String.format("Unsupported definition type {%s}.", definition.getClass())); + } + }); + + return new GraphQLProjectionInfo(rootProjections, relationshipMap, attributeMap); + } + + /** + * Root projection would be an operation applied on an single entity class. + * The EntityProjection tree would be constructed recursively to add all child projections. + * + * @param selectionSet a root-level selection set + */ + private void addRootProjection(SelectionSet selectionSet) { + List selections = selectionSet.getSelections(); + + selections.stream().forEach(rootSelection -> { + if (!(rootSelection instanceof Field)) { + throw new InvalidEntityBodyException("Entity selection must be a graphQL field."); + } + Field rootSelectionField = (Field) rootSelection; + String entityName = rootSelectionField.getName(); + String aliasName = rootSelectionField.getAlias(); + if (SCHEMA.equals(entityName) || TYPE.equals(entityName)) { + // '__schema' and '__type' would not be handled by entity projection + return; + } + Class entityType = entityDictionary.getEntityClass(rootSelectionField.getName()); + if (entityType == null) { + throw new InvalidEntityBodyException(String.format("Unknown entity {%s}.", + rootSelectionField.getName())); + } + + + String keyName = GraphQLProjectionInfo.computeProjectionKey(aliasName, entityName); + if (rootProjections.containsKey(keyName)) { + throw new InvalidEntityBodyException( + String.format("Found two root level query for Entity {%s} with same alias name", + entityName)); + } + rootProjections.put(keyName, + createProjection(entityType, rootSelectionField)); + }); + + } + + /** + * Construct an {@link EntityProjection} from a GraphQL {@link Field} for an entity type. + * + * @param entityType type of entity to be projected + * @param entityField graphQL field definition + * @return constructed {@link EntityProjection} + */ + private EntityProjection createProjection(Class entityType, Field entityField) { + final EntityProjectionBuilder projectionBuilder = EntityProjection.builder() + .type(entityType); + + entityField.getSelectionSet().getSelections().forEach(selection -> addSelection(selection, projectionBuilder)); + entityField.getArguments().forEach(argument -> addArgument(argument, projectionBuilder)); + + return projectionBuilder.build(); + } + + /** + * Add a graphQL {@link Selection} to an {@link EntityProjection} + * + * @param fieldSelection field/fragment to add + * @param projectionBuilder projection that is being built + */ + private void addSelection(Selection fieldSelection, final EntityProjectionBuilder projectionBuilder) { + if (fieldSelection instanceof FragmentSpread) { + addFragment((FragmentSpread) fieldSelection, projectionBuilder); + } else if (fieldSelection instanceof Field) { + if (EDGES.equals(((Field) fieldSelection).getName()) + || NODE.equals(((Field) fieldSelection).getName())) { + // if this graphql field is 'edges' or 'node', go one level deeper in the graphql document + ((Field) fieldSelection).getSelectionSet().getSelections().forEach( + selection -> addSelection(selection, projectionBuilder)); + } else { + addField((Field) fieldSelection, projectionBuilder); + } + } else { + throw new InvalidEntityBodyException( + String.format("Unsupported selection type {%s}.", fieldSelection.getClass())); + } + } + + /** + * Resolve a graphQL {@link FragmentSpread} into {@link Selection}s and add them to an {@link EntityProjection} + * + * @param fragment graphQL fragment + * @param projectionBuilder projection that is being built + */ + private void addFragment(FragmentSpread fragment, EntityProjectionBuilder projectionBuilder) { + String fragmentName = fragment.getName(); + + FragmentDefinition fragmentDefinition = fragmentResolver.get(fragmentName); + + // type name in type condition of the Fragment must match the entity projection type name + if (entityDictionary.getJsonAliasFor(projectionBuilder.getType()) + .equals(fragmentDefinition.getTypeCondition().getName())) { + fragmentDefinition.getSelectionSet().getSelections() + .forEach(selection -> addSelection(selection, projectionBuilder)); + } + } + + /** + * Add a new graphQL {@link Field} into an {@link EntityProjection} + * + * @param field graphQL field + * @param projectionBuilder projection that is being built + */ + private void addField(Field field, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String fieldName = field.getName(); + + // this field would either be a relationship field or an attribute field + if (entityDictionary.getRelationshipType(parentType, fieldName) != RelationshipType.NONE) { + // handle the case of a relationship field + addRelationship(field, projectionBuilder); + } else if (TYPENAME.equals(fieldName)) { + // '__typename' would not be handled by entityProjection + return; + } else if (PAGE_INFO.equals(fieldName)) { + // only 'totalRecords' needs to be added into the projection's pagination + if (field.getSelectionSet().getSelections().stream() + .anyMatch(selection -> selection instanceof Field + && PAGE_INFO_TOTAL_RECORDS.equals(((Field) selection).getName()))) { + addPageTotal(projectionBuilder); + } + } else { + addAttributeField(field, projectionBuilder); + } + } + + /** + * Create a relationship with projection and add it to the parent projection. + * + * @param relationshipField graphQL field for a relationship + * @param projectionBuilder projection that is being built + */ + private void addRelationship(Field relationshipField, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String relationshipName = relationshipField.getName(); + String relationshipAlias = + relationshipField.getAlias() == null ? relationshipName : relationshipField.getAlias(); + + final Class relationshipType = entityDictionary.getParameterizedType(parentType, relationshipName); + + // build new entity projection with only entity type and entity dictionary + EntityProjection relationshipProjection = createProjection(relationshipType, relationshipField); + Relationship relationship = Relationship.builder() + .name(relationshipName) + .alias(relationshipAlias) + .projection(relationshipProjection) + .build(); + + relationshipMap.put(relationshipField.getSourceLocation(), relationship); + + // add this relationship projection to its parent projection + projectionBuilder.relationship(relationship); + } + + /** + * Add an attribute to an entity projection. + * + * @param attributeField graphQL field for an attribute + * @param projectionBuilder projection that is being built + */ + private void addAttributeField(Field attributeField, EntityProjectionBuilder projectionBuilder) { + Class parentType = projectionBuilder.getType(); + String attributeName = attributeField.getName(); + String attributeAlias = attributeField.getAlias() == null ? attributeName : attributeField.getAlias(); + + Class attributeType = entityDictionary.getType(parentType, attributeName); + if (attributeType != null) { + Attribute attribute = Attribute.builder() + .type(attributeType) + .name(attributeName) + .alias(attributeAlias) + .arguments( + attributeField.getArguments().stream() + .map(graphQLArgument -> com.yahoo.elide.request.Argument.builder() + .name(graphQLArgument.getName()) + .value( + variableResolver.resolveValue( + graphQLArgument.getValue())) + .build()) + .collect(Collectors.toList())) + .build(); + + projectionBuilder.attribute(attribute); + attributeMap.put(attributeField.getSourceLocation(), attribute); + } else { + throw new InvalidEntityBodyException(String.format( + "Unknown attribute field {%s.%s}.", + entityDictionary.getJsonAliasFor(projectionBuilder.getType()), + attributeName)); + } + } + + /** + * Construct Elide {@link Pagination}, {@link Sorting}, {@link Attribute} from GraphQL {@link Argument} and + * add it to the {@link EntityProjection}. + * + * @param argument graphQL argument + * @param projectionBuilder projection that is being built + */ + private void addArgument(Argument argument, EntityProjectionBuilder projectionBuilder) { + String argumentName = argument.getName(); + + if (isPaginationArgument(argumentName)) { + addPagination(argument, projectionBuilder); + } else if (isSortingArgument(argumentName)) { + addSorting(argument, projectionBuilder); + } else if (ModelBuilder.ARGUMENT_FILTER.equals(argumentName)) { + addFilter(argument, projectionBuilder); + } else if (!ModelBuilder.ARGUMENT_OPERATION.equals(argumentName) + && !(ModelBuilder.ARGUMENT_IDS.equals(argumentName)) + && !(ModelBuilder.ARGUMENT_DATA.equals(argumentName))) { + addAttributeArgument(argument, projectionBuilder); + } + } + + /** + * Returns whether or not a GraphQL argument name corresponding to a pagination argument. + * + * @param argumentName Name key of the GraphQL argument + * + * @return {@code true} if the name equals to {@link ModelBuilder#ARGUMENT_FIRST} or + * {@link ModelBuilder#ARGUMENT_AFTER} + */ + private static boolean isPaginationArgument(String argumentName) { + return ModelBuilder.ARGUMENT_FIRST.equals(argumentName) || ModelBuilder.ARGUMENT_AFTER.equals(argumentName); + } + + /** + * Create a {@link Pagination} object from pagination GraphQL argument and attach it to the building + * {@link EntityProjection}. + * + * @param argument graphQL argument + * @param projectionBuilder projection that is being built + */ + private void addPagination(Argument argument, EntityProjectionBuilder projectionBuilder) { + Pagination pagination = projectionBuilder.getPagination() == null + ? Pagination.getDefaultPagination(elideSettings) + : projectionBuilder.getPagination(); + + Object argumentValue = variableResolver.resolveValue(argument.getValue()); + int value = argumentValue instanceof BigInteger + ? ((BigInteger) argumentValue).intValue() + : Integer.parseInt((String) argumentValue); + if (ModelBuilder.ARGUMENT_FIRST.equals(argument.getName())) { + pagination.setLimit(value); + } else if (ModelBuilder.ARGUMENT_AFTER.equals(argument.getName())) { + pagination.setOffset(value); + } + + projectionBuilder.pagination(pagination); + } + + /** + * Make projection return page total records. + * If the projection already has a pagination, use limit and offset from the existing pagination, + * else use the default pagination vales. + * + * @param projectionBuilder projection that is being built + */ + private void addPageTotal(EntityProjectionBuilder projectionBuilder) { + if (projectionBuilder.getPagination() == null) { + Optional pagination = Pagination.fromOffsetAndFirst( + Optional.empty(), + Optional.empty(), + true, + elideSettings + ); + pagination.ifPresent(projectionBuilder::pagination); + } else { + Optional pagination = Pagination.fromOffsetAndFirst( + Optional.of(String.valueOf(projectionBuilder.getPagination().getLimit())), + Optional.of(String.valueOf(projectionBuilder.getPagination().getOffset())), + true, + elideSettings + ); + pagination.ifPresent(projectionBuilder::pagination); + } + } + + /** + * Returns whether or not a GraphQL argument name corresponding to a sorting argument. + * + * @param argumentName Name key of the GraphQL argument + * + * @return {@code true} if the name equals to {@link ModelBuilder#ARGUMENT_SORT} + */ + private static boolean isSortingArgument(String argumentName) { + return ModelBuilder.ARGUMENT_SORT.equals(argumentName); + } + + /** + * Creates a {@link Sorting} object from sorting GraphQL argument value and attaches it to the entity sorted + * according to the newly created {@link Sorting} object. + * + * @param argument An argument that contains the value of sorting spec + * @param projectionBuilder projection that is being built + */ + private void addSorting(Argument argument, EntityProjectionBuilder projectionBuilder) { + String sortRule = (String) variableResolver.resolveValue(argument.getValue()); + Sorting sorting = Sorting.parseSortRule(sortRule); + + // validate sorting rule + try { + sorting.getValidSortingRules(projectionBuilder.getType(), entityDictionary); + } catch (InvalidValueException e) { + throw new BadRequestException("Invalid sorting clause " + sortRule + + " for type " + entityDictionary.getJsonAliasFor(projectionBuilder.getType())); + } + + projectionBuilder.sorting(sorting); + } + + /** + * Add a new filter expression to the entityProjection + * + * @param argument filter argument + * @param projectionBuilder projection that is being built + */ + private void addFilter(Argument argument, EntityProjectionBuilder projectionBuilder) { + FilterExpression filter = buildFilter( + entityDictionary.getJsonAliasFor(projectionBuilder.getType()), + variableResolver.resolveValue(argument.getValue())); + + if (projectionBuilder.getFilterExpression() != null) { + projectionBuilder.filterExpression( + new AndFilterExpression(projectionBuilder.getFilterExpression(), filter)); + } else { + projectionBuilder.filterExpression(filter); + } + } + + /** + * Construct a filter expression from a string + * + * @param typeName class type name to apply this filter + * @param filterString Elide filter in string format + * @return constructed filter expression + */ + private FilterExpression buildFilter(String typeName, Object filterString) { + if (!(filterString instanceof String)) { + throw new BadRequestException("Filter of type " + typeName + " is not StringValue."); + } + + try { + return filterDialect.parseTypedExpression(typeName, toQueryParams(typeName, filterString)).get(typeName); + } catch (ParseException e) { + log.debug("Filter parse exception caught", e); + throw new InvalidPredicateException("Could not parse filter " + filterString + " for type: " + typeName); + } + } + + /** + * Convert a type name and filter string to a map that mimic query params comes from request. + * + * @param typeName class type name to apply this filter + * @param filterString Elide filter in string format + * @return constructed map + */ + private static MultivaluedHashMap toQueryParams(String typeName, Object filterString) { + return new MultivaluedHashMap() { + { + put("filter[" + typeName + "]", Collections.singletonList((String) filterString)); + } + }; + } + + /** + * Add argument for a field/relationship of an entity + * + * @param argument an argument which name should match a field name/alias + * @param projectionBuilder projection that is being built + */ + private void addAttributeArgument(Argument argument, EntityProjectionBuilder projectionBuilder) { + String argumentName = argument.getName(); + Class entityType = projectionBuilder.getType(); + + Attribute existingAttribute = projectionBuilder.getAttributeByAlias(argumentName); + + com.yahoo.elide.request.Argument elideArgument = com.yahoo.elide.request.Argument.builder() + .name(argumentName) + .value(variableResolver.resolveValue(argument.getValue())) + .build(); + + if (existingAttribute != null) { + // add a new argument to the existing attribute + Attribute toAdd = Attribute.builder() + .type(existingAttribute.getType()) + .name(existingAttribute.getName()) + .alias(existingAttribute.getAlias()) + .argument(elideArgument) + .build(); + + projectionBuilder.attribute(toAdd); + } else { + Class attributeType = entityDictionary.getType(entityType, argumentName); + if (attributeType == null) { + throw new InvalidEntityBodyException( + String.format("Invalid attribute field/alias for argument: {%s}.{%s}", + entityType, + argumentName) + ); + } + + // create a new attribute if this attribute doesn't exist in the projection + Attribute toAdd = Attribute.builder() + .type(attributeType) + .name(argumentName) + .alias(argumentName) + .argument(elideArgument) + .build(); + + projectionBuilder.attribute(toAdd); + } + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java new file mode 100644 index 0000000000..d36d389e33 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLProjectionInfo.java @@ -0,0 +1,37 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; +import graphql.language.SourceLocation; +import lombok.AllArgsConstructor; +import lombok.Getter; + +import java.util.Map; + +/** + * A helper class that contains a collection of root-level entity projections and relationship map constructed from + * a {@link graphql.language.Document}. + */ +@AllArgsConstructor +public class GraphQLProjectionInfo { + @Getter private final Map projections; + + @Getter private final Map relationshipMap; + + @Getter private final Map attributeMap; + + public EntityProjection getProjection(String aliasName, String entityName) { + return projections.get(computeProjectionKey(aliasName, entityName)); + } + + public static String computeProjectionKey(String aliasName, String entityName) { + return (aliasName == null ? "" : aliasName) + ":" + entityName; + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java new file mode 100644 index 0000000000..a2a6e6dc85 --- /dev/null +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/VariableResolver.java @@ -0,0 +1,122 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.graphql.parser; + +import graphql.language.ArrayValue; +import graphql.language.BooleanValue; +import graphql.language.EnumValue; +import graphql.language.FloatValue; +import graphql.language.IntValue; +import graphql.language.NonNullType; +import graphql.language.NullValue; +import graphql.language.ObjectField; +import graphql.language.ObjectValue; +import graphql.language.OperationDefinition; +import graphql.language.StringValue; +import graphql.language.Type; +import graphql.language.Value; +import graphql.language.VariableDefinition; +import graphql.language.VariableReference; + +import java.util.HashMap; +import java.util.Map; +import java.util.stream.Collectors; + +import javax.ws.rs.BadRequestException; + +/** + * Class that contains variables provided in graphql request and can resolve variables based on + * {@link graphql.language.OperationDefinition} scope. + * 1. variables defined in request is global + * 2. variables defined in each operation is operation-scoped + */ +class VariableResolver { + private final Map requestVariables; + private final Map scopeVariables = new HashMap<>(); + + VariableResolver(Map variables) { + this.requestVariables = new HashMap<>(variables); + } + + /** + * Start a new variable scope for operation, clear all variables in the previous scope and add request variables + * into every new scope. + * + * @param operation operation definition + */ + public void newScope(OperationDefinition operation) { + this.scopeVariables.clear(); + this.scopeVariables.putAll(requestVariables); + operation.getVariableDefinitions().forEach(this::addVariable); + } + + /** + * Resolve {@link VariableDefinition} and store result in the variable map. + * We don't need to worry about resolving graphql {@link graphql.language.TypeName} here because Elide-core + * knows the correct type of each field/argument. + * + * @param definition definition to resolve + */ + private void addVariable(VariableDefinition definition) { + Type variableType = definition.getType(); + String variableName = definition.getName(); + Value defaultValue = definition.getDefaultValue(); + + if (defaultValue == null) { + if (variableType instanceof NonNullType && scopeVariables.get(variableName) == null) { + // value of non-null variable must be resolvable + throw new BadRequestException("Undefined non-null variable " + variableName); + } else { + // this would put 'null' for this variable if it is not stored in the map + scopeVariables.put(variableName, scopeVariables.get(variableName)); + } + } else { + if (!scopeVariables.containsKey(variableName)) { + // create a new variable with default value + scopeVariables.put(variableName, resolveValue(defaultValue)); + } + } + } + + /** + * Resolve the real value of a GraphQL {@link Value} object. Use variables in request if necessary. + * + * @param value requested variable value + * @return resolved value of given variable + */ + public Object resolveValue(Value value) { + if (value instanceof BooleanValue) { + return ((BooleanValue) value).isValue(); + } else if (value instanceof EnumValue) { + // TODO + throw new BadRequestException("Enum value is not supported."); + } else if (value instanceof FloatValue) { + return ((FloatValue) value).getValue(); + } else if (value instanceof IntValue) { + return ((IntValue) value).getValue(); + } else if (value instanceof NullValue) { + return null; + } else if (value instanceof StringValue) { + return ((StringValue) value).getValue(); + } else if (value instanceof ObjectValue) { + return ((ObjectValue) value).getObjectFields().stream() + .collect(Collectors.toMap(ObjectField::getName, ObjectField::getValue)); + } else if (value instanceof ArrayValue) { + return ((ArrayValue) value).getValues().stream() + .map(this::resolveValue) + .collect(Collectors.toList()); + } else if (value instanceof VariableReference) { + String variableName = ((VariableReference) value).getName(); + if (!scopeVariables.containsKey(variableName)) { + throw new BadRequestException("Can't resolve variable reference " + variableName); + } + + return scopeVariables.get(variableName); + } + throw new BadRequestException("Unknown variable value type " + value.getClass()); + } +} diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java index 6c395c7e6f..7f8146a610 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherDeleteTest.java @@ -81,6 +81,17 @@ public void testNestedBadInput() { assertQueryFails(graphQLRequest); } + @Test + public void testBadArgument() { + String graphQLRequest = "mutation { " + + "author(unknown: \"1\") { " + + "books(op:DELETE) { " + + "edges { node { id } } " + + "} " + + "} " + + "}"; + assertParsingFails(graphQLRequest); + } @Test public void testNestedSingleId() throws Exception { diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java index d28092f42c..06aa20e348 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherFetchTest.java @@ -8,11 +8,10 @@ import static org.junit.jupiter.api.Assertions.assertTrue; -import com.yahoo.elide.core.DataStoreTransaction; -import com.yahoo.elide.core.RequestScope; import org.junit.jupiter.api.Test; import graphql.ExecutionResult; +import java.util.HashMap; /** * Test the Fetch operation. @@ -27,6 +26,11 @@ public void testRootSingle() throws Exception { runComparisonTest("rootSingle"); } + @Test + public void testRootUnknownField() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/rootUnknownField.graphql")); + } + @Test public void testRootMultipleIds() throws Exception { runComparisonTest("rootMultipleIds"); @@ -52,6 +56,11 @@ public void testRootCollectionSort() throws Exception { runComparisonTest("rootCollectionSort"); } + @Test + public void testRootCollectionInvalidSort() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/rootCollectionInvalidSort.graphql")); + } + @Test public void testRootCollectionMultiSort() throws Exception { runComparisonTest("rootCollectionMultiSort"); @@ -98,10 +107,7 @@ public void testDateLessThanFilter() throws Exception { } @Test - public void testFailuresWithBody() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - + public void testFailuresWithBody() { String graphQLRequest = "{ " + "book(ids: [\"1\"], data: [{\"id\": \"1\"}]) { " + "edges { node { " @@ -110,8 +116,8 @@ public void testFailuresWithBody() throws Exception { + "}}" + "} " + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); - assertTrue(!result.getErrors().isEmpty()); + + assertParsingFails(graphQLRequest); } @Test @@ -145,32 +151,82 @@ public void testComputedAttributes() throws Exception { } @Test - public void testFetchWithFragments() throws Exception { - runComparisonTest("fetchWithFragment"); + public void testFragmentCorrect() throws Exception { + runComparisonTest("fragmentCorrect"); + } + + @Test + public void testFragmentLoop() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentLoop.graphql")); + } + + @Test + public void testFragmentInline() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentInline.graphql")); + } + + @Test + public void testFragmentUnknown() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/fragmentUnknown.graphql")); + } + + @Test + public void testVariableDefinition() throws Exception { + runComparisonTest("variableDefinition"); + } + + @Test + public void testVariableUnknownReference() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/variableUnknownReference.graphql")); } @Test - public void testSchemaIntrospection() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + public void testVariableInvalidNonNull() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/variableInvalidNonNull.graphql")); + } + @Test + public void testAliasAttribute() throws Exception { + runComparisonTest("aliasAttribute"); + } + + @Test + public void testAliasRelationship() throws Exception { + runComparisonTest("aliasRelationship"); + } + + @Test + public void testAliasSameRelationship() throws Exception { + runComparisonTest("aliasSameRelationship"); + } + + @Test + public void testAliasAmbiguous() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/aliasAmbiguous.graphql")); + } + + @Test + public void testAliasPartialQueryAmbiguous() throws Exception { + assertParsingFails(loadGraphQLRequest("fetch/aliasPartialQueryAmbiguous.graphql")); + } + + @Test + public void testSchemaIntrospection() { String graphQLRequest = "{" - + "__schema {" - + "types {" - + " name" - + "}" - + "}" - + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); + + "__schema {" + + "types {" + + " name" + + "}" + + "}" + + "}"; + + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); assertTrue(result.getErrors().isEmpty()); } @Test - public void testTypeIntrospection() throws Exception { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - + public void testTypeIntrospection() { String graphQLRequest = "{" + "__type(name: \"author\") {" + " name" @@ -179,7 +235,7 @@ public void testTypeIntrospection() throws Exception { + " }" + "}" + "}"; - ExecutionResult result = api.execute(graphQLRequest, requestScope); + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); assertTrue(result.getErrors().isEmpty()); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java index 2a1a757435..c4adac8d66 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherRemoveTest.java @@ -54,6 +54,18 @@ public void testNestedBadInput() { assertQueryFails(graphQLRequest); } + @Test + public void testBadArgument() { + String graphQLRequest = "mutation { " + + "author(unknown: \"1\") { " + + "books(op:REMOVE) { " + + "edges { node { id } } " + + "} " + + "} " + + "}"; + assertParsingFails(graphQLRequest); + } + @Test public void testRootCollection() throws Exception { // Part 1: Delete the objects diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java index ef27346e5e..326e65ef2e 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/FetcherReplaceTest.java @@ -5,7 +5,6 @@ */ package com.yahoo.elide.graphql; -import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; public class FetcherReplaceTest extends PersistentResourceFetcherTest { @@ -25,11 +24,9 @@ public void testReplaceEmptyCollections() throws Exception { runComparisonTest("replaceEmptyCollections"); } - // FIXME: Remove stack traces from error handler... - @Disabled + @Test public void testReplaceWithIdsFails() throws Exception { - String expectedMessage = "Exception while fetching data: javax.ws.rs.BadRequestException: REPLACE " - + "must not include ids argument"; + String expectedMessage = "Exception while fetching data (/book) : REPLACE must not include ids argument"; runErrorComparisonTest("replaceWithIdsFails", expectedMessage); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java index 4138940153..396012cf1e 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java @@ -337,22 +337,8 @@ void testPartialResponse() throws IOException, JSONException { ) ).toQuery(); - String expectedData = document( - selection( - field( - "book", - selections( - field("user1SecretField", "null", false), - field("id", "1"), - field("title", "My first book") - ) - ) - ) - ).toResponse(); - Response response = endpoint.post(user2, graphQLRequestToJSON(graphQLRequest)); assertHasErrors(response); - assert200DataEqual(response, expectedData); } @Test @@ -791,6 +777,233 @@ void testQueryAMapWithBadFields() throws IOException { assertHasErrors(response); } + + @Test + public void testMultipleRoot() throws JSONException { + String graphQLRequest = document( + selections( + field( + "author", + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "book", + selections( + field("id"), + field("title"), + field( + "authors", + selection( + field("name") + ) + ) + ) + ) + ) + ).toQuery(); + + String graphQLResponse = document( + selections( + field( + "author", + selections( + field("id", "1"), + field("name", "Ricky Carmichael"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ), + selections( + field("id", "2"), + field("name", "The Silent Author"), + field( + "books", "", false + ) + ) + ), + field( + "book", + selections( + field("id", "1"), + field("title", "My first book"), + field( + "authors", + selection( + field("name", "Ricky Carmichael") + ) + ) + ) + ) + ) + ).toResponse(); + + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest)); + assert200EqualBody(response, graphQLResponse); + } + + @Test + public void testMultipleQueryWithAlias() throws JSONException { + String graphQLRequest = document( + selections( + field( + "AuthorBook", + "author", + selections( + field("id"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "AuthorName", + "author", + selections( + field("id"), + field("name") + ) + ) + ) + ).toQuery(); + String graphQLResponse = document( + selections( + field( + "AuthorBook", + selections( + field("id", "1"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ), + selections( + field("id", "2"), + field( + "books", "", false + ) + ) + ), + field( + "AuthorName", + selections( + field("id", "1"), + field("name", "Ricky Carmichael") + ), + selections( + field("id", "2"), + field("name", "The Silent Author") + ) + ) + ) + ).toResponse(); + + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest)); + assert200EqualBody(response, graphQLResponse); + } + + @Test + public void testMultipleQueryWithAliasAndArguments() throws JSONException { + String graphQLRequest = document( + query( + "myQuery", + variableDefinitions( + variableDefinition("author1", "[String]"), + variableDefinition("author2", "[String]") + ), + selections( + field( + "Author_1", + "author", + arguments( + argument("ids", "$author1") + ), + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ), + field( + "Author_2", + "author", + arguments( + argument("ids", "$author2") + ), + selections( + field("id"), + field("name"), + field( + "books", + selection( + field("title") + ) + ) + ) + ) + ) + ) + ).toQuery(); + String graphQLResponse = document( + selections( + field( + "Author_1", + selections( + field("id", "1"), + field("name", "Ricky Carmichael"), + field( + "books", + selections( + field("title", "My first book") + ) + ) + ) + ), + field( + "Author_2", + selections( + field("id", "2"), + field("name", "The Silent Author"), + field( + "books", "", false + ) + ) + ) + ) + ).toResponse(); + + + Map variables = new HashMap<>(); + variables.put("author1", "1"); + variables.put("author2", "2"); + + Response response = endpoint.post(user1, graphQLRequestToJSON(graphQLRequest, variables)); + assert200EqualBody(response, graphQLResponse); + } + private static String graphQLRequestToJSON(String request) { return graphQLRequestToJSON(request, new HashMap<>()); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java index d3da3f6031..417b4678ab 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLTest.java @@ -21,7 +21,7 @@ /** * Bootstrap for GraphQL tests. */ -public class GraphQLTest { +public abstract class GraphQLTest { protected EntityDictionary dictionary; public GraphQLTest() { diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java index 02d381ce15..2f80c76c29 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java @@ -7,13 +7,16 @@ package com.yahoo.elide.graphql; import static graphql.Assert.assertNotNull; +import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertFalse; import static org.junit.jupiter.api.Assertions.assertNotEquals; import static org.junit.jupiter.api.Assertions.assertNull; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; +import com.yahoo.elide.core.ArgumentType; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.sort.Sorting; import example.Author; import example.Book; @@ -31,6 +34,7 @@ import graphql.schema.GraphQLSchema; import java.util.Collections; +import java.util.HashSet; import java.util.Set; import java.util.stream.Collectors; @@ -189,6 +193,24 @@ public void testBuild() { assertTrue(booksInputType.getWrappedType().equals(bookInputType)); } + @Test + public void checkAttributeArguments() { + Set arguments = new HashSet<>(); + arguments.add(new ArgumentType(SORT, Sorting.SortOrder.class)); + arguments.add(new ArgumentType(TYPE, String.class)); + dictionary.addArgumentsToAttribute(Book.class, PUBLISH_DATE, arguments); + + DataFetcher fetcher = mock(DataFetcher.class); + ModelBuilder builder = new ModelBuilder(dictionary, fetcher); + + GraphQLSchema schema = builder.build(); + + GraphQLObjectType bookType = getConnectedType((GraphQLObjectType) schema.getType(BOOK), null); + assertEquals(2, bookType.getFieldDefinition(PUBLISH_DATE).getArguments().size()); + assertTrue(bookType.getFieldDefinition(PUBLISH_DATE).getArgument(SORT).getType() instanceof GraphQLEnumType); + assertTrue(bookType.getFieldDefinition(PUBLISH_DATE).getArgument(TYPE).getType().equals(Scalars.GraphQLString)); + } + private GraphQLObjectType getConnectedType(GraphQLObjectType root, String connectionName) { GraphQLList edgesType = (GraphQLList) root.getFieldDefinition(EDGES).getType(); GraphQLObjectType rootType = (GraphQLObjectType) diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java index a1bfd44d71..eb55bccb9b 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/PersistentResourceFetcherTest.java @@ -7,25 +7,25 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNotEquals; +import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.fail; import com.yahoo.elide.ElideSettings; import com.yahoo.elide.ElideSettingsBuilder; import com.yahoo.elide.core.DataStoreTransaction; -import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; import com.yahoo.elide.core.datastore.inmemory.InMemoryDataStore; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.graphql.parser.GraphQLEntityProjectionMaker; +import com.yahoo.elide.graphql.parser.GraphQLProjectionInfo; import com.yahoo.elide.utils.coerce.CoerceUtil; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.ObjectMapper; - import example.Author; import example.Book; import example.Pseudonym; import example.Publisher; - import org.apache.tools.ant.util.FileUtils; import org.junit.jupiter.api.AfterEach; import org.junit.jupiter.api.BeforeEach; @@ -36,7 +36,6 @@ import graphql.ExecutionResult; import graphql.GraphQL; import graphql.GraphQLError; - import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; @@ -44,6 +43,7 @@ import java.util.Arrays; import java.util.Collections; import java.util.Date; +import java.util.HashMap; import java.util.HashSet; import java.util.List; import java.util.Map; @@ -85,7 +85,7 @@ public PersistentResourceFetcherTest() { inMemoryDataStore.populateEntityDictionary(dictionary); - ModelBuilder builder = new ModelBuilder(dictionary, new PersistentResourceFetcher(settings)); + ModelBuilder builder = new ModelBuilder(dictionary, new PersistentResourceFetcher()); api = new GraphQL(builder.build()); @@ -171,7 +171,9 @@ protected void assertQueryEquals(String graphQLRequest, String expectedResponse, boolean isMutation = graphQLRequest.startsWith("mutation"); DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + GraphQLProjectionInfo projectionInfo = + new GraphQLEntityProjectionMaker(settings, variables).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); ExecutionResult result = api.execute(graphQLRequest, requestScope, variables); // NOTE: We're forcing commit even in case of failures. GraphQLEndpoint tests should ensure we do not commit on @@ -196,7 +198,8 @@ protected void assertQueryFailsWith(String graphQLRequest, String expectedMessag boolean isMutation = graphQLRequest.startsWith("mutation"); DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); + GraphQLProjectionInfo projectionInfo = new GraphQLEntityProjectionMaker(settings).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); ExecutionResult result = api.execute(graphQLRequest, requestScope); if (isMutation) { @@ -214,10 +217,7 @@ protected void assertQueryFailsWith(String graphQLRequest, String expectedMessag } protected void assertQueryFails(String graphQLRequest) { - DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); - RequestScope requestScope = new GraphQLRequestScope(tx, null, settings); - - ExecutionResult result = api.execute(graphQLRequest, requestScope); + ExecutionResult result = runGraphQLRequest(graphQLRequest, new HashMap<>()); //debug for errors LOG.debug("Errors = [" + errorsToString(result.getErrors()) + "]"); @@ -225,6 +225,18 @@ protected void assertQueryFails(String graphQLRequest) { assertNotEquals(result.getErrors().size(), 0); } + protected void assertParsingFails(String graphQLRequest) { + assertThrows(Exception.class, () -> new GraphQLEntityProjectionMaker(settings).make(graphQLRequest)); + } + + protected ExecutionResult runGraphQLRequest(String graphQLRequest, Map variables) { + DataStoreTransaction tx = inMemoryDataStore.beginTransaction(); + GraphQLProjectionInfo projectionInfo = new GraphQLEntityProjectionMaker(settings).make(graphQLRequest); + GraphQLRequestScope requestScope = new GraphQLRequestScope(tx, null, settings, projectionInfo); + + return api.execute(graphQLRequest, requestScope, variables); + } + protected String errorsToString(List errors) { return errors.stream() .map(GraphQLError::getMessage) diff --git a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java index e78d7ae535..75b21f4f19 100644 --- a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java +++ b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java @@ -28,7 +28,7 @@ import javax.persistence.OneToOne; import javax.persistence.Transient; -@Include +@Include(rootLevel = true) @Entity @CreatePermission(expression = Author.PERMISSION) @ReadPermission(expression = Author.PERMISSION) diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql new file mode 100644 index 0000000000..2b9517d791 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAmbiguous.graphql @@ -0,0 +1,10 @@ +{ + book(ids: ["1"]) { + edges { + node { + alias: id + alias: title + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql new file mode 100644 index 0000000000..2949278f50 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasAttribute.graphql @@ -0,0 +1,9 @@ +{ + book(ids: ["1"]) { + edges { + node { + id_alias: id + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql new file mode 100644 index 0000000000..b11f611aa7 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasPartialQueryAmbiguous.graphql @@ -0,0 +1,23 @@ +{ + book(ids: ["1"]) { + edges { + node { + alias: id + } + } + } + book(ids: ["1"]) { + edges { + node { + alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql new file mode 100644 index 0000000000..742c5d8178 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasRelationship.graphql @@ -0,0 +1,18 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + title + authors_alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql new file mode 100644 index 0000000000..b7b6c5622a --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/aliasSameRelationship.graphql @@ -0,0 +1,26 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + authors_alias: authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fetchWithFragment.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentCorrect.graphql similarity index 100% rename from elide-graphql/src/test/resources/graphql/requests/fetch/fetchWithFragment.graphql rename to elide-graphql/src/test/resources/graphql/requests/fetch/fragmentCorrect.graphql diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql new file mode 100644 index 0000000000..b0d1d28833 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentInline.graphql @@ -0,0 +1,26 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ... on book { + edges { + node { + id + title + } + } + } + __typename + } + __typename + } + __typename + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql new file mode 100644 index 0000000000..b0b40e694e --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentLoop.graphql @@ -0,0 +1,31 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ...BookFields + __typename + } + __typename + } + __typename + } +} + +fragment BookFields on book { + edges { + node { + id + title + authors { + ...AuthorFields + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql new file mode 100644 index 0000000000..5783bf0ca9 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/fragmentUnknown.graphql @@ -0,0 +1,31 @@ +query namedQuery { + author { + ...AuthorFields + } +} + +fragment AuthorFields on author { + edges { + node { + id + books { + ...UnknownFields + __typename + } + __typename + } + __typename + } +} + +fragment BookFields on book { + edges { + node { + id + title + authors { + ...AuthorFields + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql index cceb5ee145..a1a115b58c 100644 --- a/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/nestedCollectionFilter.graphql @@ -1,8 +1,8 @@ -{ +query nestedCollectionFilter($Filter: String = "title==\"Libro U*\"") { author(ids: ["1"]) { edges { node { - books(filter: "title==\"Libro U*\"") { + books(filter: $Filter) { edges { node { id diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql new file mode 100644 index 0000000000..68eae41272 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootCollectionInvalidSort.graphql @@ -0,0 +1,10 @@ +{ + book(sort: "-title|unknown") { + edges { + node { + id + title + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql new file mode 100644 index 0000000000..c98de93477 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootMultiple.graphql @@ -0,0 +1,16 @@ +{ + book { + edges { + node { + id + } + } + } + author { + edges { + node { + id + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql new file mode 100644 index 0000000000..5a84ed8bd0 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/rootUnknownField.graphql @@ -0,0 +1,18 @@ +{ + book(ids: ["1"]) { + edges { + node { + id + unknown + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql new file mode 100644 index 0000000000..fdcf2e461c --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableDefinition.graphql @@ -0,0 +1,18 @@ +query testBook($ids : [String] = ["1"]) { + book(ids: $ids) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql new file mode 100644 index 0000000000..3143220866 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableInvalidNonNull.graphql @@ -0,0 +1,18 @@ +query testBook($ids : [String]!) { + book(ids: $ids) { + edges { + node { + id + title + authors { + edges { + node { + id + name + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql b/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql new file mode 100644 index 0000000000..f7597e6fee --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/requests/fetch/variableUnknownReference.graphql @@ -0,0 +1,16 @@ +{ + author(ids: ["1"]) { + edges { + node { + books(filter: $UnknownFilter) { + edges { + node { + id + title + } + } + } + } + } + } +} diff --git a/elide-graphql/src/test/resources/graphql/requests/replace/replaceWithidsFails.graphql b/elide-graphql/src/test/resources/graphql/requests/replace/replaceWithIdsFails.graphql similarity index 100% rename from elide-graphql/src/test/resources/graphql/requests/replace/replaceWithidsFails.graphql rename to elide-graphql/src/test/resources/graphql/requests/replace/replaceWithIdsFails.graphql diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json new file mode 100644 index 0000000000..5c6d4a43c9 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasAttribute.json @@ -0,0 +1,11 @@ +{ + "book": { + "edges": [ + { + "node": { + "id_alias": "1" + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json new file mode 100644 index 0000000000..d4187ec800 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasRelationship.json @@ -0,0 +1,22 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors_alias": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json new file mode 100644 index 0000000000..b29162a650 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/aliasSameRelationship.json @@ -0,0 +1,32 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + }, + "authors_alias": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/fetchWithFragment.json b/elide-graphql/src/test/resources/graphql/responses/fetch/fragmentCorrect.json similarity index 100% rename from elide-graphql/src/test/resources/graphql/responses/fetch/fetchWithFragment.json rename to elide-graphql/src/test/resources/graphql/responses/fetch/fragmentCorrect.json diff --git a/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json b/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json new file mode 100644 index 0000000000..7138cfd022 --- /dev/null +++ b/elide-graphql/src/test/resources/graphql/responses/fetch/variableDefinition.json @@ -0,0 +1,22 @@ +{ + "book": { + "edges": [ + { + "node": { + "id": "1", + "title": "Libro Uno", + "authors": { + "edges": [ + { + "node": { + "id": "1", + "name": "Mark Twain" + } + } + ] + } + } + } + ] + } +} diff --git a/elide-integration-tests/pom.xml b/elide-integration-tests/pom.xml index 84f388eb31..497a138ded 100644 --- a/elide-integration-tests/pom.xml +++ b/elide-integration-tests/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -25,7 +25,7 @@ - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java index b97332f7d0..6d3029bea8 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorObjectsIT.java @@ -144,7 +144,7 @@ public void graphQLMutationError() { @Test public void graphQLFetchError() { String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); + String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json"); given() .contentType(GRAPHQL_CONTENT_TYPE) .accept(GRAPHQL_CONTENT_TYPE) diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java index 8c61529bb7..c058a137ab 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/EncodedErrorResponsesIT.java @@ -149,7 +149,7 @@ public void graphQLMutationError() { @Test public void graphQLFetchError() { String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); + String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json"); given() .contentType(GRAPHQL_CONTENT_TYPE) .accept(GRAPHQL_CONTENT_TYPE) diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java index a8d5b7b1a5..38ca669385 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/errorEncodingTests/VerboseEncodedErrorResponsesIT.java @@ -145,18 +145,4 @@ public void graphQLMutationError() { .statusCode(HttpStatus.SC_OK) .body(equalTo(expected)); } - - @Test - public void graphQLFetchError() { - String request = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.req.json"); - String expected = jsonParser.getJson("/EncodedErrorResponsesIT/graphQLFetchError.json"); - given() - .contentType(GRAPHQL_CONTENT_TYPE) - .accept(GRAPHQL_CONTENT_TYPE) - .body(request) - .post("/graphQL") - .then() - .statusCode(HttpStatus.SC_OK) - .body(equalTo(expected)); - } } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java index 9f367bf4eb..91a91fdd32 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/AbstractApiResourceInitializer.java @@ -57,6 +57,7 @@ public final void setUpServer() throws Exception { String restassuredPort = System.getProperty("restassured.port", System.getenv("restassured.port")); RestAssured.port = Integer.parseInt(StringUtils.isNotEmpty(restassuredPort) ? restassuredPort : "9999"); + RestAssured.enableLoggingOfRequestAndResponseIfValidationFails(); // embedded jetty server server = new Server(RestAssured.port); @@ -70,12 +71,6 @@ public final void setUpServer() throws Exception { servletHolder.setInitParameter("jersey.config.server.provider.packages", packageName); servletHolder.setInitParameter("javax.ws.rs.Application", resourceConfig); - ServletHolder graphqlServlet = servletContextHandler.addServlet(ServletContainer.class, "/graphQL/*"); - graphqlServlet.setInitOrder(2); - graphqlServlet.setInitParameter("jersey.config.server.provider.packages", - com.yahoo.elide.graphql.GraphQLEndpoint.class.getPackage().getName()); - graphqlServlet.setInitParameter("javax.ws.rs.Application", resourceConfig); - log.debug("...Starting Server..."); server.start(); } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java index 9d0a3a8199..cc1720f2d7 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/EncodedErrorResponsesTestBinder.java @@ -50,11 +50,14 @@ public EncodedErrorResponsesTestBinder(final AuditLogger auditLogger, ServiceLoc @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java index 0b30ca9edf..fa00775443 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsIntegrationTestApplicationResourceConfig.java @@ -7,13 +7,18 @@ import com.yahoo.elide.audit.TestAuditLogger; +import org.glassfish.hk2.api.ServiceLocator; import org.glassfish.jersey.server.ResourceConfig; +import javax.inject.Inject; + /** * Resource configuration for error objects integration tests. */ public class ErrorObjectsIntegrationTestApplicationResourceConfig extends ResourceConfig { - public ErrorObjectsIntegrationTestApplicationResourceConfig() { - register(new ErrorObjectsTestBinder(new TestAuditLogger())); + + @Inject + public ErrorObjectsIntegrationTestApplicationResourceConfig(ServiceLocator injector) { + register(new ErrorObjectsTestBinder(new TestAuditLogger(), injector)); } } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java index 74b4f76d96..658c1267b6 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/ErrorObjectsTestBinder.java @@ -19,6 +19,7 @@ import example.TestCheckMappings; import org.glassfish.hk2.api.Factory; +import org.glassfish.hk2.api.ServiceLocator; import org.glassfish.hk2.utilities.binding.AbstractBinder; import java.util.Arrays; @@ -28,18 +29,23 @@ */ public class ErrorObjectsTestBinder extends AbstractBinder { private final AuditLogger auditLogger; + private final ServiceLocator injector; - public ErrorObjectsTestBinder(final AuditLogger auditLogger) { + public ErrorObjectsTestBinder(final AuditLogger auditLogger, ServiceLocator injector) { this.auditLogger = auditLogger; + this.injector = injector; } @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java index dcd1a0a63e..55ce4ccb9b 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/IntegrationTest.java @@ -80,7 +80,7 @@ protected IntegrationTest(final Class resourceConfig, } } - private DataStoreTestHarness createHarness() { + protected DataStoreTestHarness createHarness() { try { final String dataStoreSupplierName = System.getProperty("dataStoreHarness"); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java index 993f3a2f95..c308c37954 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/initialization/StandardTestBinder.java @@ -24,6 +24,7 @@ import org.glassfish.hk2.utilities.binding.AbstractBinder; import java.util.Arrays; +import java.util.Calendar; /** * Typical-use test binder for integration test resource configs. @@ -41,11 +42,14 @@ public StandardTestBinder(final AuditLogger auditLogger, final ServiceLocator in @Override protected void configure() { + EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); + + bind(dictionary).to(EntityDictionary.class); + // Elide instance bindFactory(new Factory() { @Override public Elide provide() { - EntityDictionary dictionary = new EntityDictionary(TestCheckMappings.MAPPINGS, injector::inject); DefaultFilterDialect defaultFilterStrategy = new DefaultFilterDialect(dictionary); RSQLFilterDialect rsqlFilterStrategy = new RSQLFilterDialect(dictionary); @@ -59,6 +63,7 @@ public Elide provide() { .withJoinFilterDialect(multipleFilterStrategy) .withSubqueryFilterDialect(multipleFilterStrategy) .withEntityDictionary(dictionary) + .withISO8601Dates("yyyy-MM-dd'T'HH:mm'Z'", Calendar.getInstance().getTimeZone()) .build()); } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java index 1cb131c2d5..f91cd83297 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/GraphQLIT.java @@ -37,7 +37,6 @@ import java.util.ArrayList; import java.util.Arrays; import java.util.Collection; -import java.util.Collections; import java.util.HashMap; import java.util.Map; import javax.ws.rs.core.MediaType; @@ -360,150 +359,6 @@ public void runUpdateAndFetchDifferentTransactionsBatch() throws IOException { ); } - @Test - public void runMultipleRequestsSameTransaction() throws IOException { - // This test demonstrates that multiple roots can be manipulated within a _single_ transaction - - String graphQLRequest = document( - selections( - field( - "book", - argument( - argument( - "ids", - Arrays.asList("1") - ) - ), - selections( - field("id"), - field("title"), - field( - "authors", - selections( - field("id"), - field("name") - ) - ) - ) - ), - field( - "author", - selections( - field("id"), - field("name") - ) - ) - ) - ).toQuery(); - - String expectedResponse = document( - selections( - field( - "book", - selections( - field("id", "1"), - field("title", "1984"), - field( - "authors", - selections( - field("id", "1"), - field("name", "George Orwell") - ) - ) - ) - ), - field( - "author", - selections( - field("id", "1"), - field("name", "George Orwell") - ) - ) - ) - ).toResponse(); - - runQueryWithExpectedResult(graphQLRequest, expectedResponse); - } - - @Test - public void runMultipleRequestsSameTransactionMutation() throws IOException { - // This test demonstrates that multiple roots can be manipulated within a _single_ transaction - // and results are consistent across a mutation. - Author author = new Author(); - author.setId(2L); - author.setName("Stephen King"); - - String graphQLRequest = document( - mutation( - selections( - field( - "book", - argument( - argument( - "ids", - Collections.singletonList("1") - ) - ), - selections( - field("id"), - field("title"), - field( - "authors", - arguments( - argument("op", "UPSERT"), - argument("data", author) - ), - selections( - field("id"), - field("name") - ) - ) - ) - ), - field( - "author", - selections( - field("id"), - field("name") - ) - ) - ) - ) - ).toQuery(); - - String expectedResponse = document( - selections( - field( - "book", - selections( - field("id", "1"), - field("title", "1984"), - field( - "authors", - selections( - field("id", "2"), - field("name", "Stephen King") - ) - ) - ) - ), - field( - "author", - selections( - field("id", "1"), - field("name", "George Orwell") - ), - selections( - field("id", "2"), - field("name", "Stephen King") - ) - ) - ) - ).toResponse(); - - runQueryWithExpectedResult(graphQLRequest, expectedResponse); - } - @Test public void runMultipleRequestsSameTransactionWithAliases() throws IOException { // This test demonstrates that multiple roots can be manipulated within a _single_ transaction diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java index c55f7442ae..94b8a85bf7 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java @@ -48,6 +48,7 @@ import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.initialization.IntegrationTest; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.executors.BypassPermissionExecutor; import com.yahoo.elide.utils.JsonParser; @@ -75,7 +76,6 @@ import java.io.IOException; import java.util.HashMap; import java.util.HashSet; -import java.util.Optional; import java.util.Set; import java.util.stream.Stream; @@ -915,7 +915,7 @@ public void testGetIncludeBadRelation() { .accept(JSONAPI_CONTENT_TYPE) .get("/parent/1?include=children.BadRelation") .then() - .statusCode(HttpStatus.SC_NOT_FOUND); + .statusCode(HttpStatus.SC_BAD_REQUEST); } @Test @@ -2676,7 +2676,12 @@ public void testSpecialCharacterLikeQueryHQL(FilterPredicate filterPredicate, in when(scope.getDictionary()).thenReturn(dictionary); Pagination pagination = mock(Pagination.class); when(pagination.isGenerateTotals()).thenReturn(true); - tx.loadObjects(Book.class, Optional.of(filterPredicate), Optional.empty(), Optional.of(pagination), scope); + tx.loadObjects(EntityProjection.builder() + .type(Book.class) + + .filterExpression(filterPredicate) + .pagination(pagination) + .build(), scope); tx.commit(scope); tx.close(); verify(pagination).setPageTotals(noOfRecords); diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json deleted file mode 100644 index 65e8020780..0000000000 --- a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchError.json +++ /dev/null @@ -1,10 +0,0 @@ -{ - "errors":[ - { - "message":"Invalid Syntax", - "locations":[ - {"line":1,"column":43} - ] - } - ] -} diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json new file mode 100644 index 0000000000..3a66d8d63d --- /dev/null +++ b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorObjectEncoded.json @@ -0,0 +1,7 @@ +{ + "errors":[ + { + "message": "InvalidEntityBodyException: Bad Request Body'Can't parse query: {invoice(sort: "<script>"){edges{node{id}}}'" + } + ] +} diff --git a/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json new file mode 100644 index 0000000000..85019c586a --- /dev/null +++ b/elide-integration-tests/src/test/resources/EncodedErrorResponsesIT/graphQLFetchErrorResponseEncoded.json @@ -0,0 +1,5 @@ +{ + "errors":[ + "InvalidEntityBodyException: Bad Request Body'Can't parse query: {invoice(sort: "<script>"){edges{node{id}}}'" + ] +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/pom.xml b/elide-spring/elide-spring-boot-autoconfigure/pom.xml index 2219b63a94..9b567cbefe 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/pom.xml +++ b/elide-spring/elide-spring-boot-autoconfigure/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-autoconfigure - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Spring Boot Autoconfigure Elide Spring Boot Autoconfigure @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -54,35 +54,42 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-graphql - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-annotations - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT + true + + + + com.yahoo.elide + elide-datastore-aggregation + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-datastore-jpa - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true com.yahoo.elide elide-swagger - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT true @@ -144,7 +151,7 @@ com.yahoo.elide elide-test-helpers - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java index 55d9b38fbb..29a10bf969 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java @@ -13,8 +13,13 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; +import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; import com.yahoo.elide.datastores.jpa.JpaDataStore; import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; +import com.yahoo.elide.datastores.multiplex.MultiplexManager; import org.springframework.beans.factory.config.AutowireCapableBeanFactory; import org.springframework.boot.autoconfigure.condition.ConditionalOnMissingBean; @@ -96,11 +101,19 @@ public T instantiate(Class cls) { */ @Bean @ConditionalOnMissingBean - public DataStore buildDataStore(EntityManagerFactory entityManagerFactory) throws ClassNotFoundException { + public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, + QueryEngineFactory queryEngineFactory, + ElideConfigProperties settings) throws ClassNotFoundException { + MetaDataStore metaDataStore = new MetaDataStore(); - return new JpaDataStore( + AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + + JpaDataStore jpaDataStore = new JpaDataStore( () -> { return entityManagerFactory.createEntityManager(); }, (em -> { return new NonJtaTransaction(em); })); + + // meta data store needs to be put at first to populate meta data models + return new MultiplexManager(jpaDataStore, metaDataStore, aggregationDataStore); } /** @@ -122,4 +135,15 @@ public Swagger buildSwagger(EntityDictionary dictionary, ElideConfigProperties s return swagger; } + + /** + * Configure the QueryEngineFactory that the Aggregation Data Store uses. + * @param entityManagerFactory Needed by the SQLQueryEngine + * @return a SQLQueryEngineFactory + */ + @Bean + @ConditionalOnMissingBean + public QueryEngineFactory buildQueryEngineFactory(EntityManagerFactory entityManagerFactory) { + return new SQLQueryEngineFactory(entityManagerFactory); + } } diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java new file mode 100644 index 0000000000..df6feca4b6 --- /dev/null +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/aggregation/Stats.java @@ -0,0 +1,42 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.spring.models.aggregation; + +import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; +import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; +import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlSum; +import lombok.EqualsAndHashCode; +import lombok.ToString; + +import javax.persistence.Id; + +@Include(rootLevel = true) +@Cardinality(size = CardinalitySize.LARGE) +@EqualsAndHashCode +@ToString +@FromTable(name = "stats") +public class Stats { + + /** + * PK. + */ + @Id + private String id; + + /** + * A metric. + */ + @MetricAggregation(function = SqlSum.class) + private long measure; + + /** + * A degenerate dimension. + */ + private String dimension; +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java similarity index 95% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java index 0277a2eebf..30b846bd50 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactGroup.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactGroup.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java similarity index 94% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java index 7c75d6d5d0..73158a14ec 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactProduct.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactProduct.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java similarity index 92% rename from elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java rename to elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java index 8ffa0cd6e6..b31c64ec2e 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/ArtifactVersion.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/models/jpa/ArtifactVersion.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.spring.models; +package com.yahoo.elide.spring.models.jpa; import com.yahoo.elide.annotation.Include; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java new file mode 100644 index 0000000000..ec6395e513 --- /dev/null +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java @@ -0,0 +1,54 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.spring.tests; + +import static com.jayway.restassured.RestAssured.when; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attr; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attributes; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.data; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.id; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.resource; +import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.type; +import static org.hamcrest.Matchers.equalTo; + +import com.yahoo.elide.core.HttpStatus; + +import org.junit.jupiter.api.Test; +import org.springframework.test.context.jdbc.Sql; + +/** + * Example functional tests for Aggregation Store. + */ +public class AggregationStoreTest extends IntegrationTest { + /** + * This test demonstrates an example test using the aggregation store. + */ + @Test + @Sql(statements = { + "DROP TABLE Stats IF EXISTS;", + "CREATE TABLE Stats(id int, measure int, dimension VARCHAR(255));", + "INSERT INTO Stats (id, measure, dimension) VALUES\n" + + "\t\t(1,100,'Foo')," + + "\t\t(2,150,'Bar');" + }) + public void jsonApiGetTest() { + when() + .get("/json/stats?fields[stats]=measure") + .then() + .body(equalTo( + data( + resource( + type("stats"), + id("0"), + attributes( + attr("measure", 250) + ) + ) + ).toJSON()) + ) + .statusCode(HttpStatus.SC_OK); + } +} diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java index 5adf672a5f..f89f863ad4 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/ControllerTest.java @@ -29,7 +29,8 @@ import com.yahoo.elide.contrib.testhelpers.graphql.GraphQLDSL; import com.yahoo.elide.core.HttpStatus; import com.yahoo.elide.spring.controllers.JsonApiController; -import com.yahoo.elide.spring.models.ArtifactGroup; +import com.yahoo.elide.spring.models.jpa.ArtifactGroup; + import org.junit.jupiter.api.Test; import org.springframework.test.context.jdbc.Sql; import org.springframework.test.context.jdbc.SqlMergeMode; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml b/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml index a108b0f6ea..2af8bdec42 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/resources/application.yaml @@ -2,6 +2,7 @@ server: port: 4001 elide: + modelPackage: 'com.yahoo.elide.spring.models' json-api: path: /json enabled: true @@ -25,7 +26,6 @@ spring: naming: physical-strategy: 'org.hibernate.boot.model.naming.PhysicalNamingStrategyStandardImpl' ddl-auto: 'create' - datasource: url: 'jdbc:h2:mem:db1;DB_CLOSE_DELAY=-1' username: 'sa' diff --git a/elide-spring/elide-spring-boot-starter/pom.xml b/elide-spring/elide-spring-boot-starter/pom.xml index 73d736998c..1f52a73450 100644 --- a/elide-spring/elide-spring-boot-starter/pom.xml +++ b/elide-spring/elide-spring-boot-starter/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-starter - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Spring Boot Starter Elide Spring Boot Starter @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -50,31 +50,37 @@ com.yahoo.elide elide-core - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-graphql - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-annotations - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-datastore-jpa - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT + + + + com.yahoo.elide + elide-datastore-aggregation + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-swagger - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -97,7 +103,7 @@ com.yahoo.elide elide-spring-boot-autoconfigure - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -114,6 +120,24 @@ + + + org.eclipse.jetty + jetty-servlets + ${version.jetty} + + + org.eclipse.jetty.websocket + websocket-server + ${version.jetty} + + + org.eclipse.jetty.websocket + javax-websocket-server-impl + ${version.jetty} + + + org.springframework.boot diff --git a/elide-spring/pom.xml b/elide-spring/pom.xml index c84868ff82..359b3e0c4d 100644 --- a/elide-spring/pom.xml +++ b/elide-spring/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT diff --git a/elide-standalone/pom.xml b/elide-standalone/pom.xml index 55f9b56e18..b572f98879 100644 --- a/elide-standalone/pom.xml +++ b/elide-standalone/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-standalone - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT jar Elide Standalone Elide Standalone Application @@ -16,7 +16,7 @@ com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -61,17 +61,17 @@ com.yahoo.elide elide-datastore-jpa - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-graphql - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-swagger - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT @@ -201,7 +201,7 @@ com.yahoo.elide elide-test-helpers - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT test diff --git a/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java b/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java index f3ba3533bf..78e400c899 100644 --- a/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java +++ b/elide-standalone/src/main/java/com/yahoo/elide/standalone/ElideStandalone.java @@ -111,7 +111,7 @@ public void start(boolean block) throws Exception { if (elideStandaloneSettings.enableGraphQL()) { ServletHolder jerseyServlet = context.addServlet(ServletContainer.class, - elideStandaloneSettings.getGraphQLApiPathSepc()); + elideStandaloneSettings.getGraphQLApiPathSpec()); jerseyServlet.setInitOrder(0); jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.yahoo.elide.graphql"); jerseyServlet.setInitParameter("javax.ws.rs.Application", ElideResourceConfig.class.getCanonicalName()); @@ -135,7 +135,7 @@ public void start(boolean block) throws Exception { if (!elideStandaloneSettings.enableSwagger().isEmpty()) { ServletHolder jerseyServlet = context.addServlet(ServletContainer.class, - elideStandaloneSettings.getSwaggerPathSepc()); + elideStandaloneSettings.getSwaggerPathSpec()); jerseyServlet.setInitOrder(0); jerseyServlet.setInitParameter("jersey.config.server.provider.packages", "com.yahoo.elide.contrib.swagger.resources"); diff --git a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java index e96edc3ad8..66d05be847 100644 --- a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java +++ b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java @@ -91,7 +91,7 @@ public T instantiate(Class cls) { .withSubqueryFilterDialect(new RSQLFilterDialect(dictionary)) .withAuditLogger(getAuditLogger()); - if (enableIS06081Dates()) { + if (enableISO8601Dates()) { builder = builder.withISO8601Dates("yyyy-MM-dd'T'HH:mm'Z'", TimeZone.getTimeZone("UTC")); } @@ -148,8 +148,8 @@ default String getJsonApiPathSpec() { * * @return Default: /graphql/api/v1 */ - default String getGraphQLApiPathSepc() { - return "/graphql/api/v1"; + default String getGraphQLApiPathSpec() { + return "/graphql/api/v1/*"; } @@ -158,7 +158,7 @@ default String getGraphQLApiPathSepc() { * * @return Default: /swagger/* */ - default String getSwaggerPathSepc() { + default String getSwaggerPathSpec() { return "/swagger/*"; } @@ -184,7 +184,7 @@ default boolean enableGraphQL() { * Whether Dates should be ISO8601 strings (true) or epochs (false). * @return */ - default boolean enableIS06081Dates() { + default boolean enableISO8601Dates() { return true; } diff --git a/pom.xml b/pom.xml index 4778c09435..b062fe89df 100644 --- a/pom.xml +++ b/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-parent-pom - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT pom Elide: Parent Pom Parent pom for Elide project @@ -103,12 +103,12 @@ com.yahoo.elide elide-annotations - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT com.yahoo.elide elide-example-models - 4.5.14-SNAPSHOT + 5.0.0-pr6-SNAPSHOT org.projectlombok From a53db70cce2fb78e602a1bdc6a0d5917bd4b2bb0 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Sat, 18 Jan 2020 16:41:18 -0600 Subject: [PATCH 02/16] Refactoring Elide Security Checks (#1144) --- .../yahoo/elide/security/checks/Check.java | 32 +-- .../elide/security/checks/CommitCheck.java | 25 -- .../elide/security/checks/InlineCheck.java | 16 -- .../elide/security/checks/OperationCheck.java | 21 +- .../elide/security/checks/UserCheck.java | 19 +- .../security/checks/prefab/Collections.java | 6 +- .../elide/security/checks/prefab/Common.java | 3 +- .../expression/CanPaginateVisitor.java | 4 +- .../PermissionToFilterExpressionVisitor.java | 2 +- .../elide/security/FilterExpressionCheck.java | 10 +- .../expressions/CheckExpression.java | 11 +- .../elide/core/utils/ClassScannerTest.java | 2 +- .../PermissionExpressionVisitorTest.java | 2 +- .../security/PermissionExecutorTest.java | 261 +++++++++--------- elide-core/src/test/java/example/Child.java | 25 +- .../java/example/NegativeChildIdCheck.java | 5 - .../example/NegativeIntegerUserCheck.java | 5 - elide-core/src/test/java/example/Parent.java | 32 +-- .../test/java/example/TestCheckMappings.java | 9 +- .../src/test/java/example/UserIdChecks.java | 20 -- .../security/CommitChecks.java | 4 +- .../src/test/java/example/Child.java | 18 +- .../src/test/java/example/NoCommitEntity.java | 4 +- .../src/test/java/example/Parent.java | 19 +- .../test/java/example/TestCheckMappings.java | 2 - .../src/test/java/example/User.java | 5 - 26 files changed, 185 insertions(+), 377 deletions(-) delete mode 100644 elide-annotations/src/main/java/com/yahoo/elide/security/checks/CommitCheck.java delete mode 100644 elide-annotations/src/main/java/com/yahoo/elide/security/checks/InlineCheck.java diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/Check.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/Check.java index 6a18e17cb4..d2fd64d6c7 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/Check.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/Check.java @@ -4,39 +4,9 @@ * See LICENSE file in project root for terms. */ package com.yahoo.elide.security.checks; - -import com.yahoo.elide.security.ChangeSpec; -import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.User; - -import java.util.Optional; - /** * Custom security access that verifies whether a user belongs to a role. * Permissions are assigned as a set of checks that grant access to the permission. - * @param Type of record for Check */ -public interface Check { - - /** - * Determines whether the user can access the resource. - * - * @param object Fully modified object - * @param requestScope Request scope object - * @param changeSpec Summary of modifications - * @return true if security check passed - */ - boolean ok(T object, RequestScope requestScope, Optional changeSpec); - - /** - * Method reserved for user checks. - * - * @param user User to check - * @return True if user check passes, false otherwise - */ - boolean ok(User user); - - default String checkIdentifier() { - return this.getClass().getName(); - } +public interface Check { } diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/CommitCheck.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/CommitCheck.java deleted file mode 100644 index e62d3909aa..0000000000 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/CommitCheck.java +++ /dev/null @@ -1,25 +0,0 @@ -/* - * Copyright 2016, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.security.checks; - -import com.yahoo.elide.security.User; - -/** - * Commit check interface. - * @see Check - * - * Commit checks are run immediately before a transaction is about to commit but after all changes have been made. - * Objects passed to this check are guaranteed to be in their final state. - * - * @param Type parameter - */ -public abstract class CommitCheck implements Check { - /* NOTE: Operation checks and user checks are intended to be _distinct_ */ - @Override - public final boolean ok(User user) { - throw new UnsupportedOperationException(); - } -} diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/InlineCheck.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/InlineCheck.java deleted file mode 100644 index 8f9c38cb3a..0000000000 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/InlineCheck.java +++ /dev/null @@ -1,16 +0,0 @@ -/* - * Copyright 2016, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.security.checks; - -/** - * Intermediate check representing the hierarchical structure of checks. - * For instance, Read/Delete permissions can take any type of InlineCheck - * while Create/Update permissions can be of any Check type. - * - * @param type parameter - */ -public abstract class InlineCheck implements Check { -} diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/OperationCheck.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/OperationCheck.java index d02472239b..8b72185f13 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/OperationCheck.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/OperationCheck.java @@ -5,7 +5,10 @@ */ package com.yahoo.elide.security.checks; -import com.yahoo.elide.security.User; +import com.yahoo.elide.security.ChangeSpec; +import com.yahoo.elide.security.RequestScope; + +import java.util.Optional; /** * Operation check interface. @@ -18,10 +21,14 @@ * * @param Type parameter */ -public abstract class OperationCheck extends InlineCheck { - /* NOTE: Operation checks and user checks are intended to be _distinct_ */ - @Override - public final boolean ok(User user) { - throw new UnsupportedOperationException(); - } +public abstract class OperationCheck implements Check { + /** + * Determines whether the user can access the resource. + * + * @param object Fully modified object + * @param requestScope Request scope object + * @param changeSpec Summary of modifications + * @return true if security check passed + */ + public abstract boolean ok(T object, RequestScope requestScope, Optional changeSpec); } diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/UserCheck.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/UserCheck.java index f44a953060..41e62c68bc 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/UserCheck.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/UserCheck.java @@ -5,19 +5,18 @@ */ package com.yahoo.elide.security.checks; -import com.yahoo.elide.security.ChangeSpec; -import com.yahoo.elide.security.RequestScope; - -import java.util.Optional; +import com.yahoo.elide.security.User; /** * Custom security access that verifies whether a user belongs to a role. * Permissions are assigned as a set of checks that grant access to the permission. */ -public abstract class UserCheck extends InlineCheck { - /* NOTE: Operation checks and user checks are intended to be _distinct_ */ - @Override - public final boolean ok(Object object, RequestScope requestScope, Optional changeSpec) { - return ok(requestScope.getUser()); - } +public abstract class UserCheck implements Check { + /** + * Method reserved for user checks. + * + * @param user User to check + * @return True if user check passes, false otherwise + */ + public abstract boolean ok(User user); } diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Collections.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Collections.java index e91bfc77a9..7a21a3668b 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Collections.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Collections.java @@ -8,7 +8,7 @@ import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; +import com.yahoo.elide.security.checks.OperationCheck; import java.util.Collection; import java.util.Optional; @@ -27,7 +27,7 @@ private Collections() { * * @param type collection to be validated */ - public static class AppendOnly extends CommitCheck { + public static class AppendOnly extends OperationCheck { @Override public boolean ok(T record, RequestScope requestScope, Optional changeSpec) { @@ -47,7 +47,7 @@ public boolean ok(T record, RequestScope requestScope, Optional chan * * @param type parameter */ - public static class RemoveOnly extends CommitCheck { + public static class RemoveOnly extends OperationCheck { @Override public boolean ok(T record, RequestScope requestScope, Optional changeSpec) { diff --git a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Common.java b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Common.java index cd6600983c..1b75722215 100644 --- a/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Common.java +++ b/elide-annotations/src/main/java/com/yahoo/elide/security/checks/prefab/Common.java @@ -9,7 +9,6 @@ import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.PersistentResource; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import java.util.Optional; @@ -42,7 +41,7 @@ public boolean ok(T record, RequestScope requestScope, Optional chan * but at the same time allows the removal of the child from the relationship with the existing parent * @param the type of object that this check guards */ - public static class FieldSetToNull extends CommitCheck { + public static class FieldSetToNull extends OperationCheck { @Override public boolean ok(T record, RequestScope requestScope, Optional changeSpec) { return changeSpec.map((c) -> { return c.getModified() == null; }) diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/expression/CanPaginateVisitor.java b/elide-core/src/main/java/com/yahoo/elide/parsers/expression/CanPaginateVisitor.java index 4cc0615419..449abf86bd 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/expression/CanPaginateVisitor.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/expression/CanPaginateVisitor.java @@ -208,14 +208,14 @@ public PaginationStatus visitPAREN(ExpressionParser.PARENContext ctx) { @Override public PaginationStatus visitPermissionClass(ExpressionParser.PermissionClassContext ctx) { - Check check = getCheck(dictionary, ctx.getText()); + Check check = getCheck(dictionary, ctx.getText()); //Filter expression checks can always be pushed to the DataStore so pagination is possible if (check instanceof FilterExpressionCheck) { return PaginationStatus.CAN_PAGINATE; } if (check instanceof UserCheck) { - if (check.ok(scope.getUser())) { + if (((UserCheck) check).ok(scope.getUser())) { return PaginationStatus.USER_CHECK_TRUE; } return PaginationStatus.USER_CHECK_FALSE; diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitor.java b/elide-core/src/main/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitor.java index a8622f1bbc..5413b8cd35 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitor.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/expression/PermissionToFilterExpressionVisitor.java @@ -177,7 +177,7 @@ public FilterExpression visitPermissionClass(ExpressionParser.PermissionClassCon } if (check instanceof UserCheck) { - boolean userCheckResult = check.ok(requestScope.getUser()); + boolean userCheckResult = ((UserCheck) check).ok(requestScope.getUser()); return userCheckResult ? TRUE_USER_CHECK_EXPRESSION : FALSE_USER_CHECK_EXPRESSION; } diff --git a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java index 6a03492201..dff681d709 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/FilterExpressionCheck.java @@ -12,8 +12,8 @@ import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.parsers.expression.FilterExpressionCheckEvaluationVisitor; -import com.yahoo.elide.security.checks.InlineCheck; +import com.yahoo.elide.security.checks.OperationCheck; import lombok.extern.slf4j.Slf4j; import java.util.Optional; @@ -27,7 +27,7 @@ * @param Type of class */ @Slf4j -public abstract class FilterExpressionCheck extends InlineCheck { +public abstract class FilterExpressionCheck extends OperationCheck { @Inject protected EntityDictionary dictionary; @@ -41,12 +41,6 @@ public abstract class FilterExpressionCheck extends InlineCheck { */ public abstract FilterExpression getFilterExpression(Class entityClass, RequestScope requestScope); - /* NOTE: Filter Expression checks and user checks are intended to be _distinct_ */ - @Override - public final boolean ok(User user) { - throw new UnsupportedOperationException(); - } - /** * The filter expression is evaluated in memory if it cannot be pushed to the data store by elide for any reason. * diff --git a/elide-core/src/main/java/com/yahoo/elide/security/permissions/expressions/CheckExpression.java b/elide-core/src/main/java/com/yahoo/elide/security/permissions/expressions/CheckExpression.java index 2593a50458..36174d8232 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/permissions/expressions/CheckExpression.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/permissions/expressions/CheckExpression.java @@ -15,7 +15,7 @@ import com.yahoo.elide.security.PersistentResource; import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.Check; -import com.yahoo.elide.security.checks.InlineCheck; +import com.yahoo.elide.security.checks.OperationCheck; import com.yahoo.elide.security.checks.UserCheck; import com.yahoo.elide.security.permissions.ExpressionResult; import com.yahoo.elide.security.permissions.ExpressionResultCache; @@ -76,7 +76,7 @@ public ExpressionResult evaluate(EvaluationMode mode) { return result; } - if (mode == EvaluationMode.INLINE_CHECKS_ONLY && ! (check instanceof InlineCheck)) { + if (mode == EvaluationMode.INLINE_CHECKS_ONLY && (resource != null && resource.isNewlyCreated())) { result = DEFERRED; return result; } @@ -112,7 +112,12 @@ public ExpressionResult evaluate(EvaluationMode mode) { */ private ExpressionResult computeCheck() { Object entity = (resource == null) ? null : resource.getObject(); - result = check.ok(entity, requestScope, changeSpec) ? PASS : FAIL; + + if (check instanceof UserCheck) { + result = ((UserCheck) check).ok(requestScope.getUser()) ? PASS : FAIL; + } else { + result = ((OperationCheck) check).ok(entity, requestScope, changeSpec) ? PASS : FAIL; + } return result; } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java index 15174cf479..41888644b4 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java @@ -46,7 +46,7 @@ public void testGetAllAnnotatedClasses() { @Test public void testGetAnyAnnotatedClasses() { Set> classes = ClassScanner.getAnnotatedClasses(ReadPermission.class, UpdatePermission.class); - assertEquals(37, classes.size()); + assertEquals(36, classes.size()); for (Class cls : classes) { assertTrue(cls.isAnnotationPresent(ReadPermission.class) || cls.isAnnotationPresent(UpdatePermission.class)); diff --git a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java index c83f8b28e3..1e7b05f9d5 100644 --- a/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/parsers/expression/PermissionExpressionVisitorTest.java @@ -145,7 +145,7 @@ public ExpressionResult evaluate(EvaluationMode ignored) { if (check instanceof UserCheck) { result = ((UserCheck) check).ok(null); } else { - result = check.ok(null, null, null); + result = ((OperationCheck) check).ok(null, null, null); } if (result) { diff --git a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java index 7bdbcad758..2d285aba41 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java @@ -5,6 +5,7 @@ */ package com.yahoo.elide.security; +import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertThrows; import com.yahoo.elide.ElideSettings; @@ -18,10 +19,10 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.TestDictionary; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import com.yahoo.elide.security.checks.UserCheck; +import com.yahoo.elide.security.permissions.ExpressionResult; import org.junit.jupiter.api.Test; import java.util.Optional; @@ -39,10 +40,12 @@ public void testSuccessfulOperationCheck() throws Exception { @UpdatePermission(expression = "sampleOperation") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, false); RequestScope requestScope = resource.getRequestScope(); ChangeSpec cspec = new ChangeSpec(null, null, null, null); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec); + assertEquals(ExpressionResult.PASS, + requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec)); + requestScope.getPermissionExecutor().executeCommitChecks(); } @@ -53,25 +56,27 @@ public void testFailOperationCheckAll() throws Exception { @UpdatePermission(expression = "sampleOperation AND deny all") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource)); - requestScope.getPermissionExecutor().executeCommitChecks(); } @Test - public void testFailOperationCheckAny() throws Exception { + public void testFailOperationCheckDeferred() throws Exception { @Entity @Include - @UpdatePermission(expression = "sampleCommit OR sampleOperation") + @UpdatePermission(expression = "sampleOperation") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource); - // Update permissions are deferred. In the case of "any," commit checks must execute before failure can be detected + + // Because the object is newly created, the check is DEFERRED. + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource)); + assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @@ -79,41 +84,32 @@ class Model { } public void testSuccessfulCommitChecks() throws Exception { @Entity @Include - @UpdatePermission(expression = "sampleCommit") + @UpdatePermission(expression = "sampleOperation") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); ChangeSpec cspec = new ChangeSpec(null, null, null, null); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec); - requestScope.getPermissionExecutor().executeCommitChecks(); - } + // Because the object is newly created, the check is DEFERRED. + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec)); - @Test - public void testFailCommitChecks() throws Exception { - @Entity - @Include - @UpdatePermission(expression = "sampleCommit") - class Model { } - - PersistentResource resource = newResource(new Model(), Model.class); - RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource); - assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); + requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testReadFieldAwareSuccessAllAnyField() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, new ChangeSpec(null, null, null, null)); + assertEquals(ExpressionResult.PASS, + requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, new ChangeSpec(null, null, null, null))); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testReadFieldAwareSuccessFailureAnyField() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -123,15 +119,16 @@ public void testReadFieldAwareSuccessFailureAnyField() { @Test public void testReadFieldAwareSuccessAll() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), ReadPermission.class, "allVisible"); + assertEquals(ExpressionResult.PASS, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), ReadPermission.class, "allVisible")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testReadFieldAwareFailureAllSpecificField() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -142,7 +139,7 @@ public void testReadFieldAwareFailureAllSpecificField() { @Test public void testReadFieldAwareFailureAllNoOverride() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -153,7 +150,7 @@ public void testReadFieldAwareFailureAllNoOverride() { @Test public void testReadFieldAwareFailureAll() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -164,15 +161,16 @@ public void testReadFieldAwareFailureAll() { @Test public void testReadFieldAwareSuccessAny() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), ReadPermission.class, "mayFailInCommit"); + assertEquals(ExpressionResult.PASS, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), ReadPermission.class, "mayFailInCommit")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testReadFieldAwareFailureAny() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -183,15 +181,16 @@ public void testReadFieldAwareFailureAny() { @Test public void testUpdateFieldAwareSuccessAll() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "allVisible"); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "allVisible")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testUpdateFieldAwareFailureAll() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, true); RequestScope requestScope = resource.getRequestScope(); requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "allVisible"); assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); @@ -199,15 +198,16 @@ public void testUpdateFieldAwareFailureAll() { @Test public void testUpdateFieldAwareSuccessAny() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "mayFailInCommit"); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "mayFailInCommit")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testUpdateFieldAwareFailureAny() { - PersistentResource resource = newResource(SampleBean.class); + PersistentResource resource = newResource(SampleBean.class, true); RequestScope requestScope = resource.getRequestScope(); requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "mayFailInCommit"); assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); @@ -215,16 +215,16 @@ public void testUpdateFieldAwareFailureAny() { @Test public void testReadFieldAwareEntireOpenBean() { - PersistentResource resource = newResource(OpenBean.class); + PersistentResource resource = newResource(OpenBean.class, false); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, ReadPermission.class, "open"); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource)); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, ReadPermission.class, "open")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testReadFailureFieldAwareOpenBean() { - PersistentResource resource = newResource(OpenBean.class); + PersistentResource resource = newResource(OpenBean.class, false); RequestScope requestScope = resource.getRequestScope(); assertThrows( ForbiddenAccessException.class, @@ -237,18 +237,19 @@ public void testReadFailureFieldAwareOpenBean() { public void testPassAnyFieldAwareFailOperationSuccessCommit() { @Entity @Include - @UpdatePermission(expression = "deny all AND passingCommit") + @UpdatePermission(expression = "deny all AND passingOp") class Model { @Id public Long id; - @UpdatePermission(expression = "deny all OR passingCommit") + @UpdatePermission(expression = "deny all OR passingOp") public String field = "some data"; } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource)); requestScope.getPermissionExecutor().executeCommitChecks(); } @@ -261,13 +262,14 @@ class Model { @Id public Long id; - @UpdatePermission(expression = "allow all AND FailAtCommit") + @UpdatePermission(expression = "allow all AND FailOp") public String field = "some data"; } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource)); assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @@ -275,18 +277,19 @@ class Model { public void testPassAnySpecificFieldAwareFailOperationSuccessCommit() { @Entity @Include - @UpdatePermission(expression = "deny all AND passingCommit") + @UpdatePermission(expression = "deny all AND passingOp") class Model { @Id public Long id; - @UpdatePermission(expression = "deny all OR passingCommit") + @UpdatePermission(expression = "deny all OR passingOp") public String field = "some data"; } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "field"); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "field")); requestScope.getPermissionExecutor().executeCommitChecks(); } @@ -299,13 +302,14 @@ class Model { @Id public Long id; - @UpdatePermission(expression = "allow all AND FailAtCommit") + @UpdatePermission(expression = "allow all AND FailOp") public String field = "some data"; } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "field"); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "field")); assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @@ -316,7 +320,7 @@ public void testBadInstance() { @UpdatePermission(expression = "privatePermission") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); assertThrows( IllegalArgumentException.class, @@ -326,70 +330,65 @@ class Model { } @Test public void testSpecificFieldOveriddenOperationCheckSucceed() { - PersistentResource resource = newResource(CommitCheckEntity.class); + PersistentResource resource = newResource(CheckedEntity.class, true); RequestScope requestScope = resource.getRequestScope(); // Should succeed in operation check despite the commit check failure - requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "hello"); + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, null, UpdatePermission.class, "hello")); requestScope.getPermissionExecutor().executeCommitChecks(); } @Test public void testSpecificFieldCommitCheckFailByOveriddenField() { - PersistentResource resource = newResource(CommitCheckEntity.class); + PersistentResource resource = newResource(CheckedEntity.class, true); RequestScope requestScope = resource.getRequestScope(); - // Should succeed in commit check despite the operation check failure + assertEquals(ExpressionResult.DEFERRED, + requestScope.getPermissionExecutor().checkSpecificFieldPermissions(resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "hello")); assertThrows( ForbiddenAccessException.class, - () -> requestScope.getPermissionExecutor().checkSpecificFieldPermissions( - resource, new ChangeSpec(null, null, null, null), UpdatePermission.class, "hello")); - requestScope.getPermissionExecutor().executeCommitChecks(); + () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @Test - public void testReadCheckExpressionAlwaysInline() { + public void testReadCheckExpressionForNewlyCreatedObject() { @Entity @Include - @ReadPermission(expression = "FailAtCommit") + @ReadPermission(expression = "FailOp") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); requestScope.getDictionary().bindEntity(Model.class); - assertThrows( - ForbiddenAccessException.class, - () -> requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource)); - // NOTE: This check should throw a ForbiddenAccess since commit-time checks should be converted - // to inline checks. As a result, DO NOT call executeCommitChecks() in this test. + assertEquals(ExpressionResult.DEFERRED, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource)); + assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @Test - public void testDeleteCheckExpressionAlwaysInline() { + public void testDeleteCheckExpressionForNewlyCreatedObject() { @Entity @Include - @DeletePermission(expression = "FailAtCommit") + @DeletePermission(expression = "FailOp") class Model { } - PersistentResource resource = newResource(new Model(), Model.class); + PersistentResource resource = newResource(new Model(), Model.class, true); RequestScope requestScope = resource.getRequestScope(); requestScope.getDictionary().bindEntity(Model.class); - assertThrows( - ForbiddenAccessException.class, - () -> requestScope.getPermissionExecutor().checkPermission(DeletePermission.class, resource)); - // NOTE: This check should throw a ForbiddenAccess since commit-time checks should be converted - // to inline checks. As a result, DO NOT call executeCommitChecks() in this test. + assertEquals(ExpressionResult.DEFERRED, requestScope.getPermissionExecutor().checkPermission(DeletePermission.class, resource)); + assertThrows(ForbiddenAccessException.class, () -> requestScope.getPermissionExecutor().executeCommitChecks()); } @Test public void testCache() { - PersistentResource resource = newResource(AnnotationOnlyRecord.class); + PersistentResource resource = newResource(AnnotationOnlyRecord.class, false); RequestScope requestScope = resource.getRequestScope(); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource)); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource)); } + @Test public void testNoCache() { - PersistentResource resource = newResource(AnnotationOnlyRecord.class); + PersistentResource resource = newResource(AnnotationOnlyRecord.class, false); RequestScope requestScope = resource.getRequestScope(); ChangeSpec cspec = new ChangeSpec(null, null, null, null); assertThrows( @@ -402,30 +401,31 @@ public void testNoCache() { @Test public void testUserCheckCache() { - PersistentResource resource = newResource(UserCheckCacheRecord.class); + PersistentResource resource = newResource(UserCheckCacheRecord.class, false); RequestScope requestScope = resource.getRequestScope(); ChangeSpec cspec = new ChangeSpec(null, null, null, null); // This should cache for updates, reads, etc. - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec); - requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, cspec); - requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, cspec); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec)); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(UpdatePermission.class, resource, cspec)); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, cspec)); + assertEquals(ExpressionResult.PASS, requestScope.getPermissionExecutor().checkPermission(ReadPermission.class, resource, cspec)); } - public PersistentResource newResource(T obj, Class cls) { + public PersistentResource newResource(T obj, Class cls, boolean markNew) { EntityDictionary dictionary = TestDictionary.getTestDictionary(); dictionary.bindEntity(cls); RequestScope requestScope = new RequestScope(null, null, null, null, null, getElideSettings(dictionary)); - return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); + PersistentResource resource = new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); + if (markNew) { + requestScope.getNewPersistentResources().add(resource); + } + return resource; } - public PersistentResource newResource(Class cls) { - EntityDictionary dictionary = TestDictionary.getTestDictionary(); - dictionary.bindEntity(cls); - RequestScope requestScope = new RequestScope(null, null, null, null, null, getElideSettings(dictionary)); + public PersistentResource newResource(Class cls, boolean markNew) { try { T obj = cls.newInstance(); - return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); + return newResource(obj, cls, markNew); } catch (InstantiationException | IllegalAccessException e) { return null; } @@ -444,39 +444,14 @@ public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestSc } } - public static final class SampleCommitCheck extends CommitCheck { - @Override - public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { - return changeSpec.isPresent(); - } - } - public static final class SampleOperationCheckCommitInverse extends OperationCheck { + public static final class SampleOperationCheckInverse extends OperationCheck { @Override public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { return !changeSpec.isPresent(); } } - public static final class PassingCommitCheck extends CommitCheck { - @Override - public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { - return true; - } - } - - public static final class FailingCommitCheck extends CommitCheck { - @Override - public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { - return false; - } - - @Override - public String checkIdentifier() { - return "FailAtCommit"; - } - } - @ReadPermission(expression = "deny all") @UpdatePermission(expression = "deny all") @Include @@ -486,7 +461,7 @@ public static final class SampleBean { public Long id; @ReadPermission(expression = "allow all AND sampleOperation") - @UpdatePermission(expression = "allow all AND sampleCommit") + @UpdatePermission(expression = "allow all AND sampleOperation") public String allVisible = "You should see me!"; public String defaultHidden = "I'm invisible. muwahaha..."; @@ -496,7 +471,7 @@ public static final class SampleBean { public String cannotSeeMe = "hidden"; @ReadPermission(expression = "sampleOperation") - @UpdatePermission(expression = "sampleCommit OR deny all") + @UpdatePermission(expression = "sampleOperation OR deny all") public String mayFailInCommit = "aw :("; } @@ -510,23 +485,23 @@ public static final class OpenBean { public String open; - @ReadPermission(expression = "deny all AND sampleOperation") - @UpdatePermission(expression = "sampleCommit AND sampleOperation") + @ReadPermission(expression = "allow all AND sampleOperation") + @UpdatePermission(expression = "allow all AND sampleOperation") public String openAll = "all"; - @ReadPermission(expression = "sampleOperation OR sampleOperation") - @UpdatePermission(expression = "sampleCommit OR sampleOperation") + @ReadPermission(expression = "deny all OR sampleOperation") + @UpdatePermission(expression = "deny all OR sampleOperation") public String openAny = "all"; } @Entity @Include - @UpdatePermission(expression = "sampleCommit") - public static final class CommitCheckEntity { + @UpdatePermission(expression = "sampleOperation") + public static final class CheckedEntity { @Id public Long id; - @UpdatePermission(expression = "sampleOperationCommitInverse") + @UpdatePermission(expression = "sampleOperationInverse") public String hello; } @@ -540,6 +515,20 @@ public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestSc } } + public static final class PassingOperationCheck extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + return true; + } + } + + public static final class FailingOperationCheck extends OperationCheck { + @Override + public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { + return false; + } + } + @Entity @Include @ReadPermission(expression = "shouldCache") diff --git a/elide-core/src/test/java/example/Child.java b/elide-core/src/test/java/example/Child.java index e7664b78df..407a85a2d1 100644 --- a/elide-core/src/test/java/example/Child.java +++ b/elide-core/src/test/java/example/Child.java @@ -12,7 +12,6 @@ import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import com.fasterxml.jackson.annotation.JsonIgnore; @@ -32,7 +31,7 @@ @Entity(name = "childEntity") @CreatePermission(expression = "initCheck") @SharePermission -@ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheckOp") +@ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheck") @Include(rootLevel = true, type = "child") @Audit(action = Audit.Action.DELETE, operation = 0, @@ -114,7 +113,7 @@ public void setReadNoAccess(Child noReadAccess) { this.noReadAccess = noReadAccess; } - static public class InitCheck extends CommitCheck { + static public class InitCheck extends OperationCheck { @Override public boolean ok(Child child, RequestScope requestScope, Optional changeSpec) { if (child.getParents() != null) { @@ -122,25 +121,5 @@ public boolean ok(Child child, RequestScope requestScope, Optional c } return false; } - - @Override - public String checkIdentifier() { - return "initCheck"; - } - } - - static public class InitCheckOp extends OperationCheck { - @Override - public boolean ok(Child child, RequestScope requestScope, Optional changeSpec) { - if (child.getParents() != null) { - return true; - } - return false; - } - - @Override - public String checkIdentifier() { - return "initCheckOp"; - } } } diff --git a/elide-core/src/test/java/example/NegativeChildIdCheck.java b/elide-core/src/test/java/example/NegativeChildIdCheck.java index 39691db950..f516fa32ee 100644 --- a/elide-core/src/test/java/example/NegativeChildIdCheck.java +++ b/elide-core/src/test/java/example/NegativeChildIdCheck.java @@ -19,9 +19,4 @@ public class NegativeChildIdCheck extends OperationCheck { public boolean ok(Child child, RequestScope requestScope, Optional fChangeSpec) { return child.getId() >= 0; } - - @Override - public String checkIdentifier() { - return "negativeChildId"; - } } diff --git a/elide-core/src/test/java/example/NegativeIntegerUserCheck.java b/elide-core/src/test/java/example/NegativeIntegerUserCheck.java index 155e0daad4..3508b5c772 100644 --- a/elide-core/src/test/java/example/NegativeIntegerUserCheck.java +++ b/elide-core/src/test/java/example/NegativeIntegerUserCheck.java @@ -17,9 +17,4 @@ public boolean ok(User user) { Integer id = (Integer) user.getOpaqueUser(); return id >= 0; } - - @Override - public String checkIdentifier() { - return "negativeIntegerUser"; - } } diff --git a/elide-core/src/test/java/example/Parent.java b/elide-core/src/test/java/example/Parent.java index 51b379fe07..d2bf6ca5f1 100644 --- a/elide-core/src/test/java/example/Parent.java +++ b/elide-core/src/test/java/example/Parent.java @@ -12,7 +12,6 @@ import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import lombok.ToString; @@ -30,9 +29,9 @@ import javax.validation.constraints.NotNull; @CreatePermission(expression = "parentInitCheck OR allow all") -@ReadPermission(expression = "parentInitCheckOp OR allow all") +@ReadPermission(expression = "parentInitCheck OR allow all") @UpdatePermission(expression = "parentInitCheck OR allow all OR deny all") -@DeletePermission(expression = "parentInitCheckOp OR allow all OR deny all") +@DeletePermission(expression = "parentInitCheck OR allow all OR deny all") @Include(rootLevel = true, type = "parent") // optional here because class has this name @Entity @ToString @@ -100,22 +99,7 @@ public void setSpecialAttribute(String specialAttribute) { this.specialAttribute = specialAttribute; } - static public class InitCheck extends CommitCheck { - @Override - public String checkIdentifier() { - return "parentInitCheck"; - } - - @Override - public boolean ok(Parent parent, RequestScope requestScope, Optional changeSpec) { - if (parent.getChildren() != null && parent.getSpouses() != null) { - return true; - } - return false; - } - } - - static public class InitCheckOp extends OperationCheck { + static public class InitCheck extends OperationCheck { @Override public boolean ok(Parent parent, RequestScope requestScope, Optional changeSpec) { if (parent.getChildren() != null && parent.getSpouses() != null) { @@ -123,11 +107,6 @@ public boolean ok(Parent parent, RequestScope requestScope, Optional } return false; } - - @Override - public String checkIdentifier() { - return "parentInitCheckOp"; - } } static public class SpecialValue extends OperationCheck { @@ -142,10 +121,5 @@ public boolean ok(Parent object, RequestScope requestScope, Optional } return false; } - - @Override - public String checkIdentifier() { - return "specialValue"; - } } } diff --git a/elide-core/src/test/java/example/TestCheckMappings.java b/elide-core/src/test/java/example/TestCheckMappings.java index 5c64cee419..32e055cbe9 100644 --- a/elide-core/src/test/java/example/TestCheckMappings.java +++ b/elide-core/src/test/java/example/TestCheckMappings.java @@ -27,19 +27,16 @@ public class TestCheckMappings { .put("changeSpecCollection", PersistentResourceTest.ChangeSpecCollection.class) .put("changeSpecNonCollection", PersistentResourceTest.ChangeSpecNonCollection.class) .put("initCheck", Child.InitCheck.class) - .put("initCheckOp", Child.InitCheckOp.class) .put("parentInitCheck", Parent.InitCheck.class) - .put("parentInitCheckOp", Parent.InitCheckOp.class) .put("negativeIntegerUser", NegativeIntegerUserCheck.class) .put("negativeChildId", NegativeChildIdCheck.class) - .put("FailAtCommit", PermissionExecutorTest.FailingCommitCheck.class) + .put("FailOp", PermissionExecutorTest.FailingOperationCheck.class) .put("privatePermission", PrivatePermission.class) .put("sampleOperation", PermissionExecutorTest.SampleOperationCheck.class) - .put("sampleCommit", PermissionExecutorTest.SampleCommitCheck.class) - .put("sampleOperationCommitInverse", PermissionExecutorTest.SampleOperationCheckCommitInverse.class) + .put("sampleOperationInverse", PermissionExecutorTest.SampleOperationCheckInverse.class) .put("shouldCache", PermissionExecutorTest.ShouldCache.class) .put("peUserCheck", PermissionExecutorTest.UserCheckTest.class) - .put("passingCommit", PermissionExecutorTest.PassingCommitCheck.class) + .put("passingOp", PermissionExecutorTest.PassingOperationCheck.class) .put("Principal is user one", UserIdChecks.UserOneCheck.class) .put("Principal is user two", UserIdChecks.UserTwoCheck.class) .put("Principal is user three", UserIdChecks.UserThreeCheck.class) diff --git a/elide-core/src/test/java/example/UserIdChecks.java b/elide-core/src/test/java/example/UserIdChecks.java index a49bb89d6f..e3700f877b 100644 --- a/elide-core/src/test/java/example/UserIdChecks.java +++ b/elide-core/src/test/java/example/UserIdChecks.java @@ -19,11 +19,6 @@ public boolean ok(User user) { Integer id = (Integer) user.getOpaqueUser(); return id.equals(1); } - - @Override - public String checkIdentifier() { - return "UserOne"; - } } public static class UserTwoCheck extends UserCheck { @@ -32,11 +27,6 @@ public boolean ok(User user) { Integer id = (Integer) user.getOpaqueUser(); return id.equals(2); } - - @Override - public String checkIdentifier() { - return "UserTwo"; - } } public static class UserThreeCheck extends UserCheck { @@ -45,11 +35,6 @@ public boolean ok(User user) { Integer id = (Integer) user.getOpaqueUser(); return id.equals(3); } - - @Override - public String checkIdentifier() { - return "UserThree"; - } } public static class UserFourCheck extends UserCheck { @@ -58,10 +43,5 @@ public boolean ok(User user) { Integer id = (Integer) user.getOpaqueUser(); return id.equals(4); } - - @Override - public String checkIdentifier() { - return "UserFour"; - } } } diff --git a/elide-graphql/src/test/java/graphqlEndpointTestModels/security/CommitChecks.java b/elide-graphql/src/test/java/graphqlEndpointTestModels/security/CommitChecks.java index aa4054535c..fb5024b122 100644 --- a/elide-graphql/src/test/java/graphqlEndpointTestModels/security/CommitChecks.java +++ b/elide-graphql/src/test/java/graphqlEndpointTestModels/security/CommitChecks.java @@ -6,7 +6,7 @@ package graphqlEndpointTestModels.security; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; +import com.yahoo.elide.security.checks.OperationCheck; import java.security.Principal; import java.util.Optional; @@ -14,7 +14,7 @@ public abstract class CommitChecks { public static final String IS_NOT_USER_3 = "isnt user three"; - public static class IsNotUser3 extends CommitCheck { + public static class IsNotUser3 extends OperationCheck { @Override public boolean ok(Object object, RequestScope requestScope, Optional optional) { diff --git a/elide-integration-tests/src/test/java/example/Child.java b/elide-integration-tests/src/test/java/example/Child.java index 59895c18d2..2ce4c9587c 100644 --- a/elide-integration-tests/src/test/java/example/Child.java +++ b/elide-integration-tests/src/test/java/example/Child.java @@ -18,7 +18,6 @@ import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.FilterExpressionCheck; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import java.util.Optional; @@ -37,7 +36,7 @@ @Entity(name = "childEntity") @CreatePermission(expression = "initCheck") @SharePermission -@ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheckOp AND initCheckFilter") +@ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheck AND initCheckFilter") @Include(rootLevel = true, type = "child") @Audit(action = Audit.Action.DELETE, operation = 0, @@ -115,20 +114,7 @@ public String setComputedFailTest(String unused) { throw new IllegalAccessError(); } - /** - * Initialization validation check. - */ - static public class InitCheck extends CommitCheck { - @Override - public boolean ok(Child child, RequestScope requestScope, Optional changeSpec) { - if (child.getParents() != null) { - return true; - } - return false; - } - } - - static public class InitCheckOp extends OperationCheck { + static public class InitCheck extends OperationCheck { @Override public boolean ok(Child child, RequestScope requestScope, Optional changeSpec) { if (child.getParents() != null) { diff --git a/elide-integration-tests/src/test/java/example/NoCommitEntity.java b/elide-integration-tests/src/test/java/example/NoCommitEntity.java index 327e188b14..48c1d23612 100644 --- a/elide-integration-tests/src/test/java/example/NoCommitEntity.java +++ b/elide-integration-tests/src/test/java/example/NoCommitEntity.java @@ -10,7 +10,7 @@ import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; +import com.yahoo.elide.security.checks.OperationCheck; import java.util.Optional; @@ -27,7 +27,7 @@ @Entity @Table(name = "nocommit") public class NoCommitEntity extends BaseId { - static public class NoCommitCheck extends CommitCheck { + static public class NoCommitCheck extends OperationCheck { @Override public boolean ok(T record, RequestScope requestScope, Optional changeSpec) { return false; diff --git a/elide-integration-tests/src/test/java/example/Parent.java b/elide-integration-tests/src/test/java/example/Parent.java index f49b1ac1d9..18cfb108ea 100644 --- a/elide-integration-tests/src/test/java/example/Parent.java +++ b/elide-integration-tests/src/test/java/example/Parent.java @@ -14,7 +14,6 @@ import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; -import com.yahoo.elide.security.checks.CommitCheck; import com.yahoo.elide.security.checks.OperationCheck; import lombok.ToString; @@ -33,9 +32,9 @@ * Parent test bean. */ @CreatePermission(expression = "parentInitCheck OR allow all") -@ReadPermission(expression = "parentInitCheckOp OR allow all") +@ReadPermission(expression = "parentInitCheck OR allow all") @UpdatePermission(expression = "parentInitCheck OR allow all OR deny all") -@DeletePermission(expression = "parentInitCheckOp OR allow all OR deny all") +@DeletePermission(expression = "parentInitCheck OR allow all OR deny all") @SharePermission @Include(rootLevel = true, type = "parent") // optional here because class has this name @Paginate(maxLimit = 100000) @@ -104,20 +103,8 @@ public void setSpecialAttribute(String specialAttribute) { this.specialAttribute = specialAttribute; } - /** - * Initialization validation check. - */ - static public class InitCheck extends CommitCheck { - @Override - public boolean ok(Parent parent, RequestScope requestScope, Optional changeSpec) { - if (parent.getChildren() != null && parent.getSpouses() != null) { - return true; - } - return false; - } - } - static public class InitCheckOp extends OperationCheck { + static public class InitCheck extends OperationCheck { @Override public boolean ok(Parent parent, RequestScope requestScope, Optional changeSpec) { if (parent.getChildren() != null && parent.getSpouses() != null) { diff --git a/elide-integration-tests/src/test/java/example/TestCheckMappings.java b/elide-integration-tests/src/test/java/example/TestCheckMappings.java index 83a4d7094b..277c1979de 100644 --- a/elide-integration-tests/src/test/java/example/TestCheckMappings.java +++ b/elide-integration-tests/src/test/java/example/TestCheckMappings.java @@ -23,11 +23,9 @@ public class TestCheckMappings { .put("deny all", Role.NONE.class) .put("adminRoleCheck", User.AdminRoleCheck.class) .put("initCheck", Child.InitCheck.class) - .put("initCheckOp", Child.InitCheckOp.class) .put("FailCheckOp", Child.FailCheckOp.class) .put("initCheckFilter", Child.InitCheckFilter.class) .put("parentInitCheck", Parent.InitCheck.class) - .put("parentInitCheckOp", Parent.InitCheckOp.class) .put("parentSpecialValue", Parent.SpecialValue.class) .put("negativeIntegerUser", NegativeIntegerUserCheck.class) .put("negativeChildId", NegativeChildIdCheck.class) diff --git a/elide-integration-tests/src/test/java/example/User.java b/elide-integration-tests/src/test/java/example/User.java index a48e92d986..dedb7cf767 100644 --- a/elide-integration-tests/src/test/java/example/User.java +++ b/elide-integration-tests/src/test/java/example/User.java @@ -77,10 +77,5 @@ static public class AdminRoleCheck extends OperationCheck { public boolean ok(User user, RequestScope requestScope, Optional changeSpec) { return (user.getRole() == 1); } - - @Override - public String checkIdentifier() { - return "adminRoleCheck"; - } } } From 3c54d1e802403348852036bb9046b1d8d7a44ab8 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Mon, 3 Feb 2020 10:21:40 -0600 Subject: [PATCH 03/16] Removed UpdateOnCreate. Refactored AuditLogger, Pagination, & Sorting (#1146) * Removed UpdateOnCreate. Refactored AuditLogger * Refactored Sorting * Refactored Pagination * Refactored Pagination * Pagination refactor builds and tests pass * Codacy fixes * Inspection rework * Fixes build * More inspection rework * Fix build --- .../src/main/java/com/yahoo/elide/Elide.java | 2 +- .../com/yahoo/elide/ElideSettingsBuilder.java | 6 +- .../com/yahoo/elide/audit/AuditLogger.java | 4 +- .../com/yahoo/elide/audit/LogMessage.java | 195 ++-------- .../com/yahoo/elide/audit/LogMessageImpl.java | 175 +++++++++ .../com/yahoo/elide/audit/Slf4jLogger.java | 5 +- .../elide/core/DataStoreTransaction.java | 5 +- .../yahoo/elide/core/EntityDictionary.java | 2 - .../yahoo/elide/core/PersistentResource.java | 16 +- .../com/yahoo/elide/core/RequestScope.java | 10 - .../inmemory/HashMapStoreTransaction.java | 4 +- .../inmemory/InMemoryStoreTransaction.java | 21 +- .../datastore/wrapped/TransactionWrapper.java | 6 +- .../elide/core/pagination/Pagination.java | 360 ------------------ .../elide/core/pagination/PaginationImpl.java | 269 +++++++++++++ .../sort/{Sorting.java => SortingImpl.java} | 48 ++- .../elide/jsonapi/EntityProjectionMaker.java | 28 +- .../state/CollectionTerminalState.java | 12 +- .../yahoo/elide/request/EntityProjection.java | 2 - .../com/yahoo/elide/request/Pagination.java | 64 ++++ .../java/com/yahoo/elide/request/Sorting.java | 41 ++ ...ssageTest.java => LogMessageImplTest.java} | 20 +- .../yahoo/elide/audit/TestAuditLogger.java | 5 +- .../elide/core/EntityDictionaryTest.java | 2 - ...LogicTest.java => PaginationImplTest.java} | 155 ++++---- .../InMemoryStoreTransactionTest.java | 29 +- .../wrapped/TransactionWrapperTest.java | 6 +- .../jsonapi/EntityProjectionMakerTest.java | 59 +-- .../aggregation/QueryValidator.java | 6 +- .../datastores/aggregation/query/Query.java | 4 +- .../queryengines/sql/SQLQueryConstructor.java | 4 +- .../queryengines/sql/SQLQueryEngine.java | 4 +- .../aggregation/QueryValidatorTest.java | 13 +- .../queryengines/sql/QueryEngineTest.java | 29 +- .../queryengines/sql/SubselectTest.java | 7 +- .../queryengines/sql/ViewTest.java | 17 +- .../hql/AbstractHQLQueryBuilder.java | 8 +- .../RootCollectionPageTotalsQueryBuilder.java | 4 +- .../SubCollectionPageTotalsQueryBuilder.java | 4 +- .../hql/AbstractHQLQueryBuilderTest.java | 16 +- .../RootCollectionFetchQueryBuilderTest.java | 7 +- ...tCollectionPageTotalsQueryBuilderTest.java | 6 +- .../SubCollectionFetchQueryBuilderTest.java | 7 +- ...bCollectionPageTotalsQueryBuilderTest.java | 6 +- .../hibernate3/HibernateTransaction.java | 8 +- .../hibernate5/HibernateTransaction.java | 8 +- .../transaction/AbstractJpaTransaction.java | 8 +- .../multiplex/BridgeableTransaction.java | 4 +- .../multiplex/MultiplexTransaction.java | 8 +- .../bridgeable/BridgeableRedisStore.java | 11 +- .../search/SearchDataTransaction.java | 21 +- .../datastores/search/DataStoreLoadTest.java | 26 +- .../datastores/search/DependencyBinder.java | 6 +- .../com/yahoo/elide/graphql/QueryRunner.java | 2 +- .../containers/ConnectionContainer.java | 2 +- .../graphql/containers/PageInfoContainer.java | 2 +- .../parser/GraphQLEntityProjectionMaker.java | 62 +-- .../elide/graphql/GraphQLEndpointTest.java | 2 +- .../yahoo/elide/graphql/ModelBuilderTest.java | 2 +- .../com/yahoo/elide/audit/InMemoryLogger.java | 3 +- .../yahoo/elide/audit/TestAuditLogger.java | 4 +- .../com/yahoo/elide/tests/PaginateIT.java | 2 +- .../com/yahoo/elide/tests/ResourceIT.java | 8 +- .../example/EntityWithPaginateMaxLimit.java | 2 +- .../test/java/example/TestCheckMappings.java | 2 - .../src/test/java/example/User.java | 2 +- 66 files changed, 978 insertions(+), 910 deletions(-) create mode 100644 elide-core/src/main/java/com/yahoo/elide/audit/LogMessageImpl.java delete mode 100644 elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/core/pagination/PaginationImpl.java rename elide-core/src/main/java/com/yahoo/elide/core/sort/{Sorting.java => SortingImpl.java} (80%) create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Pagination.java create mode 100644 elide-core/src/main/java/com/yahoo/elide/request/Sorting.java rename elide-core/src/test/java/com/yahoo/elide/audit/{LogMessageTest.java => LogMessageImplTest.java} (86%) rename elide-core/src/test/java/com/yahoo/elide/core/{PaginationLogicTest.java => PaginationImplTest.java} (52%) diff --git a/elide-core/src/main/java/com/yahoo/elide/Elide.java b/elide-core/src/main/java/com/yahoo/elide/Elide.java index df5fd02f2e..46796624e1 100644 --- a/elide-core/src/main/java/com/yahoo/elide/Elide.java +++ b/elide-core/src/main/java/com/yahoo/elide/Elide.java @@ -289,7 +289,7 @@ protected ElideResponse handleRequest(boolean isReadOnly, Object opaqueUser, ElideResponse response = buildResponse(responder.get()); - auditLogger.commit(requestScope); + auditLogger.commit(); tx.commit(requestScope); requestScope.runQueuedPostCommitTriggers(); diff --git a/elide-core/src/main/java/com/yahoo/elide/ElideSettingsBuilder.java b/elide-core/src/main/java/com/yahoo/elide/ElideSettingsBuilder.java index e898f6e85c..b787a895ba 100644 --- a/elide-core/src/main/java/com/yahoo/elide/ElideSettingsBuilder.java +++ b/elide-core/src/main/java/com/yahoo/elide/ElideSettingsBuilder.java @@ -15,7 +15,7 @@ import com.yahoo.elide.core.filter.dialect.JoinFilterDialect; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; import com.yahoo.elide.core.filter.dialect.SubqueryFilterDialect; -import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.pagination.PaginationImpl; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.security.PermissionExecutor; import com.yahoo.elide.security.executors.ActivePermissionExecutor; @@ -45,8 +45,8 @@ public class ElideSettingsBuilder { private List joinFilterDialects; private List subqueryFilterDialects; private Map serdes; - private int defaultMaxPageSize = Pagination.MAX_PAGE_LIMIT; - private int defaultPageSize = Pagination.DEFAULT_PAGE_LIMIT; + private int defaultMaxPageSize = PaginationImpl.MAX_PAGE_LIMIT; + private int defaultPageSize = PaginationImpl.DEFAULT_PAGE_LIMIT; private boolean useFilterExpressions; private int updateStatusCode; private boolean returnErrorObjects; diff --git a/elide-core/src/main/java/com/yahoo/elide/audit/AuditLogger.java b/elide-core/src/main/java/com/yahoo/elide/audit/AuditLogger.java index be5c78721d..a3e5683c11 100644 --- a/elide-core/src/main/java/com/yahoo/elide/audit/AuditLogger.java +++ b/elide-core/src/main/java/com/yahoo/elide/audit/AuditLogger.java @@ -5,8 +5,6 @@ */ package com.yahoo.elide.audit; -import com.yahoo.elide.core.RequestScope; - import java.io.IOException; import java.util.ArrayList; import java.util.List; @@ -27,7 +25,7 @@ public void log(LogMessage message) { messages.get().add(message); } - public abstract void commit(RequestScope requestScope) throws IOException; + public abstract void commit() throws IOException; public void clear() { List remainingMessages = messages.get(); diff --git a/elide-core/src/main/java/com/yahoo/elide/audit/LogMessage.java b/elide-core/src/main/java/com/yahoo/elide/audit/LogMessage.java index 6b821b0ecd..18e383de07 100644 --- a/elide-core/src/main/java/com/yahoo/elide/audit/LogMessage.java +++ b/elide-core/src/main/java/com/yahoo/elide/audit/LogMessage.java @@ -1,202 +1,55 @@ /* - * Copyright 2015, Yahoo Inc. + * Copyright 2019, Yahoo Inc. * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ + package com.yahoo.elide.audit; -import com.yahoo.elide.annotation.Audit; -import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.RequestScope; -import com.yahoo.elide.core.ResourceLineage; import com.yahoo.elide.security.ChangeSpec; - +import com.yahoo.elide.security.PersistentResource; import com.yahoo.elide.security.User; -import de.odysseus.el.ExpressionFactoryImpl; -import de.odysseus.el.util.SimpleContext; -import java.text.MessageFormat; -import java.util.List; import java.util.Optional; -import java.util.stream.Collectors; - -import javax.el.ELException; -import javax.el.ExpressionFactory; -import javax.el.PropertyNotFoundException; -import javax.el.ValueExpression; /** - * An audit log message that can be logged to a logger. + * Elide audit entity for a CRUD action. */ -public class LogMessage { - //Supposedly this is thread safe. - private static final ExpressionFactory EXPRESSION_FACTORY = new ExpressionFactoryImpl(); - private static final String[] EMPTY_STRING_ARRAY = new String[0]; - - private final String template; - private final PersistentResource record; - private final String[] expressions; - private final int operationCode; - private final Optional changeSpec; +public interface LogMessage { /** - * Construct a log message that does not involve any templating. - * @param template - The unsubstituted text that will be logged. - * @param code - The operation code of the auditable action. + * Gets message. + * + * @return the message */ - public LogMessage(String template, int code) { - this(template, null, EMPTY_STRING_ARRAY, code, Optional.empty()); - } + public String getMessage(); /** - * Construct a log message from an Audit annotation and the record that was updated in some way. - * @param audit - The annotation containing the type of operation (UPDATE, DELETE, CREATE) - * @param record - The modified record - * @param changeSpec - Change spec of modified elements (if logging object change). empty otherwise - * @throws InvalidSyntaxException if the Audit annotation has invalid syntax. + * Gets operation code. The operation code is assigned by the developer to uniquely identify + * the type of change that is being audited. Operation code definitions are outside the scope of Elide. + * + * @return the operation code */ - public LogMessage(Audit audit, PersistentResource record, Optional changeSpec) - throws InvalidSyntaxException { - this(audit.logStatement(), record, audit.logExpressions(), audit.operation(), changeSpec); - } + public int getOperationCode(); /** - * Construct a log message. - * @param template - The log message template that requires variable substitution. - * @param record - The record which will serve as the data to substitute. - * @param expressions - A set of UEL expressions that reference record. - * @param code - The operation code of the auditable action. - * @param changeSpec - the change spec that we want to log - * @throws InvalidSyntaxException the invalid syntax exception + * Get the user principal associated with the request. + * + * @return the user principal. */ - public LogMessage(String template, - PersistentResource record, - String[] expressions, - int code, - Optional changeSpec) throws InvalidSyntaxException { - this.template = template; - this.record = record; - this.expressions = expressions; - this.operationCode = code; - this.changeSpec = changeSpec; - } + public User getUser(); /** - * Gets operation code. + * Get the change specification * - * @return the operation code + * @return the change specification. */ - public int getOperationCode() { - return operationCode; - } + public Optional getChangeSpec(); /** - * Gets message. + * Get the resource that was manipulated. * - * @return the message + * @return the resource. */ - public String getMessage() { - final SimpleContext ctx = new SimpleContext(); - final SimpleContext singleElementContext = new SimpleContext(); - - if (record != null) { - /* Create a new lineage which includes the passed in record */ - ResourceLineage lineage = new ResourceLineage(record.getLineage(), record); - - for (String name : lineage.getKeys()) { - List values = lineage.getRecord(name); - - final ValueExpression expression; - final ValueExpression singleElementExpression; - if (values.size() == 1) { - expression = EXPRESSION_FACTORY.createValueExpression(values.get(0).getObject(), Object.class); - singleElementExpression = expression; - } else { - List objects = values.stream().map(PersistentResource::getObject) - .collect(Collectors.toList()); - expression = EXPRESSION_FACTORY.createValueExpression(objects, List.class); - singleElementExpression = EXPRESSION_FACTORY.createValueExpression(values.get(values.size() - 1) - .getObject(), Object.class); - } - ctx.setVariable(name, expression); - singleElementContext.setVariable(name, singleElementExpression); - } - - final Object user = getUser(); - if (user != null) { - final ValueExpression opaqueUserValueExpression = EXPRESSION_FACTORY - .createValueExpression( - user, Object.class - ); - ctx.setVariable("opaqueUser", opaqueUserValueExpression); - singleElementContext.setVariable("opaqueUser", opaqueUserValueExpression); - } - } - - Object[] results = new Object[expressions.length]; - for (int idx = 0; idx < results.length; idx++) { - String expressionText = expressions[idx]; - - final ValueExpression expression; - final ValueExpression singleElementExpression; - try { - expression = EXPRESSION_FACTORY.createValueExpression(ctx, expressionText, Object.class); - singleElementExpression = - EXPRESSION_FACTORY.createValueExpression(singleElementContext, expressionText, Object.class); - } catch (ELException e) { - throw new InvalidSyntaxException(e); - } - - Object result; - try { - // Single element expressions are intended to allow for access to ${entityType.field} when there are - // multiple "entityType" types listed in the lineage. Without this, any access to an entityType - // without an explicit list index would otherwise result in a 500. Similarly, since we already - // supported lists (i.e. the ${entityType[idx].field} syntax), this also continues to support that. - // It should be noted, however, that list indexing is somewhat brittle unless properly accounted for - // from all possible paths. - result = singleElementExpression.getValue(singleElementContext); - } catch (PropertyNotFoundException e) { - // Try list syntax if not single element - result = expression.getValue(ctx); - } - results[idx] = result; - } - - try { - return MessageFormat.format(template, results); - } catch (IllegalArgumentException e) { - throw new InvalidSyntaxException(e); - } - } - - public RequestScope getRequestScope() { - if (record != null) { - return record.getRequestScope(); - } - return null; - } - - public Object getUser() { - RequestScope requestScope = getRequestScope(); - if (requestScope != null) { - User user = requestScope.getUser(); - if (user != null) { - return user.getOpaqueUser(); - } - } - return null; - } - - public Optional getChangeSpec() { - return changeSpec; - } - - @Override - public String toString() { - return "LogMessage{" - + "message='" + getMessage() + '\'' - + ", operationCode=" + getOperationCode() - + '}'; - } + public PersistentResource getPersistentResource(); } diff --git a/elide-core/src/main/java/com/yahoo/elide/audit/LogMessageImpl.java b/elide-core/src/main/java/com/yahoo/elide/audit/LogMessageImpl.java new file mode 100644 index 0000000000..0cb46e517c --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/audit/LogMessageImpl.java @@ -0,0 +1,175 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.audit; + +import com.yahoo.elide.annotation.Audit; +import com.yahoo.elide.core.ResourceLineage; +import com.yahoo.elide.security.ChangeSpec; +import com.yahoo.elide.security.PersistentResource; +import com.yahoo.elide.security.User; + +import de.odysseus.el.ExpressionFactoryImpl; +import de.odysseus.el.util.SimpleContext; +import lombok.EqualsAndHashCode; +import lombok.Getter; +import lombok.ToString; + +import java.text.MessageFormat; +import java.util.List; +import java.util.Optional; +import java.util.stream.Collectors; + +import javax.el.ELException; +import javax.el.ExpressionFactory; +import javax.el.PropertyNotFoundException; +import javax.el.ValueExpression; + +/** + * An audit log message that can be logged to a logger. + */ +@ToString +@EqualsAndHashCode +public class LogMessageImpl implements LogMessage { + //Supposedly this is thread safe. + private static final ExpressionFactory EXPRESSION_FACTORY = new ExpressionFactoryImpl(); + private static final String[] EMPTY_STRING_ARRAY = new String[0]; + + private final String template; + private final String[] expressions; + + @Getter + private final int operationCode; + + @Getter + private final Optional changeSpec; + + @Getter + private final User user; + + @Getter + private final PersistentResource persistentResource; + + /** + * Construct a log message that does not involve any templating. + * @param template - The unsubstituted text that will be logged. + * @param code - The operation code of the auditable action. + */ + public LogMessageImpl(String template, int code) { + this(template, null, EMPTY_STRING_ARRAY, code, Optional.empty()); + } + + /** + * Construct a log message from an Audit annotation and the record that was updated in some way. + * @param audit - The annotation containing the type of operation (UPDATE, DELETE, CREATE) + * @param record - The modified record + * @param changeSpec - Change spec of modified elements (if logging object change). empty otherwise + * @throws InvalidSyntaxException if the Audit annotation has invalid syntax. + */ + public LogMessageImpl(Audit audit, PersistentResource record, Optional changeSpec) + throws InvalidSyntaxException { + this(audit.logStatement(), record, audit.logExpressions(), audit.operation(), changeSpec); + } + + /** + * Construct a log message. + * @param template - The log message template that requires variable substitution. + * @param record - The record which will serve as the data to substitute. + * @param expressions - A set of UEL expressions that reference record. + * @param code - The operation code of the auditable action. + * @param changeSpec - the change spec that we want to log + * @throws InvalidSyntaxException the invalid syntax exception + */ + public LogMessageImpl(String template, + PersistentResource record, + String[] expressions, + int code, + Optional changeSpec) throws InvalidSyntaxException { + this.template = template; + this.persistentResource = record; + this.expressions = expressions; + this.operationCode = code; + this.changeSpec = changeSpec; + this.user = (record == null ? null : record.getRequestScope().getUser()); + } + + @Override + public String getMessage() { + final SimpleContext ctx = new SimpleContext(); + final SimpleContext singleElementContext = new SimpleContext(); + + if (persistentResource != null) { + /* Create a new lineage which includes the passed in record */ + com.yahoo.elide.core.PersistentResource internalResource = ( + com.yahoo.elide.core.PersistentResource) persistentResource; + ResourceLineage lineage = new ResourceLineage(internalResource.getLineage(), internalResource); + + for (String name : lineage.getKeys()) { + List values = lineage.getRecord(name); + + final ValueExpression expression; + final ValueExpression singleElementExpression; + if (values.size() == 1) { + expression = EXPRESSION_FACTORY.createValueExpression(values.get(0).getObject(), Object.class); + singleElementExpression = expression; + } else { + List objects = values.stream().map(PersistentResource::getObject) + .collect(Collectors.toList()); + expression = EXPRESSION_FACTORY.createValueExpression(objects, List.class); + singleElementExpression = EXPRESSION_FACTORY.createValueExpression(values.get(values.size() - 1) + .getObject(), Object.class); + } + ctx.setVariable(name, expression); + singleElementContext.setVariable(name, singleElementExpression); + } + + final Object user = getUser().getOpaqueUser(); + if (user != null) { + final ValueExpression opaqueUserValueExpression = EXPRESSION_FACTORY + .createValueExpression( + user, Object.class + ); + ctx.setVariable("opaqueUser", opaqueUserValueExpression); + singleElementContext.setVariable("opaqueUser", opaqueUserValueExpression); + } + } + + Object[] results = new Object[expressions.length]; + for (int idx = 0; idx < results.length; idx++) { + String expressionText = expressions[idx]; + + final ValueExpression expression; + final ValueExpression singleElementExpression; + try { + expression = EXPRESSION_FACTORY.createValueExpression(ctx, expressionText, Object.class); + singleElementExpression = + EXPRESSION_FACTORY.createValueExpression(singleElementContext, expressionText, Object.class); + } catch (ELException e) { + throw new InvalidSyntaxException(e); + } + + Object result; + try { + // Single element expressions are intended to allow for access to ${entityType.field} when there are + // multiple "entityType" types listed in the lineage. Without this, any access to an entityType + // without an explicit list index would otherwise result in a 500. Similarly, since we already + // supported lists (i.e. the ${entityType[idx].field} syntax), this also continues to support that. + // It should be noted, however, that list indexing is somewhat brittle unless properly accounted for + // from all possible paths. + result = singleElementExpression.getValue(singleElementContext); + } catch (PropertyNotFoundException e) { + // Try list syntax if not single element + result = expression.getValue(ctx); + } + results[idx] = result; + } + + try { + return MessageFormat.format(template, results); + } catch (IllegalArgumentException e) { + throw new InvalidSyntaxException(e); + } + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/audit/Slf4jLogger.java b/elide-core/src/main/java/com/yahoo/elide/audit/Slf4jLogger.java index f649c369f5..181e1b2f73 100644 --- a/elide-core/src/main/java/com/yahoo/elide/audit/Slf4jLogger.java +++ b/elide-core/src/main/java/com/yahoo/elide/audit/Slf4jLogger.java @@ -5,10 +5,7 @@ */ package com.yahoo.elide.audit; -import com.yahoo.elide.core.RequestScope; - import lombok.extern.slf4j.Slf4j; - import java.io.IOException; /** @@ -18,7 +15,7 @@ public class Slf4jLogger extends AuditLogger { @Override - public void commit(RequestScope requestScope) throws IOException { + public void commit() throws IOException { try { for (LogMessage message : messages.get()) { log.info("{} {} {}", System.currentTimeMillis(), message.getOperationCode(), message.getMessage()); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java index 8344da8fc8..32522a9ccd 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java @@ -8,10 +8,10 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import java.io.Closeable; @@ -268,9 +268,10 @@ default boolean supportsSorting(Class entityClass, Sorting sorting) { /** * Whether or not the transaction can paginate the provided class. * @param entityClass The entity class that is being paged. + * @param expression The filter expression * @return true if pagination is possible */ - default boolean supportsPagination(Class entityClass) { + default boolean supportsPagination(Class entityClass, FilterExpression expression) { return true; } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index 78f21b2b42..ee096a0708 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -24,7 +24,6 @@ import com.yahoo.elide.security.checks.Check; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; -import com.yahoo.elide.security.checks.prefab.Common; import com.yahoo.elide.security.checks.prefab.Role; import com.yahoo.elide.utils.ClassScanner; import com.yahoo.elide.utils.coerce.CoerceUtil; @@ -143,7 +142,6 @@ private void initializeChecks() { addPrefabCheck("Prefab.Role.None", Role.NONE.class); addPrefabCheck("Prefab.Collections.AppendOnly", AppendOnly.class); addPrefabCheck("Prefab.Collections.RemoveOnly", RemoveOnly.class); - addPrefabCheck("Prefab.Common.UpdateOnCreate", Common.UpdateOnCreate.class); } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java index e1aa4b7419..4aeab2a1f2 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java @@ -13,6 +13,7 @@ import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.audit.InvalidSyntaxException; import com.yahoo.elide.audit.LogMessage; +import com.yahoo.elide.audit.LogMessageImpl; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; import com.yahoo.elide.core.exceptions.InternalServerErrorException; import com.yahoo.elide.core.exceptions.InvalidAttributeException; @@ -23,8 +24,6 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.models.Data; import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; @@ -34,6 +33,8 @@ import com.yahoo.elide.request.Argument; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.permissions.ExpressionResult; import com.yahoo.elide.utils.coerce.CoerceUtil; @@ -341,7 +342,7 @@ public static Set loadRecords( .copyOf() .filterExpression(filterExpression) .sorting(sorting) - .pagination(Optional.ofNullable(pagination).map(p -> p.evaluate(loadClass)).orElse(null)) + .pagination(pagination) .build(); Set existingResources = filter(ReadPermission.class, @@ -1050,9 +1051,6 @@ private Set getRelationUnchecked(com.yahoo.elide.request.Rel throw new InvalidAttributeException(relationName, this.getType()); } - Optional computedPagination = Optional.ofNullable(pagination) - .map(p -> p.evaluate(relationClass)); - //Invoke filterExpressionCheck and then merge with filterExpression. Optional permissionFilter = getPermissionFilterExpression(relationClass, requestScope); Optional computedFilters = Optional.ofNullable(filterExpression); @@ -1069,7 +1067,7 @@ private Set getRelationUnchecked(com.yahoo.elide.request.Rel .projection(relationship.getProjection().copyOf() .filterExpression(computedFilters.orElse(null)) .sorting(sorting) - .pagination(computedPagination.orElse(null)) + .pagination(pagination) .build() ).build(); @@ -1767,7 +1765,7 @@ protected void auditField(final ChangeSpec changeSpec) { } for (Audit annotation : annotations) { if (annotation.action().length == 1 && annotation.action()[0] == Audit.Action.UPDATE) { - LogMessage message = new LogMessage(annotation, this, Optional.of(changeSpec)); + LogMessage message = new LogMessageImpl(annotation, this, Optional.of(changeSpec)); getRequestScope().getAuditLogger().log(message); } else { throw new InvalidSyntaxException("Only Audit.Action.UPDATE is allowed on fields."); @@ -1790,7 +1788,7 @@ protected void auditClass(Audit.Action action, ChangeSpec changeSpec) { for (Audit annotation : annotations) { for (Audit.Action auditAction : annotation.action()) { if (auditAction == action) { // compare object reference - LogMessage message = new LogMessage(annotation, this, Optional.ofNullable(changeSpec)); + LogMessage message = new LogMessageImpl(annotation, this, Optional.ofNullable(changeSpec)); getRequestScope().getAuditLogger().log(message); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java index f036d0ea16..a714c7010e 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java @@ -26,8 +26,6 @@ import com.yahoo.elide.core.filter.dialect.ParseException; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.request.EntityProjection; @@ -67,8 +65,6 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final AuditLogger auditLogger; @Getter private final Optional> queryParams; @Getter private final Map> sparseFields; - @Getter private final Pagination pagination; - @Getter private final Sorting sorting; @Getter private final PermissionExecutor permissionExecutor; @Getter private final ObjectEntityCache objectEntityCache; @Getter private final Set newPersistentResources; @@ -181,12 +177,8 @@ public RequestScope(String path, } this.sparseFields = parseSparseFields(queryParams); - this.sorting = Sorting.parseQueryParams(queryParams); - this.pagination = Pagination.parseQueryParams(queryParams, this.getElideSettings()); } else { this.sparseFields = Collections.emptyMap(); - this.sorting = Sorting.getDefaultEmptyInstance(); - this.pagination = Pagination.getDefaultPagination(this.getElideSettings()); } } @@ -207,8 +199,6 @@ protected RequestScope(String path, JsonApiDocument jsonApiDocument, RequestScop this.auditLogger = outerRequestScope.auditLogger; this.queryParams = Optional.empty(); this.sparseFields = Collections.emptyMap(); - this.sorting = Sorting.getDefaultEmptyInstance(); - this.pagination = Pagination.getDefaultPagination(outerRequestScope.getElideSettings()); this.objectEntityCache = outerRequestScope.objectEntityCache; this.newPersistentResources = outerRequestScope.newPersistentResources; this.permissionExecutor = outerRequestScope.getPermissionExecutor(); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java index a1ba45304d..e8fa05d324 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java @@ -10,10 +10,10 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.TransactionException; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import lombok.extern.slf4j.Slf4j; import java.io.IOException; @@ -175,7 +175,7 @@ public boolean supportsSorting(Class entityClass, Sorting sorting) { } @Override - public boolean supportsPagination(Class entityClass) { + public boolean supportsPagination(Class entityClass, FilterExpression expression) { return false; } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java index 3eba0004b9..679cf11995 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java @@ -7,7 +7,6 @@ package com.yahoo.elide.core.datastore.inmemory; import com.yahoo.elide.core.DataStoreTransaction; -import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.PersistentResource; import com.yahoo.elide.core.RequestScope; @@ -15,11 +14,11 @@ import com.yahoo.elide.core.filter.expression.FilterPredicatePushdownExtractor; import com.yahoo.elide.core.filter.expression.InMemoryExecutionVerifier; import com.yahoo.elide.core.filter.expression.InMemoryFilterExecutor; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import org.apache.commons.lang3.tuple.Pair; @@ -258,8 +257,7 @@ private Object fetchData(DataFetcher fetcher, Optional inMemorySort = sortSplit.getRight(); Pair, Optional> paginationSplit = splitPagination(entityClass, - pagination, inMemoryFilter.isPresent(), inMemorySort.isPresent()); - + filterExpression.orElse(null), pagination, inMemoryFilter.isPresent(), inMemorySort.isPresent()); Optional dataStorePagination = paginationSplit.getLeft(); Optional inMemoryPagination = paginationSplit.getRight(); @@ -279,7 +277,6 @@ private Object fetchData(DataFetcher fetcher, return sortAndPaginateLoadedData( loadedRecords, - entityClass, inMemorySort, inMemoryPagination, scope); @@ -287,7 +284,6 @@ private Object fetchData(DataFetcher fetcher, private Iterable sortAndPaginateLoadedData(Iterable loadedRecords, - Class entityClass, Optional sorting, Optional pagination, RequestScope scope) { @@ -297,10 +293,8 @@ private Iterable sortAndPaginateLoadedData(Iterable loadedRecord return loadedRecords; } - EntityDictionary dictionary = scope.getDictionary(); - Map sortRules = sorting - .map((s) -> s.getValidSortingRules(entityClass, dictionary)) + .map((s) -> s.getSortingPaths()) .orElse(new HashMap<>()); // No sorting required for this type & no pagination. @@ -333,8 +327,8 @@ private List paginateInMemory(List records, Pagination paginatio endIdx = records.size(); } - if (pagination.isGenerateTotals()) { - pagination.setPageTotals(records.size()); + if (pagination.returnPageTotals()) { + pagination.setPageTotals((long) records.size()); } return records.subList(offset, endIdx); } @@ -457,11 +451,12 @@ private Pair, Optional> splitSorting( */ private Pair, Optional> splitPagination( Class entityClass, + FilterExpression expression, Optional pagination, boolean filteredInMemory, boolean sortedInMemory ) { - if (!tx.supportsPagination(entityClass) + if (!tx.supportsPagination(entityClass, expression) || filteredInMemory || sortedInMemory) { return Pair.of(Optional.empty(), pagination); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java index 3171047efc..5bb8f5fba0 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java @@ -9,10 +9,10 @@ import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; import lombok.Data; @@ -91,8 +91,8 @@ public boolean supportsSorting(Class entityClass, Sorting sorting) { } @Override - public boolean supportsPagination(Class entityClass) { - return tx.supportsPagination(entityClass); + public boolean supportsPagination(Class entityClass, FilterExpression expression) { + return tx.supportsPagination(entityClass, expression); } @Override diff --git a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java b/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java deleted file mode 100644 index 860146762e..0000000000 --- a/elide-core/src/main/java/com/yahoo/elide/core/pagination/Pagination.java +++ /dev/null @@ -1,360 +0,0 @@ -/* - * Copyright 2015, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.core.pagination; - -import com.yahoo.elide.ElideSettings; -import com.yahoo.elide.annotation.Paginate; -import com.yahoo.elide.core.exceptions.InvalidValueException; - -import com.google.common.collect.ImmutableMap; - -import lombok.Getter; -import lombok.ToString; - -import java.util.HashMap; -import java.util.Map; -import java.util.Optional; -import java.util.stream.Collectors; - -import javax.ws.rs.core.MultivaluedMap; - -/** - * Encapsulates the pagination strategy. - */ - -@ToString -public class Pagination { - /** - * Denotes the internal field names for paging. - */ - public enum PaginationKey { offset, number, size, limit, totals } - - public static final int DEFAULT_OFFSET = 0; - public static final int DEFAULT_PAGE_LIMIT = 500; - public static final int MAX_PAGE_LIMIT = 10000; - - // For specifying which page of records is to be returned in the response - public static final String PAGE_NUMBER_KEY = "page[number]"; - - // For specifying the page size - essentially an alias for page[limit] - public static final String PAGE_SIZE_KEY = "page[size]"; - - // For specifying the first row to be returned in the response - public static final String PAGE_OFFSET_KEY = "page[offset]"; - - // For limiting the number of records returned - public static final String PAGE_LIMIT_KEY = "page[limit]"; - - // For requesting total pages/records be included in the response page meta data - public static final String PAGE_TOTALS_KEY = "page[totals]"; - - public static final Map PAGE_KEYS = new HashMap<>(); - static { - PAGE_KEYS.put(PAGE_NUMBER_KEY, PaginationKey.number); - PAGE_KEYS.put(PAGE_SIZE_KEY, PaginationKey.size); - PAGE_KEYS.put(PAGE_OFFSET_KEY, PaginationKey.offset); - PAGE_KEYS.put(PAGE_LIMIT_KEY, PaginationKey.limit); - PAGE_KEYS.put(PAGE_TOTALS_KEY, PaginationKey.totals); - } - - private long pageTotals = 0; - - private static final String PAGE_KEYS_CSV = PAGE_KEYS.keySet().stream().collect(Collectors.joining(", ")); - - // For holding the page query parameters until they can be evaluated - private Map pageData; - - @Getter - private int offset; - - @Getter - private int limit; - - @Getter - private boolean generateTotals; - - private final int defaultMaxPageSize; - private final int defaultPageSize; - - private Pagination(Map pageData, int defaultMaxPageSize, int defaultPageSize) { - this.pageData = pageData; - this.defaultMaxPageSize = defaultMaxPageSize; - this.defaultPageSize = defaultPageSize; - } - - /** - * Set limit. - * - * @param perPage page size. - */ - public void setLimit(Integer perPage) { - this.limit = perPage; - pageData.put(PaginationKey.limit, perPage); - } - - public void setOffset(Integer offset) { - this.offset = offset; - pageData.put(PaginationKey.offset, offset); - } - - /** - * TODO - Refactor Pagination. - * IMPORTANT - This method should only be used for testing until Pagination is refactored. The - * member field values of this class change depending on evaluation later from the Pagination annotation. - * The existing implementation is too complex because logic resides in the wrong places. - * - * @param limit The page size - * @param offset The page offset - * @param generatePageTotals Whether or not to return page totals - * @return A new pagination object. - */ - public static Pagination fromOffsetAndLimit(int limit, int offset, boolean generatePageTotals) { - - ImmutableMap.Builder pageData = ImmutableMap.builder() - .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) - .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), limit); - - if (generatePageTotals) { - pageData.put(PAGE_KEYS.get(PAGE_TOTALS_KEY), 1); - } - - Pagination result = new Pagination(pageData.build(), MAX_PAGE_LIMIT, DEFAULT_PAGE_LIMIT); - result.offset = offset; - result.limit = limit; - result.generateTotals = generatePageTotals; - return result; - } - - /** - * Given an offset and first parameter from GraphQL, generate page and pageSize values. - * - * @param firstOpt Provided first string - * @param offsetOpt Provided offset string - * @param generatePageTotals True if page totals should be generated, false otherwise - * @param elideSettings Elide settings object containing default pagination values - * @return The new Pagination object. - */ - public static Optional fromOffsetAndFirst(Optional firstOpt, - Optional offsetOpt, - boolean generatePageTotals, - ElideSettings elideSettings) { - return firstOpt.map(firstString -> { - int offset; - int first; - - try { - offset = offsetOpt.map(Integer::parseInt).orElse(0); - first = Integer.parseInt(firstString); - } catch (NumberFormatException e) { - throw new InvalidValueException("Offset and first must be numeric values."); - } - - if (offset < 0) { - throw new InvalidValueException("Offset values must be non-negative."); - } else if (first < 1) { - throw new InvalidValueException("Limit values must be positive."); - } - - ImmutableMap.Builder pageData = ImmutableMap.builder() - .put(PAGE_KEYS.get(PAGE_OFFSET_KEY), offset) - .put(PAGE_KEYS.get(PAGE_LIMIT_KEY), first); - if (generatePageTotals) { - pageData.put(PAGE_KEYS.get(PAGE_TOTALS_KEY), 1); - } - - return Optional.of(getPagination(pageData.build(), elideSettings)); - }).orElseGet(() -> { - if (generatePageTotals) { - Pagination pagination = getDefaultPagination(elideSettings); - pagination.pageData.put(PAGE_KEYS.get(PAGE_TOTALS_KEY), 1); - return Optional.of(pagination); - } - return Optional.empty(); - }); - } - - /** - * Given json-api paging params, generate page and pageSize values from query params. - * - * @param queryParams The page queryParams (ImmuatableMultiValueMap). - * @param elideSettings Elide settings containing pagination default limits - * @return The new Pagination object. - * @throws InvalidValueException invalid query parameter - */ - public static Pagination parseQueryParams(final MultivaluedMap queryParams, - ElideSettings elideSettings) - throws InvalidValueException { - final Map pageData = new HashMap<>(); - queryParams.entrySet() - .forEach(paramEntry -> { - final String queryParamKey = paramEntry.getKey(); - if (PAGE_KEYS.containsKey(queryParamKey)) { - PaginationKey paginationKey = PAGE_KEYS.get(queryParamKey); - if (paginationKey.equals(PaginationKey.totals)) { - // page[totals] is a valueless parameter, use value of 0 just so that its presence can - // be recorded in the map - pageData.put(paginationKey, 0); - } else { - final String value = paramEntry.getValue().get(0); - try { - int intValue = Integer.parseInt(value, 10); - pageData.put(paginationKey, intValue); - } catch (NumberFormatException e) { - throw new InvalidValueException("page values must be integers"); - } - } - } else if (queryParamKey.startsWith("page[")) { - throw new InvalidValueException("Invalid Pagination Parameter. Accepted values are " - + PAGE_KEYS_CSV); - } - }); - return getPagination(pageData, elideSettings); - } - - /** - * Sets the total number of records for the paginated query. - * @param total the total number of records found - */ - public void setPageTotals(long total) { - this.pageTotals = total; - } - - /** - * Fetches the total number of records of the paginated query. - * @return page totals - */ - public long getPageTotals() { - return pageTotals; - } - - /** - * Construct a pagination object from page data and elide settings. - * - * @param pageData Map containing pagination information - * @param elideSettings Settings containing pagination defaults - * @return Pagination object - */ - private static Pagination getPagination(Map pageData, ElideSettings elideSettings) { - // Decidedly default settings until evaluate is called (a call to evaluate from the datastore will update this): - Pagination result = new Pagination(pageData, - elideSettings.getDefaultMaxPageSize(), elideSettings.getDefaultPageSize()); - result.offset = 0; - result.limit = elideSettings.getDefaultPageSize(); - return result; - } - - /** - * Evaluates the pagination variables for default limits. - * - * @param defaultLimit the default page size - * @param maxLimit a hard upper limit on page size - * @return the calculated {@link Pagination} - */ - private Pagination evaluate(int defaultLimit, int maxLimit) { - if (hasInvalidCombination(pageData)) { - throw new InvalidValueException("Invalid usage of pagination parameters."); - } - if (pageData.containsKey(PaginationKey.size) || pageData.containsKey(PaginationKey.number)) { - pageByPages(defaultLimit, maxLimit); - } else if (pageData.containsKey(PaginationKey.limit) || pageData.containsKey(PaginationKey.offset)) { - pageByOffset(defaultLimit, maxLimit); - } else { - limit = defaultLimit; - offset = 0; - } - - generateTotals = pageData.containsKey(PaginationKey.totals); - - return this; - } - - private boolean hasInvalidCombination(Map pageData) { - return (pageData.containsKey(PaginationKey.size) || pageData.containsKey(PaginationKey.number)) - && (pageData.containsKey(PaginationKey.limit) || pageData.containsKey(PaginationKey.offset)); - } - - private void pageByOffset(int defaultLimit, int maxLimit) { - limit = pageData.containsKey(PaginationKey.limit) ? pageData.get(PaginationKey.limit) : defaultLimit; - if (limit > maxLimit) { - throw new InvalidValueException("page[limit] value must be less than or equal to " + maxLimit); - } - if (limit < 0) { - throw new InvalidValueException("page[limit] value must contain a positive value"); - } - - offset = pageData.containsKey(PaginationKey.offset) ? pageData.get(PaginationKey.offset) : 0; - if (offset < 0) { - throw new InvalidValueException("page[offset] must contain a positive values."); - } - } - - private void pageByPages(int defaultLimit, int maxLimit) { - limit = pageData.containsKey(PaginationKey.size) ? pageData.get(PaginationKey.size) : defaultLimit; - if (limit > maxLimit) { - throw new InvalidValueException("page[size] value must be less than or equal to " + maxLimit); - } - if (limit < 0) { - throw new InvalidValueException("page[size] must contain a positive value."); - } - - int pageNumber = pageData.containsKey(PaginationKey.number) ? pageData.get(PaginationKey.number) : 1; - if (pageNumber < 1) { - throw new InvalidValueException("page[number] must contain a positive value."); - } - - offset = (pageNumber - 1) * limit; - } - - /** - * Evaluates the pagination variables. Uses the Paginate annotation if it has been set for the entity to be - * queried. - * - * @param entityClass Entity class to paginate - * @return the calculated {@link Pagination} - */ - public Pagination evaluate(final Class entityClass) { - Paginate paginate = - entityClass != null ? (Paginate) entityClass.getAnnotation(Paginate.class) : null; - - int defaultLimit = paginate != null ? paginate.defaultLimit() : defaultPageSize; - int maxLimit = paginate != null ? paginate.maxLimit() : defaultMaxPageSize; - - evaluate(defaultLimit, maxLimit); - - generateTotals = generateTotals && (paginate == null || paginate.countable()); - - return this; - } - - /** - * Know if this is the default instance. - * @return The default pagination values. - */ - public boolean isDefaultInstance() { - return pageData.isEmpty(); - } - - /** - * Alias for isDefault. - * @return true if there are no pagination rules - */ - public boolean isEmpty() { - return isDefaultInstance(); - } - - /** - * Default Instance. - * @param elideSettings general Elide settings - * @return The default instance. - */ - public static Pagination getDefaultPagination(ElideSettings elideSettings) { - Pagination defaultPagination = new Pagination(new HashMap<>(), - elideSettings.getDefaultMaxPageSize(), elideSettings.getDefaultPageSize()); - defaultPagination.offset = DEFAULT_OFFSET; - defaultPagination.limit = DEFAULT_PAGE_LIMIT; - return defaultPagination; - } -} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/pagination/PaginationImpl.java b/elide-core/src/main/java/com/yahoo/elide/core/pagination/PaginationImpl.java new file mode 100644 index 0000000000..d419a8786b --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/core/pagination/PaginationImpl.java @@ -0,0 +1,269 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.core.pagination; + +import com.yahoo.elide.ElideSettings; +import com.yahoo.elide.annotation.Paginate; +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.request.Pagination; + +import lombok.EqualsAndHashCode; +import lombok.Getter; +import lombok.Setter; +import lombok.ToString; + +import java.util.HashMap; +import java.util.Map; +import java.util.Optional; +import java.util.stream.Collectors; + +import javax.ws.rs.core.MultivaluedMap; + +/** + * Holds state associated with pagination. + */ +@ToString +@EqualsAndHashCode +public class PaginationImpl implements Pagination { + /** + * Denotes the internal field names for paging. + */ + public enum PaginationKey { offset, number, size, limit, totals } + + // For specifying which page of records is to be returned in the response + public static final String PAGE_NUMBER_KEY = "page[number]"; + + // For specifying the page size - essentially an alias for page[limit] + public static final String PAGE_SIZE_KEY = "page[size]"; + + // For specifying the first row to be returned in the response + public static final String PAGE_OFFSET_KEY = "page[offset]"; + + // For limiting the number of records returned + public static final String PAGE_LIMIT_KEY = "page[limit]"; + + // For requesting total pages/records be included in the response page meta data + public static final String PAGE_TOTALS_KEY = "page[totals]"; + + public static final Map PAGE_KEYS = new HashMap<>(); + static { + PAGE_KEYS.put(PAGE_NUMBER_KEY, PaginationKey.number); + PAGE_KEYS.put(PAGE_SIZE_KEY, PaginationKey.size); + PAGE_KEYS.put(PAGE_OFFSET_KEY, PaginationKey.offset); + PAGE_KEYS.put(PAGE_LIMIT_KEY, PaginationKey.limit); + PAGE_KEYS.put(PAGE_TOTALS_KEY, PaginationKey.totals); + } + + @Getter + @Setter + private Long pageTotals = 0L; + + private static final String PAGE_KEYS_CSV = PAGE_KEYS.keySet().stream().collect(Collectors.joining(", ")); + + @Getter + private Integer offset; + + @Getter + private Integer limit; + + private Boolean generateTotals; + + private Boolean isDefault; + + @Getter + private Class entityClass; + + /** + * Constructor. + * @param entityClass The type of collection we are paginating. + * @param clientOffset The client requested offset or null if not provided. + * @param clientLimit The client requested limit or null if not provided. + * @param systemDefaultLimit The system default limit (in terms of records). + * @param systemMaxLimit The system max limit (in terms of records). + * @param generateTotals Whether to return the total number of records. + * @param pageByPages Whether to page by pages or records. + */ + public PaginationImpl(Class entityClass, + Integer clientOffset, + Integer clientLimit, + int systemDefaultLimit, + int systemMaxLimit, + Boolean generateTotals, + Boolean pageByPages) { + + this.entityClass = entityClass; + this.isDefault = (clientOffset == null && clientLimit == null && generateTotals == null); + + Paginate paginate = entityClass != null ? (Paginate) entityClass.getAnnotation(Paginate.class) : null; + + this.limit = clientLimit != null + ? clientLimit + : (paginate != null ? paginate.defaultLimit() : systemDefaultLimit); + + int maxLimit = paginate != null ? paginate.maxLimit() : systemMaxLimit; + + String pageSizeLabel = pageByPages ? "size" : "limit"; + + if (limit > maxLimit && !isDefault) { + throw new InvalidValueException("Pagination " + + pageSizeLabel + " must be less than or equal to " + maxLimit); + } + if (limit < 1) { + throw new InvalidValueException("Pagination " + + pageSizeLabel + " must contain a positive, non-zero value."); + } + + this.generateTotals = generateTotals != null && generateTotals && (paginate == null || paginate.countable()); + + if (pageByPages) { + int pageNumber = clientOffset != null ? clientOffset : 1; + if (pageNumber < 1) { + throw new InvalidValueException("Pagination number must be a positive, non-zero value."); + } + this.offset = (pageNumber - 1) * limit; + } else { + this.offset = clientOffset != null ? clientOffset : 0; + + if (offset < 0) { + throw new InvalidValueException("Pagination offset must contain a positive value."); + } + } + } + + /** + * Whether or not the client requested to return page totals. + * @return true if page totals should be returned. + */ + @Override + public Boolean returnPageTotals() { + return generateTotals; + } + + /** + * Whether or not the client requested pagination or the system defaults are in effect. + * @return True if the system defaults are in effect. + */ + @Override + public Boolean isDefaultInstance() { + return isDefault; + } + + /** + * Given json-api paging params, generate page and pageSize values from query params. + * + * @param entityClass The collection type. + * @param queryParams The page queryParams. + * @param elideSettings Elide settings containing pagination default limits + * @return The new Pagination object. + * @throws InvalidValueException invalid query parameter + */ + public static PaginationImpl parseQueryParams(Class entityClass, + final Optional> queryParams, + ElideSettings elideSettings) + throws InvalidValueException { + + if (! queryParams.isPresent()) { + return getDefaultPagination(entityClass, elideSettings); + } + + final Map pageData = new HashMap<>(); + queryParams.get().entrySet() + .forEach(paramEntry -> { + final String queryParamKey = paramEntry.getKey(); + if (PAGE_KEYS.containsKey(queryParamKey)) { + PaginationKey paginationKey = PAGE_KEYS.get(queryParamKey); + if (paginationKey.equals(PaginationKey.totals)) { + // page[totals] is a valueless parameter, use value of 0 just so that its presence can + // be recorded in the map + pageData.put(paginationKey, 0); + } else { + final String value = paramEntry.getValue().get(0); + try { + int intValue = Integer.parseInt(value, 10); + pageData.put(paginationKey, intValue); + } catch (NumberFormatException e) { + throw new InvalidValueException("page values must be integers"); + } + } + } else if (queryParamKey.startsWith("page[")) { + throw new InvalidValueException("Invalid Pagination Parameter. Accepted values are " + + PAGE_KEYS_CSV); + } + }); + return getPagination(entityClass, pageData, elideSettings); + } + + + /** + * Construct a pagination object from page data and elide settings. + * + * @param entityClass The collection type. + * @param pageData Map containing pagination information + * @param elideSettings Settings containing pagination defaults + * @return Pagination object + */ + private static PaginationImpl getPagination(Class entityClass, Map pageData, + ElideSettings elideSettings) { + if (hasInvalidCombination(pageData)) { + throw new InvalidValueException("Invalid usage of pagination parameters."); + } + + boolean pageByPages = false; + Integer offset = pageData.getOrDefault(PaginationKey.offset, null); + Integer limit = pageData.getOrDefault(PaginationKey.limit, null); + + if (pageData.containsKey(PaginationKey.size) || pageData.containsKey(PaginationKey.number)) { + pageByPages = true; + offset = pageData.getOrDefault(PaginationKey.number, null); + limit = pageData.getOrDefault(PaginationKey.size, null); + } + + return new PaginationImpl(entityClass, + offset, + limit, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + pageData.containsKey(PaginationKey.totals) ? true : null, + pageByPages); + } + + private static boolean hasInvalidCombination(Map pageData) { + return (pageData.containsKey(PaginationKey.size) || pageData.containsKey(PaginationKey.number)) + && (pageData.containsKey(PaginationKey.limit) || pageData.containsKey(PaginationKey.offset)); + } + + + /** + * Default Instance. + * @param elideSettings general Elide settings + * @return The default instance. + */ + public static PaginationImpl getDefaultPagination(Class entityClass, ElideSettings elideSettings) { + return new PaginationImpl( + entityClass, + null, + null, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + null, + false); + } + + /** + * Default Instance. + * @return The default instance. + */ + public static PaginationImpl getDefaultPagination(Class entityClass) { + return new PaginationImpl( + entityClass, + null, + null, + DEFAULT_PAGE_LIMIT, + MAX_PAGE_LIMIT, + null, + false); + } +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java b/elide-core/src/main/java/com/yahoo/elide/core/sort/SortingImpl.java similarity index 80% rename from elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java rename to elide-core/src/main/java/com/yahoo/elide/core/sort/SortingImpl.java index ec1d091417..624d289bcc 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/sort/Sorting.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/sort/SortingImpl.java @@ -9,13 +9,16 @@ import com.yahoo.elide.core.Path; import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.request.Sorting; import lombok.EqualsAndHashCode; +import lombok.Getter; import lombok.ToString; import java.util.Arrays; import java.util.LinkedHashMap; import java.util.List; import java.util.Map; +import java.util.Optional; import java.util.stream.Collectors; import javax.ws.rs.core.MultivaluedMap; @@ -25,25 +28,29 @@ */ @ToString @EqualsAndHashCode -public class Sorting { - - /** - * Denotes the intended sort type from json-api field. - */ - public enum SortOrder { asc, desc } +public class SortingImpl implements Sorting { private final Map sortRules = new LinkedHashMap<>(); - private static final Sorting DEFAULT_EMPTY_INSTANCE = null; + private static final SortingImpl DEFAULT_EMPTY_INSTANCE = null; private static final String JSONAPI_ID_KEYWORD = "id"; + @Getter + private Class type; + + @Getter + private Map sortingPaths; + /** * Constructs a new Sorting instance. * @param sortingRules The map of sorting rules */ - public Sorting(final Map sortingRules) { + public SortingImpl(final Map sortingRules, Class type, EntityDictionary dictionary) { if (sortingRules != null) { sortRules.putAll(sortingRules); } + + this.type = type; + sortingPaths = getValidSortingRules(type, dictionary); } /** @@ -54,8 +61,8 @@ public Sorting(final Map sortingRules) { * @return The valid sorting rules - validated through the entity dictionary, or empty dictionary * @throws InvalidValueException when sorting values are not valid for the jpa entity */ - public Map getValidSortingRules(final Class entityClass, - final EntityDictionary dictionary) + private Map getValidSortingRules(final Class entityClass, + final EntityDictionary dictionary) throws InvalidValueException { Map returnMap = new LinkedHashMap<>(); for (Map.Entry entry : replaceIdRule(dictionary.getIdFieldName(entityClass)).entrySet()) { @@ -99,6 +106,7 @@ protected static boolean isValidSortRulePath(Path path, EntityDictionary diction * Informs if the structure is default instance. * @return true if this instance is empty - no sorting rules */ + @Override public boolean isDefaultInstance() { return this.sortRules.isEmpty(); } @@ -108,12 +116,18 @@ public boolean isDefaultInstance() { * @param queryParams The query params on the request. * @return The Sorting instance (default or specific). */ - public static Sorting parseQueryParams(final MultivaluedMap queryParams) { - List sortRules = queryParams.entrySet().stream() + public static Sorting parseQueryParams(final Optional> queryParams, + Class type, EntityDictionary dictionary) { + + if (! queryParams.isPresent()) { + return DEFAULT_EMPTY_INSTANCE; + } + + List sortRules = queryParams.get().entrySet().stream() .filter(entry -> entry.getKey().equals("sort")) .map(entry -> entry.getValue().get(0)) .collect(Collectors.toList()); - return parseSortRules(sortRules); + return parseSortRules(sortRules, type, dictionary); } /** @@ -121,8 +135,8 @@ public static Sorting parseQueryParams(final MultivaluedMap quer * @param sortRule Sorting string to parse * @return Sorting object. */ - public static Sorting parseSortRule(String sortRule) { - return parseSortRules(Arrays.asList(sortRule)); + public static Sorting parseSortRule(String sortRule, Class type, EntityDictionary dictionary) { + return parseSortRules(Arrays.asList(sortRule), type, dictionary); } /** @@ -130,7 +144,7 @@ public static Sorting parseSortRule(String sortRule) { * @param sortRules Sorting rules to parse * @return Sorting object containing parsed sort rules */ - private static Sorting parseSortRules(List sortRules) { + private static SortingImpl parseSortRules(List sortRules, Class type, EntityDictionary dictionary) { final Map sortingRules = new LinkedHashMap<>(); for (String sortRule : sortRules) { if (sortRule.contains(",")) { @@ -141,7 +155,7 @@ private static Sorting parseSortRules(List sortRules) { parseSortRule(sortRule, sortingRules); } } - return sortingRules.isEmpty() ? DEFAULT_EMPTY_INSTANCE : new Sorting(sortingRules); + return sortingRules.isEmpty() ? DEFAULT_EMPTY_INSTANCE : new SortingImpl(sortingRules, type, dictionary); } /** diff --git a/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java index b3e4eee9c5..d6efa564f2 100644 --- a/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java +++ b/elide-core/src/main/java/com/yahoo/elide/jsonapi/EntityProjectionMaker.java @@ -11,12 +11,16 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.exceptions.InvalidCollectionException; import com.yahoo.elide.core.filter.expression.FilterExpression; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.generated.parsers.CoreBaseVisitor; import com.yahoo.elide.generated.parsers.CoreParser; import com.yahoo.elide.parsers.JsonApiParser; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.google.common.collect.Sets; import org.apache.commons.lang3.tuple.Pair; @@ -25,7 +29,7 @@ import lombok.Data; import java.util.Arrays; -import java.util.HashSet; +import java.util.LinkedHashSet; import java.util.Map; import java.util.Set; import java.util.function.Function; @@ -136,12 +140,16 @@ public Function, NamedEntityProjection> visitRelationship(CoreParser.Re Class entityClass = getEntityClass(parentClass, entityName); FilterExpression filter = scope.getExpressionForRelation(parentClass, entityName).orElse(null); + Sorting sorting = SortingImpl.parseQueryParams(scope.getQueryParams(), entityClass, dictionary); + Pagination pagination = PaginationImpl.parseQueryParams(entityClass, + scope.getQueryParams(), scope.getElideSettings()); + return NamedEntityProjection.builder() .name(entityName) .projection(EntityProjection.builder() .filterExpression(filter) - .sorting(scope.getSorting()) - .pagination(scope.getPagination()) + .sorting(sorting) + .pagination(pagination) .type(entityClass) .build() ).build(); @@ -261,12 +269,16 @@ private Function, NamedEntityProjection> visitTerminalCollection(CorePa filter = scope.getExpressionForRelation(parentClass, collectionNameText).orElse(null); } + Sorting sorting = SortingImpl.parseQueryParams(scope.getQueryParams(), entityClass, dictionary); + Pagination pagination = PaginationImpl.parseQueryParams(entityClass, + scope.getQueryParams(), scope.getElideSettings()); + return NamedEntityProjection.builder() .name(collectionNameText) .projection(EntityProjection.builder() .filterExpression(filter) - .sorting(scope.getSorting()) - .pagination(scope.getPagination()) + .sorting(sorting) + .pagination(pagination) .relationships(toRelationshipSet(getRequiredRelationships(entityClass))) .attributes(getSparseAttributes(entityClass)) .type(entityClass) @@ -309,7 +321,7 @@ private Map getIncludedRelationships(Class entityCl } private Set getSparseAttributes(Class entityClass) { - Set allAttributes = new HashSet<>(dictionary.getAttributes(entityClass)); + Set allAttributes = new LinkedHashSet<>(dictionary.getAttributes(entityClass)); Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { @@ -325,7 +337,7 @@ private Set getSparseAttributes(Class entityClass) { } private Map getSparseRelationships(Class entityClass) { - Set allRelationships = new HashSet<>(dictionary.getRelationships(entityClass)); + Set allRelationships = new LinkedHashSet<>(dictionary.getRelationships(entityClass)); Set sparseFieldsForEntity = sparseFields.get(dictionary.getJsonAliasFor(entityClass)); if (sparseFieldsForEntity == null || sparseFieldsForEntity.isEmpty()) { @@ -368,7 +380,7 @@ private Set getIncludePaths(Class entityClass) { .collect(Collectors.toSet()); } - return new HashSet<>(); + return new LinkedHashSet<>(); } private Set toRelationshipSet(Map relationships) { diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java index 7ca47bdd33..0ab469974c 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/CollectionTerminalState.java @@ -14,7 +14,6 @@ import com.yahoo.elide.core.exceptions.InvalidObjectIdentifierException; import com.yahoo.elide.core.exceptions.InvalidValueException; import com.yahoo.elide.core.exceptions.UnknownEntityException; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.jsonapi.JsonApiMapper; import com.yahoo.elide.jsonapi.document.processors.DocumentProcessor; import com.yahoo.elide.jsonapi.document.processors.IncludedProcessor; @@ -24,6 +23,7 @@ import com.yahoo.elide.jsonapi.models.Relationship; import com.yahoo.elide.jsonapi.models.Resource; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.fasterxml.jackson.core.JsonProcessingException; import com.fasterxml.jackson.databind.JsonNode; @@ -78,16 +78,20 @@ public Supplier> handleGet(StateContext state) { DocumentProcessor includedProcessor = new IncludedProcessor(); includedProcessor.execute(jsonApiDocument, collection, queryParams); + Pagination pagination = parentProjection.getPagination(); + if (parent.isPresent()) { + pagination = parentProjection.getRelationship(relationName.get()).get().getProjection().getPagination(); + } + // Add pagination meta data - Pagination pagination = requestScope.getPagination(); - if (!pagination.isEmpty()) { + if (!pagination.isDefaultInstance()) { Map pageMetaData = new HashMap<>(); pageMetaData.put("number", (pagination.getOffset() / pagination.getLimit()) + 1); pageMetaData.put("limit", pagination.getLimit()); // Get total records if it has been requested and add to the page meta data - if (pagination.isGenerateTotals()) { + if (pagination.returnPageTotals()) { Long totalRecords = pagination.getPageTotals(); pageMetaData.put("totalPages", totalRecords / pagination.getLimit() + ((totalRecords % pagination.getLimit()) > 0 ? 1 : 0)); diff --git a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java index da7a5c8caa..fd199678db 100644 --- a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java +++ b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java @@ -7,8 +7,6 @@ package com.yahoo.elide.request; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.google.common.collect.Sets; import lombok.AllArgsConstructor; diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java b/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java new file mode 100644 index 0000000000..a0d69f8170 --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java @@ -0,0 +1,64 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +/** + * Represents a client request to paginate a collection. + */ +public interface Pagination { + + /** + * Default offset (in records) it client does not provide one. + */ + public static final int DEFAULT_OFFSET = 0; + + /** + * Default page limit (in records) it client does not provide one. + */ + public static final int DEFAULT_PAGE_LIMIT = 500; + + /** + * Maximum allowable page limit (in records). + */ + public static final int MAX_PAGE_LIMIT = 10000; + + /** + * Get the page offset. + * @return record offset. + */ + Integer getOffset(); + + /** + * Get the page limit. + * @return record limit. + */ + Integer getLimit(); + + /** + * Whether or not to fetch the collection size or not. + * @return true if the client wants the total size of the collection. + */ + Boolean returnPageTotals(); + + /** + * Get the total size of the collection + * @return total record count. + */ + Long getPageTotals(); + + /** + * Set the total size of the collection. + * @param pageTotals the total size. + */ + void setPageTotals(Long pageTotals); + + /** + * Is this the default instance (not present). + * @return true if pagination wasn't requested. False otherwise. + */ + public Boolean isDefaultInstance(); +} diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Sorting.java b/elide-core/src/main/java/com/yahoo/elide/request/Sorting.java new file mode 100644 index 0000000000..67a25bcfcb --- /dev/null +++ b/elide-core/src/main/java/com/yahoo/elide/request/Sorting.java @@ -0,0 +1,41 @@ +/* + * Copyright 2019, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ + +package com.yahoo.elide.request; + +import com.yahoo.elide.core.Path; + +import java.util.Map; + +/** + * Represents a client request to sort a collection. + */ +public interface Sorting { + + /** + * Denotes the intended sort direction (ascending or descending). + */ + public enum SortOrder { asc, desc } + + /** + * Return an ordered map of paths and their sort order. + * @param The type to sort. + * @return An ordered map of paths and their sort order. + */ + public Map getSortingPaths(); + + /** + * Get the type of the collection to sort. + * @return the collection type. + */ + public Class getType(); + + /** + * Is this sorting the default instance (not present). + * @return true if sorting wasn't requested. False otherwise. + */ + public boolean isDefaultInstance(); +} diff --git a/elide-core/src/test/java/com/yahoo/elide/audit/LogMessageTest.java b/elide-core/src/test/java/com/yahoo/elide/audit/LogMessageImplTest.java similarity index 86% rename from elide-core/src/test/java/com/yahoo/elide/audit/LogMessageTest.java rename to elide-core/src/test/java/com/yahoo/elide/audit/LogMessageImplTest.java index db3a4b3c34..f1c18f53fe 100644 --- a/elide-core/src/test/java/com/yahoo/elide/audit/LogMessageTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/audit/LogMessageImplTest.java @@ -33,7 +33,7 @@ import java.util.concurrent.Executors; import java.util.concurrent.ThreadLocalRandom; -public class LogMessageTest { +public class LogMessageImplTest { private static transient PersistentResource childRecord; private static transient PersistentResource friendRecord; @@ -73,7 +73,7 @@ public String getName() { @Test public void verifyOpaqueUserExpressions() { final String[] expressions = { "${opaqueUser.name}", "${opaqueUser.name}" }; - final LogMessage message = new LogMessage("{0} {1}", childRecord, expressions, 1, Optional.empty()); + final LogMessageImpl message = new LogMessageImpl("{0} {1}", childRecord, expressions, 1, Optional.empty()); assertEquals("aaron aaron", message.getMessage(), "JEXL substitution evaluates correctly."); assertEquals(Optional.empty(), message.getChangeSpec()); } @@ -81,7 +81,7 @@ public void verifyOpaqueUserExpressions() { @Test public void verifyObjectExpressions() { final String[] expressions = { "${child.id}", "${parent.getId()}" }; - final LogMessage message = new LogMessage("{0} {1}", childRecord, expressions, 1, Optional.empty()); + final LogMessageImpl message = new LogMessageImpl("{0} {1}", childRecord, expressions, 1, Optional.empty()); assertEquals("5 7", message.getMessage(), "JEXL substitution evaluates correctly."); assertEquals(Optional.empty(), message.getChangeSpec()); } @@ -90,8 +90,8 @@ public void verifyObjectExpressions() { public void verifyListExpressions() { final String[] expressions = { "${child[0].id}", "${child[1].id}", "${parent.getId()}" }; final String[] expressionForDefault = { "${child.id}" }; - final LogMessage message = new LogMessage("{0} {1} {2}", friendRecord, expressions, 1, Optional.empty()); - final LogMessage defaultMessage = new LogMessage("{0}", friendRecord, expressionForDefault, 1, Optional.empty()); + final LogMessageImpl message = new LogMessageImpl("{0} {1} {2}", friendRecord, expressions, 1, Optional.empty()); + final LogMessageImpl defaultMessage = new LogMessageImpl("{0}", friendRecord, expressionForDefault, 1, Optional.empty()); assertEquals("5 9 7", message.getMessage(), "JEXL substitution evaluates correctly."); assertEquals("9", defaultMessage.getMessage(), "JEXL substitution evaluates correctly."); assertEquals(Optional.empty(), message.getChangeSpec()); @@ -103,7 +103,7 @@ public void invalidExpression() { final String[] expressions = { "${child.id}, ${%%%}" }; assertThrows( InvalidSyntaxException.class, - () -> new LogMessage("{0} {1}", childRecord, expressions, 1, Optional.empty()).getMessage()); + () -> new LogMessageImpl("{0} {1}", childRecord, expressions, 1, Optional.empty()).getMessage()); } @Test @@ -111,7 +111,7 @@ public void invalidTemplate() { final String[] expressions = { "${child.id}" }; assertThrows( InvalidSyntaxException.class, - () -> new LogMessage("{}", childRecord, expressions, 1, Optional.empty()).getMessage()); + () -> new LogMessageImpl("{}", childRecord, expressions, 1, Optional.empty()).getMessage()); } public static class TestLoggerException extends RuntimeException { @@ -141,7 +141,7 @@ public void threadSafetyTest() { public void threadSafeLogger() throws IOException, InterruptedException { TestLoggerException testException = new TestLoggerException(); - LogMessage failMessage = new LogMessage("test", 0) { + LogMessageImpl failMessage = new LogMessageImpl("test", 0) { @Override public String getMessage() { throw testException; @@ -150,7 +150,7 @@ public String getMessage() { try { testAuditLogger.log(failMessage); Thread.sleep(Math.floorMod(ThreadLocalRandom.current().nextInt(), 100)); - testAuditLogger.commit(null); + testAuditLogger.commit(); fail("Exception expected"); } catch (TestLoggerException e) { assertSame(e, testException); @@ -158,7 +158,7 @@ public String getMessage() { // should not cause another exception try { - testAuditLogger.commit(null); + testAuditLogger.commit(); } catch (TestLoggerException e) { fail("Exception not cleared from previous logger commit"); } diff --git a/elide-core/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java b/elide-core/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java index 167433aa8d..0f007f8434 100644 --- a/elide-core/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java +++ b/elide-core/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java @@ -5,15 +5,14 @@ */ package com.yahoo.elide.audit; -import com.yahoo.elide.core.RequestScope; - import java.io.IOException; import java.util.ArrayList; import java.util.List; public class TestAuditLogger extends AuditLogger { @Override - public void commit(RequestScope requestScope) throws IOException { + public void commit() throws IOException { + //NOOP } public List getMessages() { diff --git a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java index 918a3ecf61..f7da8509ac 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java @@ -32,7 +32,6 @@ import com.yahoo.elide.security.checks.UserCheck; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; -import com.yahoo.elide.security.checks.prefab.Common.UpdateOnCreate; import com.yahoo.elide.security.checks.prefab.Role; import com.google.common.collect.ImmutableList; @@ -106,7 +105,6 @@ public void testFindCheckByExpression() { assertEquals("Prefab.Role.None", getCheckIdentifier(Role.NONE.class)); assertEquals("Prefab.Collections.AppendOnly", getCheckIdentifier(AppendOnly.class)); assertEquals("Prefab.Collections.RemoveOnly", getCheckIdentifier(RemoveOnly.class)); - assertEquals("Prefab.Common.UpdateOnCreate", getCheckIdentifier(UpdateOnCreate.class)); } @Test diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PaginationLogicTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PaginationImplTest.java similarity index 52% rename from elide-core/src/test/java/com/yahoo/elide/core/PaginationLogicTest.java rename to elide-core/src/test/java/com/yahoo/elide/core/PaginationImplTest.java index ee5790c0f4..b3f6fdfb69 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PaginationLogicTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PaginationImplTest.java @@ -14,18 +14,17 @@ import com.yahoo.elide.ElideSettingsBuilder; import com.yahoo.elide.annotation.Paginate; import com.yahoo.elide.core.exceptions.InvalidValueException; -import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.pagination.PaginationImpl; import org.glassfish.jersey.internal.util.collection.MultivaluedStringMap; import org.junit.jupiter.api.Test; import java.util.Optional; - import javax.ws.rs.core.MultivaluedMap; /** * Tests parsing the page params for json-api pagination. */ -public class PaginationLogicTest { +public class PaginationImplTest { private final ElideSettings elideSettings = new ElideSettingsBuilder(null).build(); @@ -35,8 +34,8 @@ public void shouldParseQueryParamsForCurrentPageAndPageSize() { queryParams.add("page[size]", "10"); queryParams.add("page[number]", "2"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - pageData = pageData.evaluate(PaginationLogicTest.class); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); // page based strategy uses human readable paging - non-zero index // page 2 becomes (1)*10 so 10 since we shift to zero based index assertEquals(10, pageData.getOffset()); @@ -49,8 +48,8 @@ public void shouldThrowExceptionForNegativePageNumber() { queryParams.add("page[size]", "10"); queryParams.add("page[number]", "-2"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertThrows(InvalidValueException.class, () -> pageData.evaluate(PaginationLogicTest.class)); + assertThrows(InvalidValueException.class, () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test @@ -58,8 +57,9 @@ public void shouldThrowExceptionForNegativePageSize() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[size]", "-10"); queryParams.add("page[number]", "2"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertThrows(InvalidValueException.class, () -> pageData.evaluate(PaginationLogicTest.class)); + + assertThrows(InvalidValueException.class, () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test @@ -68,8 +68,8 @@ public void shouldParseQueryParamsForOffsetAndLimit() { queryParams.add("page[limit]", "10"); queryParams.add("page[offset]", "2"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - pageData = pageData.evaluate(PaginationLogicTest.class); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); // offset is direct correlation to start field in query assertEquals(2, pageData.getOffset()); assertEquals(10, pageData.getLimit()); @@ -78,76 +78,72 @@ public void shouldParseQueryParamsForOffsetAndLimit() { @Test public void shouldUseDefaultsWhenMissingCurrentPageAndPageSize() { MultivaluedMap queryParams = new MultivaluedStringMap(); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertEquals(Pagination.DEFAULT_OFFSET, pageData.getOffset()); - assertEquals(Pagination.DEFAULT_PAGE_LIMIT, pageData.getLimit()); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); + assertEquals(PaginationImpl.DEFAULT_OFFSET, pageData.getOffset()); + assertEquals(PaginationImpl.DEFAULT_PAGE_LIMIT, pageData.getLimit()); } @Test public void checkValidOffsetAndFirstRequest() { - Pagination pageData = Pagination.fromOffsetAndFirst(Optional.of("10"), Optional.of("1"), true, elideSettings).get(); - - // NOTE: This is always set to default until evaluate. Then the appropriate value should be used. - // This is because the particular root entity determines the pagination limits - assertEquals(0, pageData.getOffset()); - assertEquals(500, pageData.getLimit()); - - assertEquals(1, pageData.evaluate(PaginationLogicTest.class).getOffset()); - assertEquals(10, pageData.evaluate(PaginationLogicTest.class).getLimit()); - assertTrue(pageData.evaluate(PaginationLogicTest.class).isGenerateTotals()); + PaginationImpl pageData = new PaginationImpl(PaginationImplTest.class, + 1, + 10, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + false, + false); + + assertEquals(1, pageData.getOffset()); + assertEquals(10, pageData.getLimit()); } @Test public void checkErroneousPageLimit() { - Pagination pageData = - Pagination.fromOffsetAndFirst(Optional.of("100000"), Optional.of("1"), false, elideSettings).get(); - - // NOTE: This is always set to default until evaluate. Then the appropriate value should be used. - // This is because the particular root entity determines the pagination limits - assertEquals(0, pageData.getOffset()); - assertEquals(500, pageData.getLimit()); - assertThrows( - InvalidValueException.class, - () -> pageData.evaluate(PaginationLogicTest.class).getOffset()); assertThrows( InvalidValueException.class, - () -> pageData.evaluate(PaginationLogicTest.class).getLimit()); + () -> new PaginationImpl(PaginationImplTest.class, + 10, + 100000, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + false, + false)); } @Test public void checkBadOffset() { assertThrows( InvalidValueException.class, - () -> Pagination.fromOffsetAndFirst(Optional.of("-1"), Optional.of("1000"), false, elideSettings)); - } - - @Test - public void checkBadOffsetString() { - assertThrows( - InvalidValueException.class, - () -> Pagination.fromOffsetAndFirst(Optional.of("NaN"), Optional.of("1000"), false, elideSettings)); + () -> new PaginationImpl(PaginationImplTest.class, + -1, + 1000, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + false, + false)); } @Test public void checkBadLimit() { assertThrows( InvalidValueException.class, - () -> Pagination.fromOffsetAndFirst(Optional.of("0"), Optional.of("1"), false, elideSettings)); - } - - @Test - public void checkBadLimitString() { - assertThrows( - InvalidValueException.class, - () -> Pagination.fromOffsetAndFirst(Optional.of("1"), Optional.of("NaN"), false, elideSettings)); + () -> new PaginationImpl(PaginationImplTest.class, + 0, + -1, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), + false, + false)); } @Test public void neverExceedMaxPageSize() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[size]", "25000"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertThrows(InvalidValueException.class, () -> pageData.evaluate(PaginationLogicTest.class)); + assertThrows(InvalidValueException.class, + () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test @@ -155,48 +151,54 @@ public void invalidUsageOfPaginationParameters() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[size]", "10"); queryParams.add("page[offset]", "100"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertThrows(InvalidValueException.class, () -> pageData.evaluate(PaginationLogicTest.class)); + assertThrows(InvalidValueException.class, + () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test public void pageBasedPaginationWithDefaultSize() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[number]", "2"); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - pageData = pageData.evaluate(PaginationLogicTest.class); - assertEquals(Pagination.DEFAULT_PAGE_LIMIT, pageData.getLimit()); - assertEquals(Pagination.DEFAULT_PAGE_LIMIT, pageData.getOffset()); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImpl.class, + Optional.of(queryParams), elideSettings); + assertEquals(PaginationImpl.DEFAULT_PAGE_LIMIT, pageData.getLimit()); + assertEquals(PaginationImpl.DEFAULT_PAGE_LIMIT, pageData.getOffset()); } @Test public void shouldThrowExceptionForNonIntPageParamValues() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[size]", "2.5"); - assertThrows(InvalidValueException.class, () -> Pagination.parseQueryParams(queryParams, elideSettings)); + assertThrows(InvalidValueException.class, + () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test public void shouldThrowExceptionForInvalidPageParams() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[random]", "1"); - assertThrows(InvalidValueException.class, () -> Pagination.parseQueryParams(queryParams, elideSettings)); + assertThrows(InvalidValueException.class, + () -> PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings)); } @Test public void shouldSetGenerateTotals() { MultivaluedMap queryParams = new MultivaluedStringMap(); queryParams.add("page[totals]", null); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - pageData = pageData.evaluate(PaginationLogicTest.class); - assertTrue(pageData.isGenerateTotals()); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); + assertTrue(pageData.returnPageTotals()); } @Test public void shouldNotSetGenerateTotals() { MultivaluedMap queryParams = new MultivaluedStringMap(); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); - assertFalse(pageData.isGenerateTotals()); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); + assertFalse(pageData.returnPageTotals()); } @@ -204,12 +206,13 @@ public void shouldNotSetGenerateTotals() { public void shouldUseDefaultsWhenNoParams() { MultivaluedMap queryParams = new MultivaluedStringMap(); - Pagination pageData = Pagination.parseQueryParams(queryParams, elideSettings); + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), elideSettings); assertEquals(0, pageData.getOffset()); - assertEquals(Pagination.DEFAULT_PAGE_LIMIT, pageData.getLimit()); + assertEquals(PaginationImpl.DEFAULT_PAGE_LIMIT, pageData.getLimit()); - pageData = Pagination.parseQueryParams(queryParams, - new ElideSettingsBuilder(null) + pageData = PaginationImpl.parseQueryParams(PaginationImplTest.class, + Optional.of(queryParams), new ElideSettingsBuilder(null) .withDefaultPageSize(10) .withDefaultMaxPageSize(10) .build()); @@ -223,16 +226,14 @@ public void testClassLevelOverride() { class PaginationOverrideTest { } MultivaluedMap queryParams = new MultivaluedStringMap(); - Pagination pageData = Pagination.parseQueryParams(queryParams, + PaginationImpl pageData = PaginationImpl.parseQueryParams(PaginationOverrideTest.class, + Optional.of(queryParams), new ElideSettingsBuilder(null) - .withDefaultPageSize(0) - .withDefaultMaxPageSize(0) + .withDefaultPageSize(1) + .withDefaultMaxPageSize(1) .build()); - assertEquals(0, pageData.getOffset()); - assertEquals(0, pageData.getLimit()); - Pagination result = pageData.evaluate(PaginationOverrideTest.class); assertEquals(0, pageData.getOffset()); - assertEquals(10, result.getLimit()); + assertEquals(10, pageData.getLimit()); } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java index ba2b8b4b61..c45c4daa57 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransactionTest.java @@ -26,10 +26,11 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.google.common.collect.Lists; import com.google.common.collect.Sets; @@ -259,7 +260,7 @@ public void testSortingPushDown() { Map sortOrder = new HashMap<>(); sortOrder.put("title", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortOrder); + Sorting sorting = new SortingImpl(sortOrder, Book.class, dictionary); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -293,7 +294,7 @@ public void testDataStoreRequiresInMemorySorting() { Map sortOrder = new HashMap<>(); sortOrder.put("title", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortOrder); + Sorting sorting = new SortingImpl(sortOrder, Book.class, dictionary); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -333,7 +334,7 @@ public void testFilteringRequiresInMemorySorting() { Map sortOrder = new HashMap<>(); sortOrder.put("title", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortOrder); + Sorting sorting = new SortingImpl(sortOrder, Book.class, dictionary); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -368,7 +369,7 @@ public void testFilteringRequiresInMemorySorting() { @Test public void testPaginationPushDown() { - Pagination pagination = Pagination.getDefaultPagination(elideSettings); + PaginationImpl pagination = PaginationImpl.getDefaultPagination(Book.class, elideSettings); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -379,7 +380,7 @@ public void testPaginationPushDown() { when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); + when(wrappedTransaction.supportsPagination(eq(Book.class), any())).thenReturn(true); when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); @@ -399,7 +400,7 @@ public void testPaginationPushDown() { @Test public void testDataStoreRequiresInMemoryPagination() { - Pagination pagination = Pagination.getDefaultPagination(elideSettings); + PaginationImpl pagination = PaginationImpl.getDefaultPagination(Book.class, elideSettings); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -410,7 +411,7 @@ public void testDataStoreRequiresInMemoryPagination() { when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); - when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(false); + when(wrappedTransaction.supportsPagination(eq(Book.class), any())).thenReturn(false); when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); @@ -436,7 +437,7 @@ public void testFilteringRequiresInMemoryPagination() { FilterExpression expression = new InPredicate(new Path(Book.class, dictionary, "genre"), "Literary Fiction"); - Pagination pagination = Pagination.getDefaultPagination(elideSettings); + PaginationImpl pagination = PaginationImpl.getDefaultPagination(Book.class, elideSettings); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -448,7 +449,7 @@ public void testFilteringRequiresInMemoryPagination() { when(wrappedTransaction.supportsFiltering(eq(Book.class), any())).thenReturn(DataStoreTransaction.FeatureSupport.NONE); - when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); + when(wrappedTransaction.supportsPagination(eq(Book.class), any())).thenReturn(true); when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); @@ -470,12 +471,12 @@ public void testFilteringRequiresInMemoryPagination() { @Test public void testSortingRequiresInMemoryPagination() { - Pagination pagination = Pagination.getDefaultPagination(elideSettings); + PaginationImpl pagination = PaginationImpl.getDefaultPagination(Book.class, elideSettings); Map sortOrder = new HashMap<>(); sortOrder.put("title", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortOrder); + Sorting sorting = new SortingImpl(sortOrder, Book.class, dictionary); EntityProjection projection = EntityProjection.builder() .type(Book.class) @@ -489,7 +490,7 @@ public void testSortingRequiresInMemoryPagination() { any())).thenReturn(DataStoreTransaction.FeatureSupport.FULL); when(wrappedTransaction.supportsSorting(eq(Book.class), any())).thenReturn(false); - when(wrappedTransaction.supportsPagination(eq(Book.class))).thenReturn(true); + when(wrappedTransaction.supportsPagination(eq(Book.class), any())).thenReturn(true); when(wrappedTransaction.loadObjects(any(), eq(scope))).thenReturn(books); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java index 0141bffefb..292bb35c47 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapperTest.java @@ -158,10 +158,10 @@ public void testSupportsPagination() { DataStoreTransaction wrapped = mock(DataStoreTransaction.class); DataStoreTransaction wrapper = new TestTransactionWrapper(wrapped); - when(wrapped.supportsPagination(any())).thenReturn(true); - boolean actual = wrapper.supportsPagination(null); + when(wrapped.supportsPagination(any(), any())).thenReturn(true); + boolean actual = wrapper.supportsPagination(null, null); - verify(wrapped, times(1)).supportsPagination(any()); + verify(wrapped, times(1)).supportsPagination(any(), any()); assertTrue(actual); } diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java index e970bb0233..91a394852e 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java @@ -14,10 +14,11 @@ import com.yahoo.elide.core.TestRequestScope; import com.yahoo.elide.core.filter.InInsensitivePredicate; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Sorting; import example.Address; import example.Author; import example.Book; @@ -52,7 +53,6 @@ public void testRootCollectionNoQueryParams() { String path = "/book"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -71,7 +71,7 @@ public void testRootCollectionNoQueryParams() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build(); EntityProjection actual = maker.parsePath(path); @@ -86,7 +86,6 @@ public void testRootCollectionSparseFields() { String path = "/book"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -97,7 +96,7 @@ public void testRootCollectionSparseFields() { .relationship("authors", EntityProjection.builder() .type(Author.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build(); EntityProjection actual = maker.parsePath(path); @@ -111,7 +110,6 @@ public void testRootEntityNoQueryParams() { String path = "/book/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -143,7 +141,6 @@ public void testNestedCollectionNoQueryParams() { String path = "/author/1/books/3/publisher"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -161,7 +158,7 @@ public void testNestedCollectionNoQueryParams() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Publisher.class)) .build()) .build()) .build(); @@ -177,7 +174,6 @@ public void testNestedEntityNoQueryParams() { String path = "/author/1/books/3/publisher/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -210,7 +206,6 @@ public void testRelationshipNoQueryParams() { String path = "/author/1/relationships/books"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -218,7 +213,7 @@ public void testRelationshipNoQueryParams() { .type(Author.class) .relationship("books", EntityProjection.builder() .type(Book.class) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build()) .build(); @@ -234,7 +229,6 @@ public void testRelationshipWithSingleInclude() { String path = "/book/1/relationships/publisher"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -242,7 +236,7 @@ public void testRelationshipWithSingleInclude() { .type(Book.class) .relationship("publisher", EntityProjection.builder() .type(Publisher.class) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Publisher.class)) .build()) .relationship("authors", EntityProjection.builder() .attribute(Attribute.builder().name("name").type(String.class).build()) @@ -270,7 +264,6 @@ public void testRootCollectionWithSingleInclude() { String path = "/book"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -295,7 +288,7 @@ public void testRootCollectionWithSingleInclude() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build(); EntityProjection actual = maker.parsePath(path); @@ -310,7 +303,6 @@ public void testRootEntityWithSingleInclude() { String path = "/book/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -350,7 +342,6 @@ public void testRootCollectionWithNestedInclude() throws Exception { String path = "/author"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -389,7 +380,7 @@ public void testRootCollectionWithNestedInclude() throws Exception { .type(Author.class) .build()) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Author.class)) .build(); EntityProjection actual = maker.parsePath(path); @@ -405,7 +396,6 @@ public void testRootEntityWithNestedInclude() { String path = "/author/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -458,7 +448,6 @@ public void testNestedEntityWithSingleInclude() { String path = "/author/1/books/3/publisher/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -505,7 +494,6 @@ public void testNestedCollectionWithSingleInclude() { String path = "/author/1/books/3/publisher"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -536,7 +524,7 @@ public void testNestedCollectionWithSingleInclude() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Publisher.class)) .build()) .build()) .build(); @@ -557,7 +545,6 @@ public void testRootEntityWithNestedIncludeAndSparseFields() { String path = "/author/1"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -592,7 +579,6 @@ public void testRootCollectionWithGlobalFilter() { String path = "/book"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); FilterExpression expression = new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); @@ -601,21 +587,21 @@ public void testRootCollectionWithGlobalFilter() { EntityProjection expected = EntityProjection.builder() .type(Book.class) - .attribute(Attribute.builder().name("title").type(String.class).build()) - .attribute(Attribute.builder().name("genre").type(String.class).build()) .attribute(Attribute.builder().name("language").type(String.class).build()) + .attribute(Attribute.builder().name("genre").type(String.class).build()) + .attribute(Attribute.builder().name("title").type(String.class).build()) .attribute(Attribute.builder().name("publishDate").type(long.class).build()) .filterExpression(expression) - .relationship("authors", EntityProjection.builder() - .type(Author.class) - .build()) .relationship("publisher", EntityProjection.builder() .type(Publisher.class) .build()) .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .relationship("authors", EntityProjection.builder() + .type(Author.class) + .build()) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build(); EntityProjection actual = maker.parsePath(path); @@ -633,7 +619,6 @@ public void testNestedCollectionWithTypedFilter() { new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo"); RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -652,7 +637,7 @@ public void testNestedCollectionWithTypedFilter() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Publisher.class)) .build()) .build()) .build(); @@ -670,10 +655,9 @@ public void testRelationshipsAndIncludeWithFilterAndSort() { queryParams.add("filter[publisher]", "name=='Foo'"); queryParams.add("sort", "name"); String path = "/book/1/relationships/publisher"; - Sorting sorting = Sorting.parseSortRule("name"); + Sorting sorting = SortingImpl.parseSortRule("name", Publisher.class, dictionary); RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); EntityProjectionMaker maker = new EntityProjectionMaker(dictionary, scope); @@ -683,7 +667,7 @@ public void testRelationshipsAndIncludeWithFilterAndSort() { .type(Publisher.class) .filterExpression(new InInsensitivePredicate(new Path(Publisher.class, dictionary, "name"), "Foo")) .sorting(sorting) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Publisher.class)) .build()) .relationship("authors", EntityProjection.builder() .attribute(Attribute.builder().name("name").type(String.class).build()) @@ -712,7 +696,6 @@ public void testRootCollectionWithTypedFilter() { String path = "/book"; RequestScope scope = new TestRequestScope(dictionary, path, queryParams); - Pagination defaultPagination = scope.getPagination(); FilterExpression expression = new InInsensitivePredicate(new Path(Book.class, dictionary, "genre"), "Science Fiction"); @@ -735,7 +718,7 @@ public void testRootCollectionWithTypedFilter() { .relationship("editor", EntityProjection.builder() .type(Editor.class) .build()) - .pagination(defaultPagination) + .pagination(PaginationImpl.getDefaultPagination(Book.class)) .build(); EntityProjection actual = maker.parsePath(path); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java index a6ad1583fe..9a3dc6b960 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java @@ -13,11 +13,11 @@ import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.NotFilterExpression; import com.yahoo.elide.core.filter.expression.OrFilterExpression; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.request.Sorting; import java.util.List; import java.util.Map; @@ -32,7 +32,6 @@ public class QueryValidator { private Set allFields; private EntityDictionary dictionary; private AnalyticView queriedTable; - private Class queriedClass; private List metrics; private Set dimensionProjections; @@ -41,7 +40,6 @@ public QueryValidator(Query query, Set allFields, EntityDictionary dicti this.allFields = allFields; this.dictionary = dictionary; this.queriedTable = query.getAnalyticView(); - this.queriedClass = queriedTable.getCls(); this.metrics = query.getMetrics(); this.dimensionProjections = query.getDimensions(); } @@ -116,7 +114,7 @@ public void validateSorting() { if (sorting == null) { return; } - Map sortClauses = sorting.getValidSortingRules(queriedClass, dictionary); + Map sortClauses = sorting.getSortingPaths(); sortClauses.keySet().forEach((path) -> validateSortingPath(path, allFields)); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java index 9d6687ca79..98e2a24565 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java @@ -7,12 +7,12 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import lombok.Builder; import lombok.Data; import lombok.Singular; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java index e35ae1184c..a90d708e5c 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -17,7 +17,6 @@ import com.yahoo.elide.core.filter.FilterTranslator; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; @@ -32,6 +31,7 @@ import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.request.Sorting; import org.hibernate.annotations.Subselect; import java.util.Collection; @@ -112,7 +112,7 @@ public SQLQuery resolveTemplate(Query clientQuery, } if (sorting != null) { - Map sortClauses = sorting.getValidSortingRules(tableCls, dictionary); + Map sortClauses = sorting.getSortingPaths(); builder.orderByClause(extractOrderBy(sortClauses, template)); joinPaths.addAll(extractJoinPaths(sortClauses)); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java index b4404131a8..ab9c9b0176 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -13,7 +13,6 @@ import com.yahoo.elide.core.exceptions.InvalidPredicateException; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; @@ -29,6 +28,7 @@ import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.utils.coerce.CoerceUtil; import org.hibernate.jpa.QueryHints; @@ -104,7 +104,7 @@ public Iterable executeQuery(Query query) { jpaQuery.setFirstResult(pagination.getOffset()); jpaQuery.setMaxResults(pagination.getLimit()); - if (pagination.isGenerateTotals()) { + if (pagination.returnPageTotals()) { SQLQuery paginationSQL = toPageTotalSQL(sql); javax.persistence.Query pageTotalQuery = diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java index e0b7f2ba45..9e63cbf3f6 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java @@ -11,13 +11,14 @@ import com.yahoo.elide.core.exceptions.InvalidOperationException; import com.yahoo.elide.core.filter.dialect.ParseException; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.request.Sorting; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -42,7 +43,7 @@ public void testNoMetricQuery() { Query query = Query.builder() .analyticView(playerStatsTable) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); QueryValidator validator = new QueryValidator(query, Collections.singleton("overallRating"), dictionary); @@ -61,7 +62,7 @@ public void testSortingOnId() { .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("id"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); Set allFields = new HashSet<>(Arrays.asList("id", "overallRating", "lowScore")); @@ -80,7 +81,7 @@ public void testSortingOnNotQueriedDimension() { .analyticView(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); @@ -99,7 +100,7 @@ public void testSortingOnNotQueriedMetric() { .analyticView(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); Set allFields = new HashSet<>(Arrays.asList("overallRating", "lowScore")); @@ -118,7 +119,7 @@ public void testSortingOnNestedDimensionField() { .analyticView(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); Set allFields = new HashSet<>(Arrays.asList("country", "lowScore")); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java index 6ba6d1f548..6043dfa550 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -12,8 +12,8 @@ import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.Operator; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; @@ -21,6 +21,7 @@ import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.request.Sorting; import com.google.common.collect.Lists; import org.junit.jupiter.api.BeforeAll; @@ -201,7 +202,7 @@ public void testSortJoin() { .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -236,7 +237,15 @@ public void testSortJoin() { */ @Test public void testPagination() { - Pagination pagination = Pagination.fromOffsetAndLimit(1, 0, true); + PaginationImpl pagination = new PaginationImpl( + PlayerStats.class, + 0, + 1, + PaginationImpl.DEFAULT_PAGE_LIMIT, + PaginationImpl.MAX_PAGE_LIMIT, + true, + false + ); Query query = Query.builder() .analyticView(playerStatsTable) @@ -343,7 +352,7 @@ public void testEdgeCasesQuery() throws Exception { PlayerStatsView.class, false)) .havingFilter(filterParser.parseFilterExpression("highScore > 300", PlayerStatsView.class, false)) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -372,7 +381,7 @@ public void testSortByMultipleColumns() { .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -416,7 +425,7 @@ public void testRelationshipHydration() { .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -525,7 +534,7 @@ public void testJoinToSort() { .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -616,7 +625,7 @@ public void testSortAggregatedMetric() { .analyticView(playerStatsTable) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -647,7 +656,7 @@ public void testAmbiguousFields() { .groupByDimension(toProjection(playerStatsTable.getDimension("playerName"))) .groupByDimension(toProjection(playerStatsTable.getDimension("player2Name"))) .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java index b0f536c9db..77adb647e8 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -9,13 +9,14 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNotNull; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.SubCountry; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.request.Sorting; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -110,7 +111,7 @@ public void testRelationshipHydration() { .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -233,7 +234,7 @@ public void testJoinToSort() throws Exception { .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java index 779029104a..fb5ca13f71 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -11,13 +11,14 @@ import static org.junit.jupiter.api.Assertions.assertThrows; import com.yahoo.elide.core.exceptions.InvalidPredicateException; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.request.Sorting; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -45,7 +46,7 @@ public void testViewRelationFailure() { .analyticView(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryView"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); assertThrows(InvalidPredicateException.class, () -> engine.executeQuery(query)); @@ -60,7 +61,7 @@ public void testViewAttribute() { .analyticView(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -94,7 +95,7 @@ public void testNestedViewAttribute() throws Exception { .analyticView(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -129,7 +130,7 @@ public void testNestedRelationshipAttribute() throws Exception { .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -164,7 +165,7 @@ public void testSortingViewAttribute() throws Exception { .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -199,7 +200,7 @@ public void testSortingNestedViewAttribute() throws Exception { .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) @@ -234,7 +235,7 @@ public void testSortingNestedRelationshipAttribute() throws Exception { .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) - .sorting(new Sorting(sortMap)) + .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java index 194e897bcb..92c7236116 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java @@ -16,9 +16,9 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.hibernate.Query; import com.yahoo.elide.core.hibernate.Session; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import org.apache.commons.lang3.StringUtils; import java.util.ArrayList; @@ -224,9 +224,7 @@ protected String extractToOneMergeJoins(Class entityClass, String alias) { protected String getSortClause(final Optional sorting, Class sortClass, boolean prefixWithAlias) { String sortingRules = ""; if (sorting.isPresent() && !sorting.get().isDefaultInstance()) { - final Map validSortingRules = sorting.get().getValidSortingRules( - sortClass, dictionary - ); + final Map validSortingRules = sorting.get().getSortingPaths(); if (!validSortingRules.isEmpty()) { final List ordering = new ArrayList<>(); // pass over the sorting rules diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java index 80be6043e3..84795e0389 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java @@ -11,8 +11,8 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.hibernate.Query; import com.yahoo.elide.core.hibernate.Session; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import java.util.Collection; import java.util.HashSet; diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java index fb999db7d2..8d96cf4b20 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java @@ -16,8 +16,8 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.hibernate.Query; import com.yahoo.elide.core.hibernate.Session; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.utils.coerce.CoerceUtil; import java.util.ArrayList; diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java index 8555f9dcbc..5ff85922d2 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java @@ -24,9 +24,11 @@ import com.yahoo.elide.core.filter.expression.OrFilterExpression; import com.yahoo.elide.core.hibernate.Query; import com.yahoo.elide.core.hibernate.hql.AbstractHQLQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; import example.Chapter; @@ -118,7 +120,7 @@ public void testSortClauseWithoutPrefix() { sorting.put(TITLE, Sorting.SortOrder.asc); sorting.put(GENRE, Sorting.SortOrder.desc); - String actual = getSortClause(Optional.of(new Sorting(sorting)), Book.class, NO_ALIAS); + String actual = getSortClause(Optional.of(new SortingImpl(sorting, Book.class, dictionary)), Book.class, NO_ALIAS); String expected = " order by title asc,genre desc"; assertEquals(expected, actual); @@ -130,7 +132,7 @@ public void testSortClauseWithPrefix() { sorting.put(TITLE, Sorting.SortOrder.asc); sorting.put(GENRE, Sorting.SortOrder.desc); - String actual = getSortClause(Optional.of(new Sorting(sorting)), Book.class, USE_ALIAS); + String actual = getSortClause(Optional.of(new SortingImpl(sorting, Book.class, dictionary)), Book.class, USE_ALIAS); String expected = " order by example_Book.title asc,example_Book.genre desc"; assertEquals(expected, actual); @@ -141,7 +143,7 @@ public void testSortClauseWithJoin() { Map sorting = new LinkedHashMap<>(); sorting.put(PUBLISHER + PERIOD + NAME, Sorting.SortOrder.asc); - String actual = getSortClause(Optional.of(new Sorting(sorting)), Book.class, NO_ALIAS); + String actual = getSortClause(Optional.of(new SortingImpl(sorting, Book.class, dictionary)), Book.class, NO_ALIAS); String expected = " order by publisher.name asc"; assertEquals(expected, actual); @@ -152,7 +154,7 @@ public void testSortClauseWithInvalidJoin() { Map sorting = new LinkedHashMap<>(); sorting.put(AUTHORS + PERIOD + NAME, Sorting.SortOrder.asc); - assertThrows(InvalidValueException.class, () -> getSortClause(Optional.of(new Sorting(sorting)), Book.class, NO_ALIAS)); + assertThrows(InvalidValueException.class, () -> getSortClause(Optional.of(new SortingImpl(sorting, Book.class, dictionary)), Book.class, NO_ALIAS)); } @Test @@ -178,7 +180,7 @@ public void testSettingQueryPagination() { Optional previousPagination = pagination; - Pagination paginationMock = mock(Pagination.class); + PaginationImpl paginationMock = mock(PaginationImpl.class); when(paginationMock.getLimit()).thenReturn(10); when(paginationMock.getOffset()).thenReturn(50); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java index e6d37f41f9..4bb4b6caa5 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionFetchQueryBuilderTest.java @@ -13,8 +13,9 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.OrFilterExpression; import com.yahoo.elide.core.hibernate.hql.RootCollectionFetchQueryBuilder; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.sort.SortingImpl; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; import example.Chapter; @@ -70,7 +71,7 @@ public void testRootFetchWithSorting() { sorting.put(TITLE, Sorting.SortOrder.asc); TestQueryWrapper query = (TestQueryWrapper) builder - .withPossibleSorting(Optional.of(new Sorting(sorting))) + .withPossibleSorting(Optional.of(new SortingImpl(sorting, Book.class, dictionary))) .build(); String expected = "SELECT example_Book FROM example.Book AS example_Book order by example_Book.title asc"; @@ -143,7 +144,7 @@ public void testRootFetchWithSortingAndFilters() { TestQueryWrapper query = (TestQueryWrapper) builder - .withPossibleSorting(Optional.of(new Sorting(sorting))) + .withPossibleSorting(Optional.of(new SortingImpl(sorting, Book.class, dictionary))) .withPossibleFilterExpression(Optional.of(idPredicate)) .build(); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java index 3b15499dad..22a2dd0d40 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/RootCollectionPageTotalsQueryBuilderTest.java @@ -15,8 +15,8 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.filter.expression.OrFilterExpression; import com.yahoo.elide.core.hibernate.hql.RootCollectionPageTotalsQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; @@ -78,7 +78,7 @@ public void testRootFetchWithSorting() { @Test public void testRootFetchWithPagination() { - Pagination pagination = mock(Pagination.class); + PaginationImpl pagination = mock(PaginationImpl.class); RootCollectionPageTotalsQueryBuilder builder = new RootCollectionPageTotalsQueryBuilder( Book.class, dictionary, new TestSessionWrapper()); diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java index 5717fcb641..72413e8adc 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionFetchQueryBuilderTest.java @@ -14,8 +14,9 @@ import com.yahoo.elide.core.filter.InPredicate; import com.yahoo.elide.core.hibernate.hql.RelationshipImpl; import com.yahoo.elide.core.hibernate.hql.SubCollectionFetchQueryBuilder; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.sort.SortingImpl; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; import example.Chapter; @@ -96,7 +97,7 @@ public void testSubCollectionFetchWithSorting() { sorting.put(TITLE, Sorting.SortOrder.asc); TestQueryWrapper query = (TestQueryWrapper) builder - .withPossibleSorting(Optional.of(new Sorting(sorting))) + .withPossibleSorting(Optional.of(new SortingImpl(sorting, Book.class, dictionary))) .build(); String expected = "SELECT example_Book FROM example.Author example_Author__fetch " @@ -183,7 +184,7 @@ public void testSubCollectionFetchWithSortingAndFilters() { TestQueryWrapper query = (TestQueryWrapper) builder .withPossibleFilterExpression(Optional.of(publisherNamePredicate)) - .withPossibleSorting(Optional.of(new Sorting(sorting))) + .withPossibleSorting(Optional.of(new SortingImpl(sorting, Book.class, dictionary))) .build(); String expected = "SELECT example_Book FROM example.Author example_Author__fetch " diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java index c2e78c9198..a1440e7c37 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/SubCollectionPageTotalsQueryBuilderTest.java @@ -16,9 +16,9 @@ import com.yahoo.elide.core.hibernate.hql.AbstractHQLQueryBuilder; import com.yahoo.elide.core.hibernate.hql.RelationshipImpl; import com.yahoo.elide.core.hibernate.hql.SubCollectionPageTotalsQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; import example.Chapter; @@ -99,7 +99,7 @@ public void testSubCollectionPageTotalsWithSorting() { @Test public void testSubCollectionPageTotalsWithPagination() { AbstractHQLQueryBuilder.Relationship relationship = mock(AbstractHQLQueryBuilder.Relationship.class); - Pagination pagination = mock(Pagination.class); + PaginationImpl pagination = mock(PaginationImpl.class); SubCollectionPageTotalsQueryBuilder builder = new SubCollectionPageTotalsQueryBuilder(relationship, dictionary, new TestSessionWrapper()); diff --git a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java index b7acfb4b6a..5b74927d74 100644 --- a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java @@ -21,12 +21,12 @@ import com.yahoo.elide.core.hibernate.hql.RootCollectionPageTotalsQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionFetchQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionPageTotalsQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate3.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate3.porting.SessionWrapper; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -167,7 +167,7 @@ public Iterable loadObjects( FilterExpression filterExpression = projection.getFilterExpression(); Sorting sorting = projection.getSorting(); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(entityClass, Optional.ofNullable(filterExpression), scope.getDictionary())); } @@ -220,7 +220,7 @@ public Object getRelation( entity, filteredVal); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(relationship, Optional.ofNullable(filterExpression), scope.getDictionary())); } diff --git a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java index f0e08aeac5..5c39cb8383 100644 --- a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java @@ -21,12 +21,12 @@ import com.yahoo.elide.core.hibernate.hql.RootCollectionPageTotalsQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionFetchQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionPageTotalsQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.hibernate5.porting.QueryWrapper; import com.yahoo.elide.datastores.hibernate5.porting.SessionWrapper; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import org.hibernate.FlushMode; @@ -172,7 +172,7 @@ public Iterable loadObjects( FilterExpression filterExpression = projection.getFilterExpression(); Sorting sorting = projection.getSorting(); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(entityClass, Optional.ofNullable(filterExpression), scope.getDictionary())); } @@ -226,7 +226,7 @@ public Object getRelation( entity, filteredVal); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(relationship, Optional.ofNullable(filterExpression), scope.getDictionary())); } diff --git a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java index 36cf668f14..9481301637 100644 --- a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java +++ b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java @@ -20,13 +20,13 @@ import com.yahoo.elide.core.hibernate.hql.RootCollectionPageTotalsQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionFetchQueryBuilder; import com.yahoo.elide.core.hibernate.hql.SubCollectionPageTotalsQueryBuilder; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.jpa.porting.EntityManagerWrapper; import com.yahoo.elide.datastores.jpa.porting.QueryWrapper; import com.yahoo.elide.datastores.jpa.transaction.checker.PersistentCollectionChecker; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import lombok.extern.slf4j.Slf4j; @@ -189,7 +189,7 @@ public Iterable loadObjects( FilterExpression filterExpression = projection.getFilterExpression(); Sorting sorting = projection.getSorting(); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(entityClass, Optional.ofNullable(filterExpression), scope.getDictionary())); } @@ -239,7 +239,7 @@ public Object getRelation( entity, filteredVal); - if (pagination != null && pagination.isGenerateTotals()) { + if (pagination != null && pagination.returnPageTotals()) { pagination.setPageTotals(getTotalRecords(relationship, Optional.ofNullable(filterExpression), scope.getDictionary())); } diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/BridgeableTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/BridgeableTransaction.java index a56796a4e6..804d87d6f2 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/BridgeableTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/BridgeableTransaction.java @@ -7,8 +7,8 @@ import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import java.io.Serializable; import java.util.Optional; diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java index b6685a055b..9734bd08a8 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java @@ -15,11 +15,11 @@ import com.yahoo.elide.core.filter.Operator; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import java.io.IOException; @@ -230,8 +230,8 @@ public boolean supportsSorting(Class entityClass, Sorting sorting) { } @Override - public boolean supportsPagination(Class entityClass) { - return getTransaction(entityClass).supportsPagination(entityClass); + public boolean supportsPagination(Class entityClass, FilterExpression expression) { + return getTransaction(entityClass).supportsPagination(entityClass, expression); } private Serializable extractId(FilterExpression filterExpression, diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java index 5cb16dc6e7..44760d87dc 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java @@ -18,15 +18,15 @@ import com.yahoo.elide.core.filter.expression.FilterExpressionVisitor; import com.yahoo.elide.core.filter.expression.NotFilterExpression; import com.yahoo.elide.core.filter.expression.OrFilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.datastores.multiplex.BridgeableStoreTest; import com.yahoo.elide.datastores.multiplex.BridgeableTransaction; import com.yahoo.elide.datastores.multiplex.MultiplexTransaction; import com.yahoo.elide.example.beans.HibernateUser; import com.yahoo.elide.example.hbase.beans.RedisActions; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.security.User; import lombok.AllArgsConstructor; @@ -162,7 +162,12 @@ public Object bridgeableLoadObject(MultiplexTransaction muxTx, Object parent, St } @Override - public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, Object parent, String relationName, Optional filterExpressionOptional, Optional sorting, Optional pagination, RequestScope scope) { + public Iterable bridgeableLoadObjects(MultiplexTransaction muxTx, + Object parent, + String relationName, + Optional filterExpressionOptional, + Optional sorting, + Optional pagination, RequestScope scope) { if (parent.getClass().equals(HibernateUser.class) && "redisActions".equals(relationName)) { EntityDictionary dictionary = scope.getDictionary(); Class entityClass = dictionary.getParameterizedType(parent, relationName); diff --git a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java index 5a0dcfb7f0..02929bd216 100644 --- a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java +++ b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java @@ -21,10 +21,10 @@ import com.yahoo.elide.core.filter.Operator; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Pagination; +import com.yahoo.elide.request.Sorting; import org.apache.lucene.search.Query; import org.apache.lucene.search.Sort; import org.hibernate.search.annotations.Field; @@ -78,7 +78,7 @@ public Iterable loadObjects(EntityProjection projection, boolean canSearch = (canSearch(projection.getType(), projection.getFilterExpression()) != NONE); - if (mustSort(Optional.ofNullable(projection.getSorting()), projection.getType())) { + if (mustSort(Optional.ofNullable(projection.getSorting()))) { canSearch = canSearch && canSort(projection.getSorting(), projection.getType()); } @@ -94,11 +94,10 @@ public Iterable loadObjects(EntityProjection projection, /** * Indicates whether sorting has been requested for this entity. * @param sorting An optional elide sorting clause. - * @param entityClass The entity to sort. * @return True if the entity must be sorted. False otherwise. */ - private boolean mustSort(Optional sorting, Class entityClass) { - return sorting.isPresent() && !sorting.get().getValidSortingRules(entityClass, dictionary).isEmpty(); + private boolean mustSort(Optional sorting) { + return sorting.isPresent() && !sorting.get().getSortingPaths().isEmpty(); } /** @@ -110,7 +109,7 @@ private boolean mustSort(Optional sorting, Class entityClass) { private boolean canSort(Sorting sorting, Class entityClass) { for (Map.Entry entry - : sorting.getValidSortingRules(entityClass, dictionary).entrySet()) { + : sorting.getSortingPaths().entrySet()) { Path path = entry.getKey(); @@ -140,7 +139,7 @@ private Sort buildSort(Sorting sorting, Class entityClass) { SortFieldContext context = null; for (Map.Entry entry - : sorting.getValidSortingRules(entityClass, dictionary).entrySet()) { + : sorting.getSortingPaths().entrySet()) { String fieldName = entry.getKey().lastElement().get().getFieldName(); @@ -246,7 +245,7 @@ private List search(Class entityClass, FilterExpression filterExpress FullTextQuery fullTextQuery = em.createFullTextQuery(query, entityClass); - if (mustSort(sorting, entityClass)) { + if (mustSort(sorting)) { fullTextQuery = fullTextQuery.setSort(buildSort(sorting.get(), entityClass)); } @@ -259,8 +258,8 @@ private List search(Class entityClass, FilterExpression filterExpress .setProjection(ProjectionConstants.THIS) .getResultList(); - if (pagination.isPresent() && pagination.get().isGenerateTotals()) { - pagination.get().setPageTotals(fullTextQuery.getResultSize()); + if (pagination.isPresent() && pagination.get().returnPageTotals()) { + pagination.get().setPageTotals((long) fullTextQuery.getResultSize()); } if (results.isEmpty()) { diff --git a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java index 0d22fb5bf9..65db9b43ae 100644 --- a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java +++ b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DataStoreLoadTest.java @@ -22,10 +22,11 @@ import com.yahoo.elide.core.datastore.inmemory.InMemoryStoreTransaction; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.search.models.Item; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Sorting; import com.yahoo.elide.utils.coerce.CoerceUtil; import com.yahoo.elide.utils.coerce.converters.ISO8601DateSerde; @@ -271,13 +272,14 @@ public void testSortingAscending() throws Exception { Map sortRules = new HashMap(); sortRules.put("name", Sorting.SortOrder.asc); sortRules.put("modifiedDate", Sorting.SortOrder.desc); - Sorting sorting = new Sorting(sortRules); + Sorting sorting = new SortingImpl(sortRules, Item.class, dictionary); FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); Iterable loaded = testTransaction.loadObjects(EntityProjection.builder() .type(Item.class) .filterExpression(filter) + .sorting(sorting) .build(), mockScope); assertListContains(loaded, Lists.newArrayList(4L, 5L, 2L)); @@ -292,7 +294,7 @@ public void testSortingDescending() throws Exception { Map sortRules = new HashMap(); sortRules.put("name", Sorting.SortOrder.desc); sortRules.put("modifiedDate", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortRules); + Sorting sorting = new SortingImpl(sortRules, Item.class, dictionary); FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); @@ -314,9 +316,13 @@ public void testPaginationPageOne() throws Exception { Map sortRules = new HashMap(); sortRules.put("name", Sorting.SortOrder.desc); sortRules.put("modifiedDate", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortRules); + Sorting sorting = new SortingImpl(sortRules, Item.class, dictionary); - Pagination pagination = Pagination.fromOffsetAndLimit(1, 0, true); + PaginationImpl pagination = new PaginationImpl(Item.class, 0, 1, + PaginationImpl.DEFAULT_PAGE_LIMIT, + PaginationImpl.MAX_PAGE_LIMIT, + true, + false); FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); @@ -340,9 +346,13 @@ public void testPaginationPageTwo() throws Exception { Map sortRules = new HashMap(); sortRules.put("name", Sorting.SortOrder.desc); sortRules.put("modifiedDate", Sorting.SortOrder.asc); - Sorting sorting = new Sorting(sortRules); + Sorting sorting = new SortingImpl(sortRules, Item.class, dictionary); - Pagination pagination = Pagination.fromOffsetAndLimit(1, 1, true); + PaginationImpl pagination = new PaginationImpl(Item.class, 1, 1, + PaginationImpl.DEFAULT_PAGE_LIMIT, + PaginationImpl.MAX_PAGE_LIMIT, + true, + false); FilterExpression filter = filterParser.parseFilterExpression("name==cymbal*", Item.class, false); diff --git a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DependencyBinder.java b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DependencyBinder.java index ff1ff33680..a390b75284 100644 --- a/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DependencyBinder.java +++ b/elide-datastore/elide-datastore-search/src/test/java/com/yahoo/elide/datastores/search/DependencyBinder.java @@ -13,7 +13,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.dialect.DefaultFilterDialect; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; -import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.pagination.PaginationImpl; import com.yahoo.elide.datastores.jpa.JpaDataStore; import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; import com.yahoo.elide.resources.DefaultOpaqueUserFunction; @@ -63,8 +63,8 @@ protected void configure() { .withAuditLogger(new Slf4jLogger()) .withEntityDictionary(dictionary) .withPermissionExecutor(VerbosePermissionExecutor::new) - .withDefaultMaxPageSize(Pagination.MAX_PAGE_LIMIT) - .withDefaultPageSize(Pagination.DEFAULT_PAGE_LIMIT) + .withDefaultMaxPageSize(PaginationImpl.MAX_PAGE_LIMIT) + .withDefaultPageSize(PaginationImpl.DEFAULT_PAGE_LIMIT) .withISO8601Dates("yyyy-MM-dd'T'HH:mm:ss'Z'", TimeZone.getTimeZone("UTC")) .withJoinFilterDialect(new RSQLFilterDialect(dictionary)) .withJoinFilterDialect(new DefaultFilterDialect(dictionary)) diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java index ca51898fb1..dbe40d32a2 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/QueryRunner.java @@ -202,7 +202,7 @@ private ElideResponse executeGraphQLRequest(ObjectMapper mapper, Object principa tx.flush(requestScope); requestScope.runQueuedPreCommitTriggers(); - elide.getAuditLogger().commit(requestScope); + elide.getAuditLogger().commit(); tx.commit(requestScope); requestScope.runQueuedPostCommitTriggers(); diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java index 05885b4ef4..b9fc9d9a73 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/ConnectionContainer.java @@ -6,11 +6,11 @@ package com.yahoo.elide.graphql.containers; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; +import com.yahoo.elide.request.Pagination; import lombok.AllArgsConstructor; import lombok.Getter; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java index 4e0bd8b7c3..e0dcd1b052 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/containers/PageInfoContainer.java @@ -6,11 +6,11 @@ package com.yahoo.elide.graphql.containers; import com.yahoo.elide.core.PersistentResource; -import com.yahoo.elide.core.pagination.Pagination; import com.yahoo.elide.graphql.Environment; import com.yahoo.elide.graphql.KeyWord; import com.yahoo.elide.graphql.PersistentResourceFetcher; +import com.yahoo.elide.request.Pagination; import lombok.Getter; import java.util.List; import java.util.Optional; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java index e0a48a3af6..8bb65af498 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/parser/GraphQLEntityProjectionMaker.java @@ -24,14 +24,16 @@ import com.yahoo.elide.core.filter.dialect.ParseException; import com.yahoo.elide.core.filter.expression.AndFilterExpression; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.core.pagination.PaginationImpl; +import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.graphql.ModelBuilder; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.EntityProjection.EntityProjectionBuilder; +import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Relationship; +import com.yahoo.elide.request.Sorting; import graphql.language.Argument; import graphql.language.Document; import graphql.language.Field; @@ -48,7 +50,6 @@ import java.util.HashMap; import java.util.List; import java.util.Map; -import java.util.Optional; import java.util.stream.Collectors; import javax.ws.rs.BadRequestException; @@ -376,7 +377,7 @@ private static boolean isPaginationArgument(String argumentName) { */ private void addPagination(Argument argument, EntityProjectionBuilder projectionBuilder) { Pagination pagination = projectionBuilder.getPagination() == null - ? Pagination.getDefaultPagination(elideSettings) + ? PaginationImpl.getDefaultPagination(projectionBuilder.getType(), elideSettings) : projectionBuilder.getPagination(); Object argumentValue = variableResolver.resolveValue(argument.getValue()); @@ -384,9 +385,23 @@ private void addPagination(Argument argument, EntityProjectionBuilder projection ? ((BigInteger) argumentValue).intValue() : Integer.parseInt((String) argumentValue); if (ModelBuilder.ARGUMENT_FIRST.equals(argument.getName())) { - pagination.setLimit(value); + pagination = new PaginationImpl( + projectionBuilder.getType(), + pagination.getOffset(), + value, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultPageSize(), + pagination.returnPageTotals(), + false); } else if (ModelBuilder.ARGUMENT_AFTER.equals(argument.getName())) { - pagination.setOffset(value); + pagination = new PaginationImpl( + projectionBuilder.getType(), + value, + pagination.getLimit(), + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultPageSize(), + pagination.returnPageTotals(), + false); } projectionBuilder.pagination(pagination); @@ -400,23 +415,28 @@ private void addPagination(Argument argument, EntityProjectionBuilder projection * @param projectionBuilder projection that is being built */ private void addPageTotal(EntityProjectionBuilder projectionBuilder) { + PaginationImpl pagination; if (projectionBuilder.getPagination() == null) { - Optional pagination = Pagination.fromOffsetAndFirst( - Optional.empty(), - Optional.empty(), + pagination = new PaginationImpl( + projectionBuilder.getType(), + null, + null, + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), true, - elideSettings - ); - pagination.ifPresent(projectionBuilder::pagination); + false); + } else { - Optional pagination = Pagination.fromOffsetAndFirst( - Optional.of(String.valueOf(projectionBuilder.getPagination().getLimit())), - Optional.of(String.valueOf(projectionBuilder.getPagination().getOffset())), + pagination = new PaginationImpl( + projectionBuilder.getType(), + projectionBuilder.getPagination().getOffset(), + projectionBuilder.getPagination().getLimit(), + elideSettings.getDefaultPageSize(), + elideSettings.getDefaultMaxPageSize(), true, - elideSettings - ); - pagination.ifPresent(projectionBuilder::pagination); + false); } + projectionBuilder.pagination(pagination); } /** @@ -439,17 +459,15 @@ private static boolean isSortingArgument(String argumentName) { */ private void addSorting(Argument argument, EntityProjectionBuilder projectionBuilder) { String sortRule = (String) variableResolver.resolveValue(argument.getValue()); - Sorting sorting = Sorting.parseSortRule(sortRule); - // validate sorting rule try { - sorting.getValidSortingRules(projectionBuilder.getType(), entityDictionary); + Sorting sorting = SortingImpl.parseSortRule(sortRule, projectionBuilder.getType(), entityDictionary); + projectionBuilder.sorting(sorting); } catch (InvalidValueException e) { throw new BadRequestException("Invalid sorting clause " + sortRule + " for type " + entityDictionary.getJsonAliasFor(projectionBuilder.getType())); } - projectionBuilder.sorting(sorting); } /** diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java index 396012cf1e..1ba35f78b5 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java @@ -566,7 +566,7 @@ void testAuditLogging() throws IOException { endpoint.post(user1, graphQLRequestToJSON(graphQLRequest)); Mockito.verify(audit, Mockito.times(1)).log(Mockito.any()); - Mockito.verify(audit, Mockito.times(1)).commit(Mockito.any()); + Mockito.verify(audit, Mockito.times(1)).commit(); Mockito.verify(audit, Mockito.times(1)).clear(); } diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java index 2f80c76c29..ef6f5da2c7 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/ModelBuilderTest.java @@ -16,8 +16,8 @@ import com.yahoo.elide.core.ArgumentType; import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Sorting; import example.Author; import example.Book; import example.Publisher; diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/audit/InMemoryLogger.java b/elide-integration-tests/src/test/java/com/yahoo/elide/audit/InMemoryLogger.java index b8fc89d712..6095cb2cc6 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/audit/InMemoryLogger.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/audit/InMemoryLogger.java @@ -5,7 +5,6 @@ */ package com.yahoo.elide.audit; -import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.security.ChangeSpec; import com.google.common.collect.Sets; @@ -20,7 +19,7 @@ public class InMemoryLogger extends AuditLogger { public final Set logMessages = Sets.newConcurrentHashSet(); @Override - public void commit(RequestScope requestScope) throws IOException { + public void commit() throws IOException { for (LogMessage message : messages.get()) { if (message.getChangeSpec().isPresent()) { logMessages.add(changeSpecToString(message.getChangeSpec().get())); diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java b/elide-integration-tests/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java index 04d4e33c2e..d110942f55 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/audit/TestAuditLogger.java @@ -5,8 +5,6 @@ */ package com.yahoo.elide.audit; -import com.yahoo.elide.core.RequestScope; - import java.io.IOException; /** @@ -23,7 +21,7 @@ public void log(LogMessage message) { } @Override - public void commit(RequestScope requestScope) throws IOException { + public void commit() throws IOException { commitCount++; } } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/PaginateIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/PaginateIT.java index 7a4ce1f116..d6b13dc92a 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/PaginateIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/PaginateIT.java @@ -606,7 +606,7 @@ void testPaginateAnnotationMaxLimit() { .get(url) .then() .body("errors", hasSize(1), - "errors[0]", containsString("page[limit] value must be less than or equal to 10")) + "errors[0]", containsString("InvalidValueException: Invalid value: Pagination limit must be less than or equal to 10")) .statusCode(BAD_REQUEST_400); } diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java index 94b8a85bf7..fb823c142e 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ResourceIT.java @@ -45,7 +45,7 @@ import com.yahoo.elide.core.filter.InfixPredicate; import com.yahoo.elide.core.filter.PostfixPredicate; import com.yahoo.elide.core.filter.PrefixPredicate; -import com.yahoo.elide.core.pagination.Pagination; +import com.yahoo.elide.core.pagination.PaginationImpl; import com.yahoo.elide.initialization.IntegrationTest; import com.yahoo.elide.jsonapi.models.JsonApiDocument; import com.yahoo.elide.request.EntityProjection; @@ -2674,8 +2674,8 @@ public void testSpecialCharacterLikeQueryHQL(FilterPredicate filterPredicate, in EntityDictionary dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(Book.class); when(scope.getDictionary()).thenReturn(dictionary); - Pagination pagination = mock(Pagination.class); - when(pagination.isGenerateTotals()).thenReturn(true); + PaginationImpl pagination = mock(PaginationImpl.class); + when(pagination.returnPageTotals()).thenReturn(true); tx.loadObjects(EntityProjection.builder() .type(Book.class) @@ -2684,7 +2684,7 @@ public void testSpecialCharacterLikeQueryHQL(FilterPredicate filterPredicate, in .build(), scope); tx.commit(scope); tx.close(); - verify(pagination).setPageTotals(noOfRecords); + verify(pagination).setPageTotals((long) noOfRecords); } @Test diff --git a/elide-integration-tests/src/test/java/example/EntityWithPaginateMaxLimit.java b/elide-integration-tests/src/test/java/example/EntityWithPaginateMaxLimit.java index 12bab645bd..96d408abe1 100644 --- a/elide-integration-tests/src/test/java/example/EntityWithPaginateMaxLimit.java +++ b/elide-integration-tests/src/test/java/example/EntityWithPaginateMaxLimit.java @@ -15,7 +15,7 @@ @Entity @Include(rootLevel = true) -@Paginate(maxLimit = 10) +@Paginate(maxLimit = 10, defaultLimit = 10) public class EntityWithPaginateMaxLimit extends BaseId { @Getter @Setter diff --git a/elide-integration-tests/src/test/java/example/TestCheckMappings.java b/elide-integration-tests/src/test/java/example/TestCheckMappings.java index 277c1979de..5944ac73c9 100644 --- a/elide-integration-tests/src/test/java/example/TestCheckMappings.java +++ b/elide-integration-tests/src/test/java/example/TestCheckMappings.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.security.checks.Check; -import com.yahoo.elide.security.checks.prefab.Common; import com.yahoo.elide.security.checks.prefab.Role; import com.google.common.collect.ImmutableMap; @@ -33,7 +32,6 @@ public class TestCheckMappings { .put("child4Parent5", Child4Parent5Check.class) .put("checkActsLikeFilter", AnotherFilterExpressionCheckObj.CheckActsLikeFilter.class) .put("noRead", CreateButNoRead.NOREAD.class) - .put("updateOnCreate", Common.UpdateOnCreate.class) .put("checkLE", FilterExpressionCheckObj.CheckLE.class) .put("checkRestrictUser", FilterExpressionCheckObj.CheckRestrictUser.class) .put("specialValue", SpecialRead.SpecialValue.class) diff --git a/elide-integration-tests/src/test/java/example/User.java b/elide-integration-tests/src/test/java/example/User.java index dedb7cf767..b18d161391 100644 --- a/elide-integration-tests/src/test/java/example/User.java +++ b/elide-integration-tests/src/test/java/example/User.java @@ -63,7 +63,7 @@ public void setReversedPassword(String reversedPassword) { this.reversedPassword = reversedPassword; } - @UpdatePermission(expression = "adminRoleCheck OR updateOnCreate") + @UpdatePermission(expression = "adminRoleCheck") public int getRole() { return role; } From f311feefd72204c9b5fff2adda369511faef2fd0 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Thu, 6 Feb 2020 21:47:01 -0600 Subject: [PATCH 04/16] Refactor share permission (#1154) * Refactored Sorting * Refactored Pagination * Refactored Pagination * Pagination refactor builds and tests pass * Codacy fixes * Refactored SharePermission to NonTransferable * Fixed build * Fixed startup bug * Fixed codacy and inspection comments * Update elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java Co-Authored-By: Jon Kilroy * Inspection rework Co-authored-by: Jon Kilroy --- .../elide/annotation/NonTransferable.java | 30 ++++ .../elide/annotation/SharePermission.java | 33 ---- .../yahoo/elide/core/EntityDictionary.java | 9 +- .../yahoo/elide/core/EntityPermissions.java | 6 +- .../yahoo/elide/core/PersistentResource.java | 16 +- .../yahoo/elide/request/EntityProjection.java | 1 - .../com/yahoo/elide/request/Pagination.java | 2 +- .../executors/ActivePermissionExecutor.java | 8 +- .../permissions/PermissionCondition.java | 4 +- .../elide/core/EntityDictionaryTest.java | 4 +- .../core/PersistenceResourceTestSetup.java | 10 +- .../elide/core/PersistentResourceTest.java | 149 +++--------------- elide-core/src/test/java/example/Author.java | 2 - elide-core/src/test/java/example/Book.java | 2 - elide-core/src/test/java/example/Child.java | 2 - elide-core/src/test/java/example/Editor.java | 2 - elide-core/src/test/java/example/Left.java | 2 + .../src/test/java/example/MapColorShape.java | 2 - .../src/test/java/example/NoShareEntity.java | 2 + elide-core/src/test/java/example/Right.java | 2 - .../test/java/example/UpdateAndCreate.java | 2 - .../ContainerWithPackageShare.java | 16 +- .../ShareableWithPackageShare.java | 4 +- .../Untransferable.java} | 6 +- .../package-info.java | 6 +- .../aggregation/metadata/MetaDataStore.java | 3 +- .../datastores/aggregation/query/Query.java | 1 - .../hql/AbstractHQLQueryBuilder.java | 1 - .../hql/AbstractHQLQueryBuilderTest.java | 1 - .../search/SearchDataTransaction.java | 1 - .../elide/models/generics/package-info.java | 3 - .../yahoo/elide/example/models/Comment.java | 2 - .../com/yahoo/elide/example/models/User.java | 2 - .../com/yahoo/elide/example/models/User.java | 2 - .../elide/graphql/GraphQLEndpointTest.java | 6 +- .../src/test/java/example/Author.java | 2 - elide-graphql/src/test/java/example/Book.java | 2 - .../graphqlEndpointTestModels/Author.java | 16 +- ...sallowShare.java => DisallowTransfer.java} | 4 +- .../{ShareableIT.java => TransferableIT.java} | 143 ++++++++--------- .../AnotherFilterExpressionCheckObj.java | 2 - .../src/test/java/example/AuditEntity.java | 2 - .../test/java/example/AuditEntityInverse.java | 2 - .../src/test/java/example/Author.java | 2 - .../src/test/java/example/Book.java | 2 - .../src/test/java/example/Chapter.java | 2 - .../src/test/java/example/Child.java | 2 - .../src/test/java/example/Container.java | 22 +-- .../src/test/java/example/Editor.java | 2 - .../example/FilterExpressionCheckObj.java | 2 - .../src/test/java/example/MapColorShape.java | 2 - ...onal.java => NoTransferBiDirectional.java} | 12 +- .../src/test/java/example/Parent.java | 2 - .../src/test/java/example/Property.java | 2 - .../src/test/java/example/Smartphone.java | 2 - .../src/test/java/example/Tractor.java | 2 - .../{Unshareable.java => Transferable.java} | 8 +- .../{Shareable.java => Untransferable.java} | 12 +- 58 files changed, 215 insertions(+), 378 deletions(-) create mode 100644 elide-annotations/src/main/java/com/yahoo/elide/annotation/NonTransferable.java delete mode 100644 elide-annotations/src/main/java/com/yahoo/elide/annotation/SharePermission.java rename elide-core/src/test/java/example/{packageshareable => nontransferable}/ContainerWithPackageShare.java (70%) rename elide-core/src/test/java/example/{packageshareable => nontransferable}/ShareableWithPackageShare.java (89%) rename elide-core/src/test/java/example/{packageshareable/UnshareableWithEntityUnshare.java => nontransferable/Untransferable.java} (85%) rename elide-core/src/test/java/example/{packageshareable => nontransferable}/package-info.java (56%) rename elide-graphql/src/test/java/graphqlEndpointTestModels/{DisallowShare.java => DisallowTransfer.java} (85%) rename elide-integration-tests/src/test/java/com/yahoo/elide/tests/{ShareableIT.java => TransferableIT.java} (77%) rename elide-integration-tests/src/test/java/example/{NoShareBiDirectional.java => NoTransferBiDirectional.java} (62%) rename elide-integration-tests/src/test/java/example/{Unshareable.java => Transferable.java} (80%) rename elide-integration-tests/src/test/java/example/{Shareable.java => Untransferable.java} (72%) diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/NonTransferable.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/NonTransferable.java new file mode 100644 index 0000000000..38c428f265 --- /dev/null +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/NonTransferable.java @@ -0,0 +1,30 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.annotation; + +import static java.lang.annotation.ElementType.PACKAGE; +import static java.lang.annotation.ElementType.TYPE; +import static java.lang.annotation.RetentionPolicy.RUNTIME; + +import java.lang.annotation.Inherited; +import java.lang.annotation.Retention; +import java.lang.annotation.Target; + +/** + * Marks that the given entity cannot be added to another collection after creation of the entity. + */ +@Target({TYPE, PACKAGE}) +@Retention(RUNTIME) +@Inherited +public @interface NonTransferable { + + /** + * If NonTransferable is used at the package level, it can be disabled for individual entities by setting + * this flag to false. + * @return true if enabled. + */ + boolean enabled() default true; +} diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/SharePermission.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/SharePermission.java deleted file mode 100644 index f53f697a6a..0000000000 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/SharePermission.java +++ /dev/null @@ -1,33 +0,0 @@ -/* - * Copyright 2016, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.annotation; - -import static java.lang.annotation.ElementType.PACKAGE; -import static java.lang.annotation.ElementType.TYPE; -import static java.lang.annotation.RetentionPolicy.RUNTIME; - -import java.lang.annotation.Inherited; -import java.lang.annotation.Retention; -import java.lang.annotation.Target; - -/** - * A permission that is checked whenever an object is loaded without the context of a lineage and assigned - * to a relationship or collection. If SharePermission is specified, checking SharePermission falls back to checking - * ReadPermission. Otherwise, the entity is not shareable. - */ -@Target({TYPE, PACKAGE}) -@Retention(RUNTIME) -@Inherited -public @interface SharePermission { - - /** - * A boolean value indicating if the entity is shareable. If not specifying, shareable is true. Setting shareable to - * false provide a way to override package level SharePermission. - * - * @return the boolean if entity is shareable - */ - boolean sharable() default true; -} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index ee096a0708..def633e65c 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -14,8 +14,8 @@ import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.MappedInterface; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.SecurityCheck; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.core.exceptions.InternalServerErrorException; import com.yahoo.elide.core.exceptions.InvalidAttributeException; @@ -830,9 +830,10 @@ public void initializeEntity(T entity) { * @param entityClass the entity type to check for the shareable permissions * @return true if entityClass is shareable. False otherwise. */ - public boolean isShareable(Class entityClass) { - return getAnnotation(entityClass, SharePermission.class) != null - && getAnnotation(entityClass, SharePermission.class).sharable(); + public boolean isTransferable(Class entityClass) { + NonTransferable nonTransferable = getAnnotation(entityClass, NonTransferable.class); + + return (nonTransferable == null || !nonTransferable.enabled()); } /** diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityPermissions.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityPermissions.java index 181fab7c03..b9aafcf256 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityPermissions.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityPermissions.java @@ -7,8 +7,8 @@ import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.DeletePermission; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.generated.parsers.ExpressionLexer; import com.yahoo.elide.generated.parsers.ExpressionParser; @@ -43,7 +43,7 @@ public class EntityPermissions implements CheckInstantiator { ReadPermission.class, CreatePermission.class, DeletePermission.class, - SharePermission.class, + NonTransferable.class, UpdatePermission.class ); @@ -79,7 +79,7 @@ public EntityPermissions(EntityDictionary dictionary, final Map fieldPermissions = new HashMap<>(); fieldOrMethodList.stream() .forEach(member -> bindMemberPermissions(fieldPermissions, member, annotationClass)); - if (annotationClass != SharePermission.class) { + if (annotationClass != NonTransferable.class) { ParseTree classPermission = bindClassPermissions(cls, annotationClass); if (classPermission != null || !fieldPermissions.isEmpty()) { bindings.put(annotationClass, new AnnotationBinding(classPermission, fieldPermissions)); diff --git a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java index 4aeab2a1f2..388cf751eb 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/PersistentResource.java @@ -8,8 +8,8 @@ import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.DeletePermission; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.audit.InvalidSyntaxException; import com.yahoo.elide.audit.LogMessage; @@ -139,7 +139,7 @@ public static PersistentResource createObject( //hashcode and equals are only based on the ID/UUID & type. assignId(newResource, id); - // Keep track of new resources for non shareable resources + // Keep track of new resources for non-transferable resources requestScope.getNewPersistentResources().add(newResource); checkPermission(CreatePermission.class, newResource); @@ -477,7 +477,7 @@ protected boolean updateToManyRelation(String fieldName, added = Sets.difference(updated, deleted); - checkSharePermission(added); + checkTransferablePermission(added); Collection collection = (Collection) this.getValueUnchecked(fieldName); @@ -539,11 +539,11 @@ protected boolean updateToOneRelation(String fieldName, if (newValue == null) { return false; } - checkSharePermission(resourceIdentifiers); + checkTransferablePermission(resourceIdentifiers); } else if (oldResource.getObject().equals(newValue)) { return false; } else { - checkSharePermission(resourceIdentifiers); + checkTransferablePermission(resourceIdentifiers); if (hasInverseRelation(fieldName)) { deleteInverseRelation(fieldName, oldResource.getObject()); oldResource.markDirty(); @@ -712,7 +712,7 @@ public void addRelation(String fieldName, PersistentResource newRelation) { if (!newRelation.isNewlyCreated() && relationshipAlreadyExists(fieldName, newRelation)) { return; } - checkSharePermission(Collections.singleton(newRelation)); + checkTransferablePermission(Collections.singleton(newRelation)); Object relation = this.getValueUnchecked(fieldName); if (relation instanceof Collection) { @@ -736,7 +736,7 @@ public void addRelation(String fieldName, PersistentResource newRelation) { * * @param resourceIdentifiers The persistent resources that are being added */ - protected void checkSharePermission(Set resourceIdentifiers) { + protected void checkTransferablePermission(Set resourceIdentifiers) { if (resourceIdentifiers == null) { return; } @@ -746,7 +746,7 @@ protected void checkSharePermission(Set resourceIdentifiers) for (PersistentResource persistentResource : resourceIdentifiers) { if (!newResources.contains(persistentResource) && !lineage.getRecord(persistentResource.getType()).contains(persistentResource)) { - checkPermission(SharePermission.class, persistentResource); + checkPermission(NonTransferable.class, persistentResource); } } } diff --git a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java index fd199678db..0b2f1ea4c4 100644 --- a/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java +++ b/elide-core/src/main/java/com/yahoo/elide/request/EntityProjection.java @@ -7,7 +7,6 @@ package com.yahoo.elide.request; import com.yahoo.elide.core.filter.expression.FilterExpression; - import com.google.common.collect.Sets; import lombok.AllArgsConstructor; import lombok.Builder; diff --git a/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java b/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java index a0d69f8170..22c8220685 100644 --- a/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java +++ b/elide-core/src/main/java/com/yahoo/elide/request/Pagination.java @@ -1,5 +1,5 @@ /* - * Copyright 2019, Yahoo Inc. + * Copyright 2020, Yahoo Inc. * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ diff --git a/elide-core/src/main/java/com/yahoo/elide/security/executors/ActivePermissionExecutor.java b/elide-core/src/main/java/com/yahoo/elide/security/executors/ActivePermissionExecutor.java index 607718aa37..139bdf90b3 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/executors/ActivePermissionExecutor.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/executors/ActivePermissionExecutor.java @@ -11,8 +11,8 @@ import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.DeletePermission; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RequestScope; @@ -98,7 +98,7 @@ public ExpressionResult checkPermission( } /** - * Check permission on class. Checking on SharePermission falls to check ReadPermission. + * Check permission on class. Checking on Transferable falls to check ReadPermission. * * @param annotationClass annotation class * @param resource resource @@ -114,8 +114,8 @@ public ExpressionResult checkPermission(Class annotati PersistentResource resource, ChangeSpec changeSpec) { Supplier expressionSupplier = () -> { - if (SharePermission.class == annotationClass) { - if (requestScope.getDictionary().isShareable(resource.getResourceClass())) { + if (NonTransferable.class == annotationClass) { + if (requestScope.getDictionary().isTransferable(resource.getResourceClass())) { return expressionBuilder.buildAnyFieldExpressions(resource, ReadPermission.class, changeSpec); } return PermissionExpressionBuilder.FAIL_EXPRESSION; diff --git a/elide-core/src/main/java/com/yahoo/elide/security/permissions/PermissionCondition.java b/elide-core/src/main/java/com/yahoo/elide/security/permissions/PermissionCondition.java index b493143d96..ece7359fd4 100644 --- a/elide-core/src/main/java/com/yahoo/elide/security/permissions/PermissionCondition.java +++ b/elide-core/src/main/java/com/yahoo/elide/security/permissions/PermissionCondition.java @@ -8,8 +8,8 @@ import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.DeletePermission; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.PersistentResource; @@ -37,7 +37,7 @@ public class PermissionCondition { .put(UpdatePermission.class, "UPDATE") .put(DeletePermission.class, "DELETE") .put(CreatePermission.class, "CREATE") - .put(SharePermission.class, "SHARE") + .put(NonTransferable.class, "NO TRANSFER") .build(); /** diff --git a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java index f7da8509ac..229d4becc4 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java @@ -424,12 +424,12 @@ public void testGetIdAnnotationsSubClass() throws Exception { @Test public void testIsSharableTrue() throws Exception { - assertTrue(isShareable(Right.class)); + assertTrue(isTransferable(Right.class)); } @Test public void testIsSharableFalse() throws Exception { - assertFalse(isShareable(Left.class)); + assertFalse(isTransferable(Left.class)); } @Test diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java index 9558f6e917..b96db6e073 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java @@ -43,13 +43,13 @@ import example.Publisher; import example.Right; import example.UpdateAndCreate; -import example.packageshareable.ContainerWithPackageShare; -import example.packageshareable.ShareableWithPackageShare; -import example.packageshareable.UnshareableWithEntityUnshare; +import example.nontransferable.ContainerWithPackageShare; +import example.nontransferable.ShareableWithPackageShare; +import example.nontransferable.Untransferable; +import nocreate.NoCreateEntity; import lombok.AllArgsConstructor; import lombok.EqualsAndHashCode; -import nocreate.NoCreateEntity; import java.util.Collection; import java.util.HashSet; @@ -97,7 +97,7 @@ protected static EntityDictionary initDictionary() { dictionary.bindEntity(ComputedBean.class); dictionary.bindEntity(ContainerWithPackageShare.class); dictionary.bindEntity(ShareableWithPackageShare.class); - dictionary.bindEntity(UnshareableWithEntityUnshare.class); + dictionary.bindEntity(Untransferable.class); return dictionary; } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java index 79c8a3f8fc..bd2a48183e 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java @@ -24,6 +24,8 @@ import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.ReadPermission; +import com.yahoo.elide.annotation.UpdatePermission; +import com.yahoo.elide.audit.AuditLogger; import com.yahoo.elide.audit.LogMessage; import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; @@ -66,25 +68,21 @@ import example.Parent; import example.Right; import example.Shape; -import example.packageshareable.ContainerWithPackageShare; -import example.packageshareable.ShareableWithPackageShare; -import example.packageshareable.UnshareableWithEntityUnshare; -<<<<<<< HEAD +import example.nontransferable.ContainerWithPackageShare; +import example.nontransferable.ShareableWithPackageShare; +import example.nontransferable.Untransferable; +import nocreate.NoCreateEntity; import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; -import org.junit.jupiter.api.Test; -import org.mockito.Answers; - -======= import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestInstance; +import org.mockito.Answers; import org.mockito.ArgumentCaptor; + import lombok.AllArgsConstructor; import lombok.EqualsAndHashCode; ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) -import nocreate.NoCreateEntity; import java.util.ArrayList; import java.util.Arrays; @@ -113,16 +111,9 @@ public class PersistentResourceTest extends PersistenceResourceTestSetup { private final User badUser = new User(-1); private DataStoreTransaction tx = mock(DataStoreTransaction.class); -<<<<<<< HEAD - public PersistentResourceTest() { - goodUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(1)); - badUserScope = buildRequestScope(mock(DataStoreTransaction.class), new User(-1)); - reset(goodUserScope.getTransaction()); -======= @BeforeEach public void beforeTest() { reset(tx); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test @@ -130,15 +121,7 @@ public void testUpdateToOneRelationHookInAddRelation() { FunWithPermissions fun = new FunWithPermissions(); Child child = newChild(1); -<<<<<<< HEAD - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation3", childResource); @@ -187,12 +170,7 @@ public void testUpdateToOneRelationHookInClearRelation() { when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child1); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); - ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); funResource.clearRelation("relation3"); @@ -205,16 +183,7 @@ public void testUpdateToManyRelationHookInAddRelationBidirection() { Parent parent = new Parent(); Child child = newChild(1); -<<<<<<< HEAD - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); - ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); parentResource.addRelation("children", childResource); @@ -594,11 +563,7 @@ public void testSuccessfulOneToOneRelationshipAdd() throws Exception { left.setId(2); right.setId(3); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "2", goodScope); @@ -666,11 +631,7 @@ public void testSuccessfulOneToOneRelationshipAddNull() throws Exception { */ public void testSuccessfulManyToManyRelationshipUpdate() throws Exception { Parent parent = new Parent(); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) Child child1 = newChild(1); Child child2 = newChild(2); @@ -867,13 +828,7 @@ public void testGetRelationWithPredicateSuccess() { MultivaluedMap queryParams = new MultivaluedHashMap<>(); queryParams.add("filter[child.name]", "paul john"); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope("/child", tx, goodUser, queryParams); -======= - - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1044,13 +999,7 @@ public void testGetRelationsNoEntityAccess2() { void testDeleteResourceSuccess() { Parent parent = newParent(1); -<<<<<<< HEAD - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); @@ -1068,13 +1017,7 @@ void testDeleteCascades() { invoice.setItems(Sets.newHashSet(item)); item.setInvoice(invoice); -<<<<<<< HEAD - User goodUser = new User(1); - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource invoiceResource = new PersistentResource<>(invoice, null, "1", goodScope); @@ -1119,15 +1062,7 @@ void testDeleteResourceForbidden() { NoDeleteEntity nodelete = new NoDeleteEntity(); nodelete.setId(1); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) - PersistentResource nodeleteResource = new PersistentResource<>(nodelete, null, "1", goodScope); assertThrows(ForbiddenAccessException.class, nodeleteResource::deleteResource); @@ -1142,15 +1077,8 @@ void testAddRelationSuccess() { Child child = newChild(1); -<<<<<<< HEAD - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); funResource.addRelation("relation1", childResource); @@ -1169,14 +1097,7 @@ void testAddRelationForbiddenByField() { Child child = newChild(1); -<<<<<<< HEAD - User badUser = new User(-1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope badScope = buildRequestScope(tx, badUser); -======= - RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", badScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", badScope); assertThrows(ForbiddenAccessException.class, () -> funResource.addRelation("relation1", childResource)); @@ -1189,14 +1110,7 @@ void testAddRelationForbiddenByEntity() { Child child = newChild(2); noUpdate.setChildren(Sets.newHashSet()); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource noUpdateResource = new PersistentResource<>(noUpdate, null, "1", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "2", goodScope); assertThrows(ForbiddenAccessException.class, () -> noUpdateResource.addRelation("children", childResource)); @@ -1208,14 +1122,7 @@ public void testAddRelationInvalidRelation() { Child child = newChild(1); -<<<<<<< HEAD - User goodUser = new User(1); - - DataStoreTransaction tx = mock(DataStoreTransaction.class); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "3", goodScope); PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.addRelation("invalid", childResource)); @@ -1229,13 +1136,7 @@ public void testRemoveToManyRelationSuccess() { Parent parent3 = newParent(3, child); child.setParents(Sets.newHashSet(parent1, parent2, parent3)); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(parent1, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1258,13 +1159,7 @@ public void testRemoveToOneRelationSuccess() { Child child = newChild(1); fun.setRelation3(child); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(child, null, "1", goodScope); @@ -2148,7 +2043,7 @@ public void testIsIdGenerated() { } @Test - public void testSharePermissionErrorOnUpdateSingularRelationship() { + public void testTransferPermissionErrorOnUpdateSingularRelationship() { example.User userModel = new example.User(); userModel.setId(1); @@ -2181,17 +2076,17 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test - public void testSharePermissionErrorOnUpdateRelationshipPackageLevel() { + public void testTransferPermissionErrorOnUpdateRelationshipPackageLevel() { ContainerWithPackageShare containerWithPackageShare = new ContainerWithPackageShare(); - UnshareableWithEntityUnshare unshareableWithEntityUnshare = new UnshareableWithEntityUnshare(); - unshareableWithEntityUnshare.setContainerWithPackageShare(containerWithPackageShare); + Untransferable untransferable = new Untransferable(); + untransferable.setContainerWithPackageShare(containerWithPackageShare); List unShareableList = new ArrayList<>(); - unShareableList.add(new ResourceIdentifier("unshareableWithEntityUnshare", "1").castToResource()); + unShareableList.add(new ResourceIdentifier("untransferable", "1").castToResource()); Relationship unShareales = new Relationship(null, new Data<>(unShareableList)); - when(tx.loadObject(any(), eq(1L), any())).thenReturn(unshareableWithEntityUnshare); + when(tx.loadObject(any(), eq(1L), any())).thenReturn(untransferable); RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); @@ -2204,11 +2099,11 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertThrows( ForbiddenAccessException.class, () -> containerResource.updateRelation( - "unshareableWithEntityUnshares", unShareales.toPersistentResources(goodScope))); + "untransferables", unShareales.toPersistentResources(goodScope))); } @Test - public void testSharePermissionSuccessOnUpdateManyRelationshipPackageLevel() { + public void testTransferPermissionSuccessOnUpdateManyRelationshipPackageLevel() { ContainerWithPackageShare containerWithPackageShare = new ContainerWithPackageShare(); ShareableWithPackageShare shareableWithPackageShare = new ShareableWithPackageShare(); @@ -2235,7 +2130,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test - public void testSharePermissionErrorOnUpdateManyRelationship() { + public void testTransferPermissionErrorOnUpdateManyRelationship() { example.User userModel = new example.User(); userModel.setId(1); @@ -2266,7 +2161,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test - public void testSharePermissionSuccessOnUpdateManyRelationship() { + public void testTransferPermissionSuccessOnUpdateManyRelationship() { example.User userModel = new example.User(); userModel.setId(1); @@ -2302,7 +2197,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test - public void testSharePermissionSuccessOnUpdateSingularRelationship() { + public void testTransferPermissionSuccessOnUpdateSingularRelationship() { example.User userModel = new example.User(); userModel.setId(1); @@ -2334,7 +2229,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) } @Test - public void testSharePermissionSuccessOnClearSingularRelationship() { + public void testTransferPermissionSuccessOnClearSingularRelationship() { example.User userModel = new example.User(); userModel.setId(1); @@ -2567,8 +2462,6 @@ public void testEqualsAndHashcode() { assertNotEquals(resourceWithDifferentId, resourceWithId); assertNotEquals(resourceWithId, resourceWithDifferentId); } -<<<<<<< HEAD -======= private PersistentResource bootstrapPersistentResource(T obj) { return bootstrapPersistentResource(obj, mock(DataStoreTransaction.class)); @@ -2646,7 +2539,6 @@ public ChangeSpecModel(final Function checkFunction) { @ReadPermission(expression = "allow all") @UpdatePermission(expression = "allow all") @DeletePermission(expression = "allow all") - @SharePermission public static final class ChangeSpecChild { @Id public long id; @@ -2689,5 +2581,4 @@ private com.yahoo.elide.request.Relationship getRelationship(Class type, Stri .build()) .build(); } ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) } diff --git a/elide-core/src/test/java/example/Author.java b/elide-core/src/test/java/example/Author.java index e6167b3abe..d3f65d4659 100644 --- a/elide-core/src/test/java/example/Author.java +++ b/elide-core/src/test/java/example/Author.java @@ -8,7 +8,6 @@ import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import lombok.Getter; import lombok.Setter; @@ -31,7 +30,6 @@ @Entity @Table(name = "author") @Include(rootLevel = true) -@SharePermission @Audit(action = Audit.Action.CREATE, operation = 10, logStatement = "{0}", diff --git a/elide-core/src/test/java/example/Book.java b/elide-core/src/test/java/example/Book.java index e645b2664a..e5ea10a586 100644 --- a/elide-core/src/test/java/example/Book.java +++ b/elide-core/src/test/java/example/Book.java @@ -24,7 +24,6 @@ import com.yahoo.elide.annotation.OnUpdatePreCommit; import com.yahoo.elide.annotation.OnUpdatePreSecurity; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; @@ -52,7 +51,6 @@ @CreatePermission(expression = "Book operation check") @UpdatePermission(expression = "Book operation check") @DeletePermission(expression = "Book operation check") -@SharePermission @Table(name = "book") @Include(rootLevel = true) @Audit(action = Audit.Action.CREATE, diff --git a/elide-core/src/test/java/example/Child.java b/elide-core/src/test/java/example/Child.java index 407a85a2d1..ded9b63af5 100644 --- a/elide-core/src/test/java/example/Child.java +++ b/elide-core/src/test/java/example/Child.java @@ -9,7 +9,6 @@ import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.OperationCheck; @@ -30,7 +29,6 @@ @Entity(name = "childEntity") @CreatePermission(expression = "initCheck") -@SharePermission @ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheck") @Include(rootLevel = true, type = "child") @Audit(action = Audit.Action.DELETE, diff --git a/elide-core/src/test/java/example/Editor.java b/elide-core/src/test/java/example/Editor.java index 486618230e..a6b58fa42c 100644 --- a/elide-core/src/test/java/example/Editor.java +++ b/elide-core/src/test/java/example/Editor.java @@ -12,7 +12,6 @@ import com.yahoo.elide.annotation.FilterExpressionPath; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.NotNullPredicate; @@ -37,7 +36,6 @@ @Entity @Table(name = "editor") @Include(rootLevel = true) -@SharePermission @Audit(action = Audit.Action.CREATE, operation = 10, logStatement = "{0}", diff --git a/elide-core/src/test/java/example/Left.java b/elide-core/src/test/java/example/Left.java index bb4d540f10..55f932a0dc 100644 --- a/elide-core/src/test/java/example/Left.java +++ b/elide-core/src/test/java/example/Left.java @@ -7,6 +7,7 @@ import com.yahoo.elide.annotation.DeletePermission; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import com.yahoo.elide.annotation.UpdatePermission; import com.fasterxml.jackson.annotation.JsonIgnore; @@ -27,6 +28,7 @@ @Entity @Table(name = "xleft") // left is SQL keyword @DeletePermission(expression = "negativeIntegerUser") +@NonTransferable public class Left { @JsonIgnore private long id; diff --git a/elide-core/src/test/java/example/MapColorShape.java b/elide-core/src/test/java/example/MapColorShape.java index 82273439f2..be768b1a3a 100644 --- a/elide-core/src/test/java/example/MapColorShape.java +++ b/elide-core/src/test/java/example/MapColorShape.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import java.util.LinkedHashMap; import java.util.Map; @@ -23,7 +22,6 @@ @Entity @Table(name = "color_shape") @Include(rootLevel = true) -@SharePermission public class MapColorShape { private long id; private String name; diff --git a/elide-core/src/test/java/example/NoShareEntity.java b/elide-core/src/test/java/example/NoShareEntity.java index 732c847a1f..15e458e87b 100644 --- a/elide-core/src/test/java/example/NoShareEntity.java +++ b/elide-core/src/test/java/example/NoShareEntity.java @@ -6,6 +6,7 @@ package example; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import javax.persistence.Entity; import javax.persistence.Id; @@ -15,6 +16,7 @@ // Hibernate @Entity @Table(name = "noshare") +@NonTransferable public class NoShareEntity { private long id; diff --git a/elide-core/src/test/java/example/Right.java b/elide-core/src/test/java/example/Right.java index 6b4f151006..f53976e2cd 100644 --- a/elide-core/src/test/java/example/Right.java +++ b/elide-core/src/test/java/example/Right.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.fasterxml.jackson.annotation.JsonIgnore; @@ -25,7 +24,6 @@ @Include(rootLevel = true, type = "right") // optional here because class has this name -@SharePermission @UpdatePermission(expression = "deny all") @Entity @Table(name = "xright") // right is SQL keyword diff --git a/elide-core/src/test/java/example/UpdateAndCreate.java b/elide-core/src/test/java/example/UpdateAndCreate.java index 1b8a8b3f01..f2a8f316dc 100644 --- a/elide-core/src/test/java/example/UpdateAndCreate.java +++ b/elide-core/src/test/java/example/UpdateAndCreate.java @@ -8,7 +8,6 @@ import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import lombok.Getter; @@ -29,7 +28,6 @@ @Include(rootLevel = true) @CreatePermission(expression = "Principal is user one OR Principal is user two") @UpdatePermission(expression = "Principal is user two") -@SharePermission @Audit(action = Audit.Action.CREATE, operation = 10, logStatement = "{0}", diff --git a/elide-core/src/test/java/example/packageshareable/ContainerWithPackageShare.java b/elide-core/src/test/java/example/nontransferable/ContainerWithPackageShare.java similarity index 70% rename from elide-core/src/test/java/example/packageshareable/ContainerWithPackageShare.java rename to elide-core/src/test/java/example/nontransferable/ContainerWithPackageShare.java index 5e4afa878e..91d01be7f9 100644 --- a/elide-core/src/test/java/example/packageshareable/ContainerWithPackageShare.java +++ b/elide-core/src/test/java/example/nontransferable/ContainerWithPackageShare.java @@ -3,9 +3,10 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package example.packageshareable; +package example.nontransferable; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import java.util.Collection; @@ -17,13 +18,14 @@ import javax.persistence.OneToMany; /** - * Container for ShareableWithPackageShare and UnshareableWithEntityUnshare. + * Container for ShareableWithPackageShare and Untransferable. */ @Entity @Include(rootLevel = true) +@NonTransferable(enabled = false) public class ContainerWithPackageShare { private long id; - private Collection unshareableWithEntityUnshares; + private Collection untransferables; private Collection shareableWithPackageShares; @Id @@ -37,12 +39,12 @@ public void setId(long id) { } @OneToMany(fetch = FetchType.LAZY) - public Collection getUnshareableWithEntityUnshares() { - return unshareableWithEntityUnshares; + public Collection getUntransferables() { + return untransferables; } - public void setUnshareableWithEntityUnshares(Collection unshareables) { - this.unshareableWithEntityUnshares = unshareables; + public void setUntransferables(Collection unshareables) { + this.untransferables = unshareables; } @OneToMany(fetch = FetchType.LAZY) diff --git a/elide-core/src/test/java/example/packageshareable/ShareableWithPackageShare.java b/elide-core/src/test/java/example/nontransferable/ShareableWithPackageShare.java similarity index 89% rename from elide-core/src/test/java/example/packageshareable/ShareableWithPackageShare.java rename to elide-core/src/test/java/example/nontransferable/ShareableWithPackageShare.java index c2316a8f06..9446397842 100644 --- a/elide-core/src/test/java/example/packageshareable/ShareableWithPackageShare.java +++ b/elide-core/src/test/java/example/nontransferable/ShareableWithPackageShare.java @@ -3,9 +3,10 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package example.packageshareable; +package example.nontransferable; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import javax.persistence.Entity; import javax.persistence.FetchType; @@ -19,6 +20,7 @@ */ @Entity @Include(rootLevel = true) +@NonTransferable(enabled = false) public class ShareableWithPackageShare { private long id; private ContainerWithPackageShare container; diff --git a/elide-core/src/test/java/example/packageshareable/UnshareableWithEntityUnshare.java b/elide-core/src/test/java/example/nontransferable/Untransferable.java similarity index 85% rename from elide-core/src/test/java/example/packageshareable/UnshareableWithEntityUnshare.java rename to elide-core/src/test/java/example/nontransferable/Untransferable.java index 41a6a3a88a..67a1d4065a 100644 --- a/elide-core/src/test/java/example/packageshareable/UnshareableWithEntityUnshare.java +++ b/elide-core/src/test/java/example/nontransferable/Untransferable.java @@ -3,10 +3,9 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package example.packageshareable; +package example.nontransferable; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import javax.persistence.Entity; import javax.persistence.FetchType; @@ -19,9 +18,8 @@ * Entity level Unshareable bean. */ @Entity -@SharePermission(sharable = false) @Include(rootLevel = true) -public class UnshareableWithEntityUnshare { +public class Untransferable { private long id; private ContainerWithPackageShare container; diff --git a/elide-core/src/test/java/example/packageshareable/package-info.java b/elide-core/src/test/java/example/nontransferable/package-info.java similarity index 56% rename from elide-core/src/test/java/example/packageshareable/package-info.java rename to elide-core/src/test/java/example/nontransferable/package-info.java index d7eb1cd923..536228a162 100644 --- a/elide-core/src/test/java/example/packageshareable/package-info.java +++ b/elide-core/src/test/java/example/nontransferable/package-info.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -@SharePermission -package example.packageshareable; +@NonTransferable +package example.nontransferable; -import com.yahoo.elide.annotation.SharePermission; +import com.yahoo.elide.annotation.NonTransferable; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index eaeb3ab5fa..a1aa10587d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -33,8 +33,7 @@ * MetaDataStore is a in-memory data store that manage data models for an {@link AggregationDataStore}. */ public class MetaDataStore extends HashMapDataStore { - public static final Package META_DATA_PACKAGE = - Package.getPackage("com.yahoo.elide.datastores.aggregation.metadata.models"); + public static final Package META_DATA_PACKAGE = Table.class.getPackage(); private static final Class[] METADATA_STORE_ANNOTATIONS = { FromTable.class, FromSubquery.class, Subselect.class, javax.persistence.Table.class}; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java index 98e2a24565..d7786de7df 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java @@ -10,7 +10,6 @@ import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; - import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; import lombok.Builder; diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java index 92c7236116..e03e29b795 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java @@ -16,7 +16,6 @@ import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.core.hibernate.Query; import com.yahoo.elide.core.hibernate.Session; - import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; import org.apache.commons.lang3.StringUtils; diff --git a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java index 5ff85922d2..46efcda9db 100644 --- a/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java +++ b/elide-datastore/elide-datastore-hibernate/src/test/java/com/yahoo/elide/datastores/hibernate/hql/AbstractHQLQueryBuilderTest.java @@ -26,7 +26,6 @@ import com.yahoo.elide.core.hibernate.hql.AbstractHQLQueryBuilder; import com.yahoo.elide.core.pagination.PaginationImpl; import com.yahoo.elide.core.sort.SortingImpl; - import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; import example.Author; diff --git a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java index 02929bd216..a6d826fa53 100644 --- a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java +++ b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataTransaction.java @@ -22,7 +22,6 @@ import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.request.EntityProjection; - import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; import org.apache.lucene.search.Query; diff --git a/elide-example-models/src/main/java/com/yahoo/elide/models/generics/package-info.java b/elide-example-models/src/main/java/com/yahoo/elide/models/generics/package-info.java index 515df0479b..3f52252e11 100644 --- a/elide-example-models/src/main/java/com/yahoo/elide/models/generics/package-info.java +++ b/elide-example-models/src/main/java/com/yahoo/elide/models/generics/package-info.java @@ -4,7 +4,4 @@ * See LICENSE file in project root for terms. */ -@SharePermission package com.yahoo.elide.models.generics; - -import com.yahoo.elide.annotation.SharePermission; diff --git a/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/Comment.java b/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/Comment.java index b2af82a8c0..cf10356127 100644 --- a/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/Comment.java +++ b/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/Comment.java @@ -7,7 +7,6 @@ import com.yahoo.elide.annotation.ComputedAttribute; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import javax.persistence.Entity; import javax.persistence.GeneratedValue; @@ -23,7 +22,6 @@ @Entity @Table(name = "comment") @Include -@SharePermission public class Comment { private long id; private Post post; diff --git a/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/User.java b/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/User.java index 9f0c5be19c..2a0d5a8073 100644 --- a/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/User.java +++ b/elide-example/elide-blog-example-resteasy/src/main/java/com/yahoo/elide/example/models/User.java @@ -6,7 +6,6 @@ package com.yahoo.elide.example.models; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import javax.persistence.Entity; import javax.persistence.GeneratedValue; @@ -20,7 +19,6 @@ @Entity @Table(name = "user") @Include(rootLevel = true) -@SharePermission public class User { private long id; private String name; diff --git a/elide-example/elide-blog-example/src/main/java/com/yahoo/elide/example/models/User.java b/elide-example/elide-blog-example/src/main/java/com/yahoo/elide/example/models/User.java index 90f30eb99c..a1d5f71359 100644 --- a/elide-example/elide-blog-example/src/main/java/com/yahoo/elide/example/models/User.java +++ b/elide-example/elide-blog-example/src/main/java/com/yahoo/elide/example/models/User.java @@ -6,7 +6,6 @@ package com.yahoo.elide.example.models; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import javax.persistence.Entity; import javax.persistence.GeneratedValue; @@ -20,7 +19,6 @@ @Entity @Table(name = "blogger") @Include(rootLevel = true) -@SharePermission public class User { private long id; private String name; diff --git a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java index 1ba35f78b5..fd3a568136 100644 --- a/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java +++ b/elide-graphql/src/test/java/com/yahoo/elide/graphql/GraphQLEndpointTest.java @@ -46,7 +46,7 @@ import graphqlEndpointTestModels.Author; import graphqlEndpointTestModels.Book; -import graphqlEndpointTestModels.DisallowShare; +import graphqlEndpointTestModels.DisallowTransfer; import graphqlEndpointTestModels.security.CommitChecks; import graphqlEndpointTestModels.security.UserChecks; @@ -137,7 +137,7 @@ public Object apply(SecurityContext securityContext) { Book book1 = new Book(); Author author1 = new Author(); Author author2 = new Author(); - DisallowShare noShare = new DisallowShare(); + DisallowTransfer noShare = new DisallowTransfer(); book1.setId(1L); book1.setTitle("My first book"); @@ -400,7 +400,7 @@ void testFailedMutationAndRead() throws IOException, JSONException { @Test void testNonShareable() throws IOException, JSONException { - DisallowShare noShare = new DisallowShare(); + DisallowTransfer noShare = new DisallowTransfer(); noShare.setId(1L); Author author = new Author(); diff --git a/elide-graphql/src/test/java/example/Author.java b/elide-graphql/src/test/java/example/Author.java index 36f694b2c5..c12e57d9b6 100644 --- a/elide-graphql/src/test/java/example/Author.java +++ b/elide-graphql/src/test/java/example/Author.java @@ -8,7 +8,6 @@ import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.ComputedAttribute; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import lombok.Builder; @@ -41,7 +40,6 @@ @Entity @Table(name = "author") @Include(rootLevel = true) -@SharePermission @Audit(action = Audit.Action.CREATE, operation = 10, logStatement = "{0}", diff --git a/elide-graphql/src/test/java/example/Book.java b/elide-graphql/src/test/java/example/Book.java index 577101cfc7..a1c6660f99 100644 --- a/elide-graphql/src/test/java/example/Book.java +++ b/elide-graphql/src/test/java/example/Book.java @@ -19,7 +19,6 @@ import com.yahoo.elide.annotation.OnUpdatePostCommit; import com.yahoo.elide.annotation.OnUpdatePreCommit; import com.yahoo.elide.annotation.OnUpdatePreSecurity; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.security.RequestScope; import com.fasterxml.jackson.annotation.JsonIgnore; @@ -50,7 +49,6 @@ * runtime error at places such as {@code entityClass.newInstance();} */ @Entity -@SharePermission @Table(name = "book") @Include(rootLevel = true) @Audit(action = Audit.Action.CREATE, diff --git a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java index 75b21f4f19..d7ac6c34b0 100644 --- a/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java +++ b/elide-graphql/src/test/java/graphqlEndpointTestModels/Author.java @@ -10,7 +10,6 @@ import com.yahoo.elide.annotation.DeletePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import graphqlEndpointTestModels.security.UserChecks; @@ -34,13 +33,12 @@ @ReadPermission(expression = Author.PERMISSION) @UpdatePermission(expression = Author.PERMISSION) @DeletePermission(expression = Author.PERMISSION) -@SharePermission public class Author { - Long id; - String name; - Set books = new HashSet<>(); - DisallowShare noShare; - Map bookTitlesAndAwards = new HashMap<>(); + private Long id; + private String name; + private Set books = new HashSet<>(); + private DisallowTransfer noShare; + private Map bookTitlesAndAwards = new HashMap<>(); public static final String PERMISSION = UserChecks.IS_USER_1 + " OR " + UserChecks.IS_USER_2; @@ -82,11 +80,11 @@ public void setBookCount(int unused) { } @OneToOne - public DisallowShare getNoShare() { + public DisallowTransfer getNoShare() { return noShare; } - public void setNoShare(DisallowShare noShare) { + public void setNoShare(DisallowTransfer noShare) { this.noShare = noShare; } diff --git a/elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowShare.java b/elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowTransfer.java similarity index 85% rename from elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowShare.java rename to elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowTransfer.java index 58133459bd..cfffbb3aa5 100644 --- a/elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowShare.java +++ b/elide-graphql/src/test/java/graphqlEndpointTestModels/DisallowTransfer.java @@ -6,6 +6,7 @@ package graphqlEndpointTestModels; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import javax.persistence.Entity; import javax.persistence.GeneratedValue; @@ -14,7 +15,8 @@ @Include @Entity -public class DisallowShare { +@NonTransferable +public class DisallowTransfer { Long id; public void setId(Long id) { diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ShareableIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java similarity index 77% rename from elide-integration-tests/src/test/java/com/yahoo/elide/tests/ShareableIT.java rename to elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java index bd5dbd2893..6cb43aee61 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/ShareableIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java @@ -36,9 +36,10 @@ import java.util.HashMap; /** - * @Shareable annotation integration tests + * @NonTransferable annotation integration tests */ -class ShareableIT extends IntegrationTest { +class TransferableIT extends IntegrationTest { + private final ObjectMapper mapper = new ObjectMapper(); @BeforeEach public void setUp() throws Exception { @@ -51,7 +52,7 @@ public void setUp() throws Exception { } @Test - public void testUnshareableForbiddenAccess() { + public void testNonTransferableForbiddenAccess() { // Create container given() .contentType(JSONAPI_CONTENT_TYPE) @@ -67,54 +68,54 @@ public void testUnshareableForbiddenAccess() { .post("/container") .then().statusCode(HttpStatus.SC_CREATED); - // Create unshareable + // Create untransferable given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("unshareable"), + type("untransferable"), id(null) ) ) ) - .post("/unshareable") + .post("/untransferable") .then().statusCode(HttpStatus.SC_CREATED); - // Fail to add unshareable to container's unshareables (unshareable is not shareable) + // Fail to add untransferable to container's untransferables (untransferable is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("unshareable"), + type("untransferable"), id("1") ) ) ) - .patch("/container/1/relationships/unshareables") + .patch("/container/1/relationships/untransferables") .then() .statusCode(HttpStatus.SC_FORBIDDEN); - // Fail to replace container's unshareables collection (unshareable is not shareable) + // Fail to replace container's untransferables collection (untransferable is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("unshareable"), + type("untransferable"), id("1") ) ) ) - .post("/container/1/relationships/unshareables") + .post("/container/1/relationships/untransferables") .then() .statusCode(HttpStatus.SC_FORBIDDEN); - // Fail to update unshareable's container (container is not shareable) + // Fail to update untransferables's container (container is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) @@ -126,11 +127,11 @@ public void testUnshareableForbiddenAccess() { ) ) ) - .patch("/unshareable/1/relationships/container") + .patch("/untransferable/1/relationships/container") .then() .statusCode(HttpStatus.SC_FORBIDDEN); - // Fail to set unshareable's container (container is not shareable) + // Fail to set untransferable's container (container is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) @@ -142,13 +143,13 @@ public void testUnshareableForbiddenAccess() { ) ) ) - .post("/unshareable/1/relationships/container") + .post("/untransferable/1/relationships/container") .then() .statusCode(HttpStatus.SC_FORBIDDEN); } @Test - public void testShareableForbiddenAccess() { + public void testTransferableForbiddenAccess() { // Create container given() .contentType(JSONAPI_CONTENT_TYPE) @@ -165,23 +166,23 @@ public void testShareableForbiddenAccess() { .then() .statusCode(HttpStatus.SC_CREATED); - // Create shareable + // Create transferable given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("shareable"), + type("transferable"), id(null) ) ) ) - .post("/shareable") + .post("/transferable") .then() .statusCode(HttpStatus.SC_CREATED); - // Fail to update shareable's container (container is not shareable) + // Fail to update transferable's container (container is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) @@ -193,11 +194,11 @@ public void testShareableForbiddenAccess() { ) ) ) - .patch("/shareable/1/relationships/container") + .patch("/transferable/1/relationships/container") .then() .statusCode(HttpStatus.SC_FORBIDDEN); - // Fail to set shareable's container (container is not shareable) + // Fail to set transferable's container (container is not transferable) given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) @@ -209,13 +210,13 @@ public void testShareableForbiddenAccess() { ) ) ) - .post("/shareable/1/relationships/container") + .post("/transferable/1/relationships/container") .then() .statusCode(HttpStatus.SC_FORBIDDEN); } @Test - public void testShareablePost() { + public void testTransferablePost() { // Create container given() .contentType(JSONAPI_CONTENT_TYPE) @@ -232,35 +233,35 @@ public void testShareablePost() { .then() .statusCode(HttpStatus.SC_CREATED); - // Create shareable + // Create transferable given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("shareable"), + type("transferable"), id(null) ) ) ) - .post("/shareable") + .post("/transferable") .then() .statusCode(HttpStatus.SC_CREATED); - // Add shareable to container's shareables + // Add transferable to container's transferables given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body( datum( resource( - type("shareable"), + type("transferable"), id("1") ) ) ) - .post("/container/1/relationships/shareables") + .post("/container/1/relationships/transferables") .then() .statusCode(HttpStatus.SC_NO_CONTENT); @@ -274,17 +275,17 @@ public void testShareablePost() { type("container"), id("1"), relationships( - relation("shareables", - linkage(type("shareable"), id("1")) + relation("transferables", + linkage(type("transferable"), id("1")) ), - relation("unshareables") + relation("untransferables") ) )).toJSON()) ); } @Test - public void testShareablePatch() { + public void testTransferablePatch() { // Create container given() .contentType(JSONAPI_CONTENT_TYPE) @@ -298,30 +299,30 @@ public void testShareablePatch() { .post("/container") .then().statusCode(HttpStatus.SC_CREATED); - // Create shareable + // Create transferable given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body(datum( resource( - type("shareable"), + type("transferable"), id(null) ) )) - .post("/shareable") + .post("/transferable") .then().statusCode(HttpStatus.SC_CREATED); - // Add shareable to container's shareables + // Add transferable to container's transferables given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) .body(datum( resource( - type("shareable"), + type("transferable"), id("1") ) )) - .patch("/container/1/relationships/shareables") + .patch("/container/1/relationships/transferables") .then() .statusCode(HttpStatus.SC_NO_CONTENT); @@ -336,17 +337,17 @@ public void testShareablePatch() { type("container"), id("1"), relationships( - relation("shareables", - linkage(type("shareable"), id("1")) + relation("transferables", + linkage(type("transferable"), id("1")) ), - relation("unshareables") + relation("untransferables") ) )).toJSON()) ); } @Test - public void testCreateContainerAndUnshareables() throws Exception { + public void testCreateContainerAndNonTransferable() throws Exception { Response response = given() .contentType(JSONAPI_CONTENT_TYPE_WITH_JSON_PATCH_EXTENSION) .accept(JSONAPI_CONTENT_TYPE_WITH_JSON_PATCH_EXTENSION) @@ -358,14 +359,14 @@ public void testCreateContainerAndUnshareables() throws Exception { + " \"id\": \"12345678-1234-1234-1234-1234567890ab\",\n" + " \"type\": \"container\",\n" + " \"relationships\": {\n" - + " \"unshareables\": {\n" + + " \"untransferables\": {\n" + " \"data\": [\n" + " {\n" - + " \"type\": \"unshareable\",\n" + + " \"type\": \"untransferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ac\"\n" + " },\n" + " {\n" - + " \"type\": \"unshareable\",\n" + + " \"type\": \"untransferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ad\"\n" + " }\n" + " ]\n" @@ -375,17 +376,17 @@ public void testCreateContainerAndUnshareables() throws Exception { + " },\n" + " {\n" + " \"op\": \"add\",\n" - + " \"path\": \"/unshareable\",\n" + + " \"path\": \"/untransferable\",\n" + " \"value\": {\n" - + " \"type\": \"unshareable\",\n" + + " \"type\": \"untransferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ac\"\n" + " }\n" + " },\n" + " {\n" + " \"op\": \"add\",\n" - + " \"path\": \"/unshareable\",\n" + + " \"path\": \"/untransferable\",\n" + " \"value\": {\n" - + " \"type\": \"unshareable\",\n" + + " \"type\": \"untransferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ad\"\n" + " }\n" + " }\n" @@ -397,18 +398,18 @@ public void testCreateContainerAndUnshareables() throws Exception { ArrayNode patchJson = (ArrayNode) mapper.readTree(response.asString()); - // Should have 3 results, 1st is container, 2nd and 3rd are unshareables + // Should have 3 results, 1st is container, 2nd and 3rd are untransferables assertEquals(3, patchJson.size()); assertEquals("container", patchJson.get(0).get("data").get("type").asText()); - assertEquals("unshareable", patchJson.get(1).get("data").get("type").asText()); - assertEquals("unshareable", patchJson.get(2).get("data").get("type").asText()); + assertEquals("untransferable", patchJson.get(1).get("data").get("type").asText()); + assertEquals("untransferable", patchJson.get(2).get("data").get("type").asText()); - // Container should have 2 unshareables - assertEquals(2, patchJson.get(0).get("data").get("relationships").get("unshareables").get("data").size()); + // Container should have 2 untransferables + assertEquals(2, patchJson.get(0).get("data").get("relationships").get("untransferables").get("data").size()); } @Test - public void testCreateContainerAndShareables() throws Exception { + public void testCreateContainerAndTransferables() throws Exception { Response patchResponse = given() .contentType(JSONAPI_CONTENT_TYPE_WITH_JSON_PATCH_EXTENSION) .accept(JSONAPI_CONTENT_TYPE_WITH_JSON_PATCH_EXTENSION) @@ -420,14 +421,14 @@ public void testCreateContainerAndShareables() throws Exception { + " \"id\": \"12345678-1234-1234-1234-1234567890ab\",\n" + " \"type\": \"container\",\n" + " \"relationships\": {\n" - + " \"shareables\": {\n" + + " \"transferables\": {\n" + " \"data\": [\n" + " {\n" - + " \"type\": \"shareable\",\n" + + " \"type\": \"transferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ac\"\n" + " },\n" + " {\n" - + " \"type\": \"shareable\",\n" + + " \"type\": \"transferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ad\"\n" + " }\n" + " ]\n" @@ -437,17 +438,17 @@ public void testCreateContainerAndShareables() throws Exception { + " },\n" + " {\n" + " \"op\": \"add\",\n" - + " \"path\": \"/shareable\",\n" + + " \"path\": \"/transferable\",\n" + " \"value\": {\n" - + " \"type\": \"shareable\",\n" + + " \"type\": \"transferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ac\"\n" + " }\n" + " },\n" + " {\n" + " \"op\": \"add\",\n" - + " \"path\": \"/shareable\",\n" + + " \"path\": \"/transferable\",\n" + " \"value\": {\n" - + " \"type\": \"shareable\",\n" + + " \"type\": \"transferable\",\n" + " \"id\": \"12345678-1234-1234-1234-1234567890ad\"\n" + " }\n" + " }\n" @@ -459,18 +460,18 @@ public void testCreateContainerAndShareables() throws Exception { ArrayNode patchJson = (ArrayNode) mapper.readTree(patchResponse.asString()); - // Should have 3 results, 1st is container, 2nd and 3rd are shareables + // Should have 3 results, 1st is container, 2nd and 3rd are transferables assertEquals(3, patchJson.size()); assertEquals("container", patchJson.get(0).get("data").get("type").asText()); - assertEquals("shareable", patchJson.get(1).get("data").get("type").asText()); - assertEquals("shareable", patchJson.get(2).get("data").get("type").asText()); + assertEquals("transferable", patchJson.get(1).get("data").get("type").asText()); + assertEquals("transferable", patchJson.get(2).get("data").get("type").asText()); - // Container should have 2 shareables - assertEquals(2, patchJson.get(0).get("data").get("relationships").get("shareables").get("data").size()); + // Container should have 2 transferables + assertEquals(2, patchJson.get(0).get("data").get("relationships").get("transferables").get("data").size()); } @Test - public void addUnsharedRelationship() { + public void addNonTransferableRelationship() { given() .contentType(JSONAPI_CONTENT_TYPE) .accept(JSONAPI_CONTENT_TYPE) diff --git a/elide-integration-tests/src/test/java/example/AnotherFilterExpressionCheckObj.java b/elide-integration-tests/src/test/java/example/AnotherFilterExpressionCheckObj.java index 2f86361d3a..8148c6e68a 100644 --- a/elide-integration-tests/src/test/java/example/AnotherFilterExpressionCheckObj.java +++ b/elide-integration-tests/src/test/java/example/AnotherFilterExpressionCheckObj.java @@ -7,7 +7,6 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.Operator; @@ -27,7 +26,6 @@ * Model for anotherFilterExpressionCheckObj. */ @Entity -@SharePermission @Table(name = "anotherFilterExpressionCheckObj") @ReadPermission(expression = "checkActsLikeFilter") @Include(rootLevel = true) diff --git a/elide-integration-tests/src/test/java/example/AuditEntity.java b/elide-integration-tests/src/test/java/example/AuditEntity.java index 810b629b19..bd31cd5da3 100644 --- a/elide-integration-tests/src/test/java/example/AuditEntity.java +++ b/elide-integration-tests/src/test/java/example/AuditEntity.java @@ -10,7 +10,6 @@ import com.yahoo.elide.annotation.DeletePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import java.util.List; @@ -28,7 +27,6 @@ @CreatePermission(expression = "allow all") @DeletePermission(expression = "allow all") @UpdatePermission(expression = "allow all") -@SharePermission public class AuditEntity extends BaseId { private AuditEntity otherEntity; private String value; diff --git a/elide-integration-tests/src/test/java/example/AuditEntityInverse.java b/elide-integration-tests/src/test/java/example/AuditEntityInverse.java index 6dcd6a43c8..347748af0a 100644 --- a/elide-integration-tests/src/test/java/example/AuditEntityInverse.java +++ b/elide-integration-tests/src/test/java/example/AuditEntityInverse.java @@ -10,7 +10,6 @@ import com.yahoo.elide.annotation.DeletePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import java.util.List; @@ -24,7 +23,6 @@ @CreatePermission(expression = "allow all") @UpdatePermission(expression = "allow all") @DeletePermission(expression = "allow all") -@SharePermission public class AuditEntityInverse extends BaseId { private List entities; diff --git a/elide-integration-tests/src/test/java/example/Author.java b/elide-integration-tests/src/test/java/example/Author.java index c9e43a24f3..a0fcbee248 100644 --- a/elide-integration-tests/src/test/java/example/Author.java +++ b/elide-integration-tests/src/test/java/example/Author.java @@ -9,7 +9,6 @@ import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.Paginate; -import com.yahoo.elide.annotation.SharePermission; import lombok.Builder; import lombok.Getter; @@ -35,7 +34,6 @@ @Entity @Table(name = "author") @Include(rootLevel = true) -@SharePermission @Paginate @Audit(action = Audit.Action.CREATE, operation = 10, diff --git a/elide-integration-tests/src/test/java/example/Book.java b/elide-integration-tests/src/test/java/example/Book.java index 738b21eea7..eca5baf394 100644 --- a/elide-integration-tests/src/test/java/example/Book.java +++ b/elide-integration-tests/src/test/java/example/Book.java @@ -11,7 +11,6 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.Paginate; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import org.hibernate.annotations.Formula; @@ -34,7 +33,6 @@ * Model for books. */ @Entity -@SharePermission @Table(name = "book") @Include(rootLevel = true) @Paginate diff --git a/elide-integration-tests/src/test/java/example/Chapter.java b/elide-integration-tests/src/test/java/example/Chapter.java index eac9f51cc8..89155aaaa0 100644 --- a/elide-integration-tests/src/test/java/example/Chapter.java +++ b/elide-integration-tests/src/test/java/example/Chapter.java @@ -7,7 +7,6 @@ import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import lombok.Getter; import lombok.Setter; @@ -21,7 +20,6 @@ @Entity @Include(rootLevel = true, type = "chapter") -@SharePermission /** * This class tests using JPA Field based access. */ diff --git a/elide-integration-tests/src/test/java/example/Child.java b/elide-integration-tests/src/test/java/example/Child.java index 2ce4c9587c..5db9c2ae5c 100644 --- a/elide-integration-tests/src/test/java/example/Child.java +++ b/elide-integration-tests/src/test/java/example/Child.java @@ -10,7 +10,6 @@ import com.yahoo.elide.annotation.CreatePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.NotNullPredicate; @@ -35,7 +34,6 @@ */ @Entity(name = "childEntity") @CreatePermission(expression = "initCheck") -@SharePermission @ReadPermission(expression = "negativeChildId AND negativeIntegerUser AND initCheck AND initCheckFilter") @Include(rootLevel = true, type = "child") @Audit(action = Audit.Action.DELETE, diff --git a/elide-integration-tests/src/test/java/example/Container.java b/elide-integration-tests/src/test/java/example/Container.java index 57bd03670c..da437b9e9d 100644 --- a/elide-integration-tests/src/test/java/example/Container.java +++ b/elide-integration-tests/src/test/java/example/Container.java @@ -6,6 +6,7 @@ package example; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.NonTransferable; import java.util.Collection; @@ -19,25 +20,26 @@ @Entity @Table(name = "container") @Include(rootLevel = true, type = "container") +@NonTransferable public class Container extends BaseId { - private Collection unshareables; - private Collection shareables; + private Collection untransferables; + private Collection transferables; @OneToMany(mappedBy = "container") - public Collection getUnshareables() { - return unshareables; + public Collection getUntransferables() { + return untransferables; } - public void setUnshareables(Collection unshareables) { - this.unshareables = unshareables; + public void setUntransferables(Collection untransferables) { + this.untransferables = untransferables; } @OneToMany(mappedBy = "container") - public Collection getShareables() { - return shareables; + public Collection getTransferables() { + return transferables; } - public void setShareables(Collection shareables) { - this.shareables = shareables; + public void setTransferables(Collection transferables) { + this.transferables = transferables; } } diff --git a/elide-integration-tests/src/test/java/example/Editor.java b/elide-integration-tests/src/test/java/example/Editor.java index 486618230e..a6b58fa42c 100644 --- a/elide-integration-tests/src/test/java/example/Editor.java +++ b/elide-integration-tests/src/test/java/example/Editor.java @@ -12,7 +12,6 @@ import com.yahoo.elide.annotation.FilterExpressionPath; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.NotNullPredicate; @@ -37,7 +36,6 @@ @Entity @Table(name = "editor") @Include(rootLevel = true) -@SharePermission @Audit(action = Audit.Action.CREATE, operation = 10, logStatement = "{0}", diff --git a/elide-integration-tests/src/test/java/example/FilterExpressionCheckObj.java b/elide-integration-tests/src/test/java/example/FilterExpressionCheckObj.java index 90b274c56a..a0387fcc5d 100644 --- a/elide-integration-tests/src/test/java/example/FilterExpressionCheckObj.java +++ b/elide-integration-tests/src/test/java/example/FilterExpressionCheckObj.java @@ -7,7 +7,6 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.Operator; @@ -28,7 +27,6 @@ @Entity @Table(name = "filterExpressionCheckObj") @Include(rootLevel = true) -@SharePermission @ReadPermission(expression = "checkLE OR deny all") //ReadPermission for object id <= 2 public class FilterExpressionCheckObj extends BaseId { private String name; diff --git a/elide-integration-tests/src/test/java/example/MapColorShape.java b/elide-integration-tests/src/test/java/example/MapColorShape.java index 2bed0e2d7a..f40022b43d 100644 --- a/elide-integration-tests/src/test/java/example/MapColorShape.java +++ b/elide-integration-tests/src/test/java/example/MapColorShape.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import java.util.LinkedHashMap; import java.util.Map; @@ -23,7 +22,6 @@ @Entity @Table(name = "color_shape") @Include(rootLevel = true) -@SharePermission public class MapColorShape extends BaseId { private Map colorShapeMap = new LinkedHashMap<>(); diff --git a/elide-integration-tests/src/test/java/example/NoShareBiDirectional.java b/elide-integration-tests/src/test/java/example/NoTransferBiDirectional.java similarity index 62% rename from elide-integration-tests/src/test/java/example/NoShareBiDirectional.java rename to elide-integration-tests/src/test/java/example/NoTransferBiDirectional.java index 590f5fb720..5751efad72 100644 --- a/elide-integration-tests/src/test/java/example/NoShareBiDirectional.java +++ b/elide-integration-tests/src/test/java/example/NoTransferBiDirectional.java @@ -6,7 +6,7 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; +import com.yahoo.elide.annotation.NonTransferable; import javax.persistence.Entity; import javax.persistence.FetchType; @@ -14,16 +14,16 @@ @Entity @Include(rootLevel = true, type = "noShareBid") -@SharePermission(sharable = false) -public class NoShareBiDirectional extends BaseId { - private NoShareBiDirectional other; +@NonTransferable +public class NoTransferBiDirectional extends BaseId { + private NoTransferBiDirectional other; @OneToOne(fetch = FetchType.LAZY) - public NoShareBiDirectional getOther() { + public NoTransferBiDirectional getOther() { return other; } - public void setOther(NoShareBiDirectional other) { + public void setOther(NoTransferBiDirectional other) { this.other = other; } } diff --git a/elide-integration-tests/src/test/java/example/Parent.java b/elide-integration-tests/src/test/java/example/Parent.java index 18cfb108ea..03fa2d327f 100644 --- a/elide-integration-tests/src/test/java/example/Parent.java +++ b/elide-integration-tests/src/test/java/example/Parent.java @@ -10,7 +10,6 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.Paginate; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.RequestScope; @@ -35,7 +34,6 @@ @ReadPermission(expression = "parentInitCheck OR allow all") @UpdatePermission(expression = "parentInitCheck OR allow all OR deny all") @DeletePermission(expression = "parentInitCheck OR allow all OR deny all") -@SharePermission @Include(rootLevel = true, type = "parent") // optional here because class has this name @Paginate(maxLimit = 100000) // Hibernate diff --git a/elide-integration-tests/src/test/java/example/Property.java b/elide-integration-tests/src/test/java/example/Property.java index 454e3f6f18..ed13127af9 100644 --- a/elide-integration-tests/src/test/java/example/Property.java +++ b/elide-integration-tests/src/test/java/example/Property.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.ToOne; import org.hibernate.annotations.Any; @@ -21,7 +20,6 @@ @Entity @Include(rootLevel = true, type = "property") -@SharePermission public class Property extends BaseId { @Setter private Device myStuff; diff --git a/elide-integration-tests/src/test/java/example/Smartphone.java b/elide-integration-tests/src/test/java/example/Smartphone.java index 701fd9366a..984022438b 100644 --- a/elide-integration-tests/src/test/java/example/Smartphone.java +++ b/elide-integration-tests/src/test/java/example/Smartphone.java @@ -7,7 +7,6 @@ import com.yahoo.elide.annotation.ComputedAttribute; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import lombok.Getter; import lombok.Setter; @@ -17,7 +16,6 @@ @Entity @Include(rootLevel = true, type = "smartphone") -@SharePermission public class Smartphone extends BaseId implements Device { @Getter @Setter private String type; diff --git a/elide-integration-tests/src/test/java/example/Tractor.java b/elide-integration-tests/src/test/java/example/Tractor.java index d647dfd995..63b85f0795 100644 --- a/elide-integration-tests/src/test/java/example/Tractor.java +++ b/elide-integration-tests/src/test/java/example/Tractor.java @@ -6,7 +6,6 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; import lombok.Getter; import lombok.Setter; @@ -15,7 +14,6 @@ @Entity @Include(rootLevel = true, type = "tractor") -@SharePermission public class Tractor extends BaseId implements Device { @Getter @Setter private int horsepower; } diff --git a/elide-integration-tests/src/test/java/example/Unshareable.java b/elide-integration-tests/src/test/java/example/Transferable.java similarity index 80% rename from elide-integration-tests/src/test/java/example/Unshareable.java rename to elide-integration-tests/src/test/java/example/Transferable.java index a1fe8a7f23..b6af6e9804 100644 --- a/elide-integration-tests/src/test/java/example/Unshareable.java +++ b/elide-integration-tests/src/test/java/example/Transferable.java @@ -13,12 +13,12 @@ import javax.persistence.Table; /** - * DisallowShare bean. + * A shareable bean. */ @Entity -@Table(name = "unshareable") -@Include(rootLevel = true, type = "unshareable") -public class Unshareable extends BaseId { +@Table(name = "transferable") +@Include(rootLevel = true, type = "transferable") +public class Transferable extends BaseId { private Container container; @ManyToOne(fetch = FetchType.LAZY) diff --git a/elide-integration-tests/src/test/java/example/Shareable.java b/elide-integration-tests/src/test/java/example/Untransferable.java similarity index 72% rename from elide-integration-tests/src/test/java/example/Shareable.java rename to elide-integration-tests/src/test/java/example/Untransferable.java index 2bd1b8e0fa..bd68ad61b8 100644 --- a/elide-integration-tests/src/test/java/example/Shareable.java +++ b/elide-integration-tests/src/test/java/example/Untransferable.java @@ -6,7 +6,7 @@ package example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.SharePermission; +import com.yahoo.elide.annotation.NonTransferable; import javax.persistence.Entity; import javax.persistence.FetchType; @@ -14,13 +14,13 @@ import javax.persistence.Table; /** - * A shareable bean. + * DisallowShare bean. */ @Entity -@SharePermission -@Table(name = "shareable") -@Include(rootLevel = true, type = "shareable") -public class Shareable extends BaseId { +@Table(name = "untransferable") +@Include(rootLevel = true, type = "untransferable") +@NonTransferable +public class Untransferable extends BaseId { private Container container; @ManyToOne(fetch = FetchType.LAZY) From 72d5f6e0392f0616cbfbddb17b6fe29a7c0d9c77 Mon Sep 17 00:00:00 2001 From: Han Chen Date: Tue, 11 Feb 2020 15:26:28 -0600 Subject: [PATCH 05/16] metadata refactor (#1179) * metadata refactor * merge table and analyticView * fix reflection package * Make Table constrcut its own columns * table json type alias * add comment --- .../com/yahoo/elide/utils/ClassScanner.java | 9 +- .../aggregation/AggregationDataStore.java | 28 ++---- .../AggregationDataStoreTransaction.java | 8 +- .../EntityProjectionTranslator.java | 11 +-- .../datastores/aggregation/QueryEngine.java | 61 +++++++++++- .../aggregation/QueryEngineFactory.java | 15 --- .../aggregation/QueryValidator.java | 12 ++- .../aggregation/metadata/MetaDataStore.java | 50 ++++------ .../metadata/models/AnalyticView.java | 48 ---------- .../metadata/models/Dimension.java | 2 +- .../aggregation/metadata/models/Metric.java | 22 ++++- .../aggregation/metadata/models/Table.java | 93 +++++++++++++++---- .../datastores/aggregation/query/Query.java | 4 +- .../queryengines/AbstractEntityHydrator.java | 6 +- .../queryengines/sql/SQLQueryConstructor.java | 54 +++++------ .../queryengines/sql/SQLQueryEngine.java | 47 +++------- .../sql/SQLQueryEngineFactory.java | 30 ------ .../sql/metadata/SQLAnalyticView.java | 34 ------- .../queryengines/sql/metadata/SQLColumn.java | 35 +------ .../sql/metadata/SQLDimension.java | 42 +++++++++ .../queryengines/sql/metadata/SQLMetric.java | 32 +++++++ .../queryengines/sql/metadata/SQLTable.java | 40 ++++---- .../sql/metadata/SQLTimeDimension.java | 42 +++++++++ .../EntityProjectionTranslatorTest.java | 16 +--- .../aggregation/QueryValidatorTest.java | 16 ++-- .../AggregationDataStoreTestHarness.java | 18 ++-- .../aggregation/framework/SQLUnitTest.java | 10 +- .../AggregationDataStoreIntegrationTest.java | 10 +- .../queryengines/sql/QueryEngineTest.java | 46 ++++----- .../queryengines/sql/SubselectTest.java | 10 +- .../queryengines/sql/ViewTest.java | 24 ++--- .../spring/config/ElideAutoConfiguration.java | 42 ++++----- 32 files changed, 474 insertions(+), 443 deletions(-) delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java diff --git a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java index 3311c28377..ea1ff6c8b8 100644 --- a/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java +++ b/elide-core/src/main/java/com/yahoo/elide/utils/ClassScanner.java @@ -10,7 +10,9 @@ import io.github.classgraph.ScanResult; import java.lang.annotation.Annotation; +import java.util.Arrays; import java.util.HashSet; +import java.util.List; import java.util.Set; import java.util.stream.Collectors; @@ -50,7 +52,7 @@ static public Set> getAnnotatedClasses(String packageName, Class> getAnnotatedClasses(Class ...annotations) { + static public Set> getAnnotatedClasses(List> annotations) { Set> result = new HashSet<>(); try (ScanResult scanResult = new ClassGraph().enableAllInfo().scan()) { for (Class annotation : annotations) { @@ -62,6 +64,11 @@ static public Set> getAnnotatedClasses(Class ...a return result; } + @SafeVarargs + static public Set> getAnnotatedClasses(Class ...annotations) { + return getAnnotatedClasses(Arrays.asList(annotations)); + } + /** * Returns all classes within a package. * @param packageName The root package to search. diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java index b32a42dd14..b4c43fc6f6 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -9,36 +9,30 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import com.yahoo.elide.utils.ClassScanner; import java.lang.annotation.Annotation; +import java.util.Arrays; +import java.util.List; /** * DataStore that supports Aggregation. Uses {@link QueryEngine} to return results. */ public class AggregationDataStore implements DataStore { - - private final QueryEngineFactory queryEngineFactory; - - private final MetaDataStore metaDataStore; - private QueryEngine queryEngine; /** * These are the classes the Aggregation Store manages. */ - private static final Class[] AGGREGATION_STORE_CLASSES = { - FromTable.class, FromSubquery.class }; + private static final List> AGGREGATION_STORE_CLASSES = + Arrays.asList(FromTable.class, FromSubquery.class); - public AggregationDataStore(QueryEngineFactory queryEngineFactory, - MetaDataStore metaDataStore) { - this.queryEngineFactory = queryEngineFactory; - this.metaDataStore = metaDataStore; + public AggregationDataStore(QueryEngine queryEngine) { + this.queryEngine = queryEngine; } /** @@ -48,17 +42,15 @@ public AggregationDataStore(QueryEngineFactory queryEngineFactory, @Override public void populateEntityDictionary(EntityDictionary dictionary) { for (Class cls : AGGREGATION_STORE_CLASSES) { - // bind non-jpa entities, including analyticViews and views + // bind non-jpa entity tables ClassScanner.getAnnotatedClasses(cls).forEach(dictionary::bindEntity); } - queryEngine = queryEngineFactory.buildQueryEngine(metaDataStore); - /* Add 'grain' argument to each TimeDimensionColumn */ - for (AnalyticView table : metaDataStore.getMetaData(AnalyticView.class)) { + for (Table table : queryEngine.getMetaDataStore().getMetaData(Table.class)) { for (TimeDimension timeDim : table.getColumns(TimeDimension.class)) { dictionary.addArgumentToAttribute( - table.getCls(), + dictionary.getEntityClass(table.getName()), timeDim.getName(), new ArgumentType("grain", String.class)); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java index fd6daed82b..9ab4679f02 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java @@ -63,9 +63,11 @@ public void close() throws IOException { @VisibleForTesting private Query buildQuery(EntityProjection entityProjection, RequestScope scope) { - Table table = queryEngine.getTable(entityProjection.getType()); - EntityProjectionTranslator translator = new EntityProjectionTranslator(table, - entityProjection, scope.getDictionary()); + Table table = queryEngine.getTable(scope.getDictionary().getJsonAliasFor(entityProjection.getType())); + EntityProjectionTranslator translator = new EntityProjectionTranslator( + table, + entityProjection, + scope.getDictionary()); return translator.getQuery(); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java index 86ed3dfd7b..2cb62dad8c 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslator.java @@ -11,7 +11,6 @@ import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; @@ -37,7 +36,7 @@ * Helper for Aggregation Data Store which does the work associated with extracting {@link Query}. */ public class EntityProjectionTranslator { - private AnalyticView queriedTable; + private Table queriedTable; private EntityProjection entityProjection; private Set dimensionProjections; @@ -48,11 +47,7 @@ public class EntityProjectionTranslator { private EntityDictionary dictionary; public EntityProjectionTranslator(Table table, EntityProjection entityProjection, EntityDictionary dictionary) { - if (!(table instanceof AnalyticView)) { - throw new InvalidOperationException("Queried table is not analyticView: " + table.getName()); - } - - this.queriedTable = (AnalyticView) table; + this.queriedTable = table; this.entityProjection = entityProjection; this.dictionary = dictionary; dimensionProjections = resolveNonTimeDimensions(); @@ -67,7 +62,7 @@ public EntityProjectionTranslator(Table table, EntityProjection entityProjection */ public Query getQuery() { Query query = Query.builder() - .analyticView(queriedTable) + .table(queriedTable) .metrics(metrics) .groupByDimensions(dimensionProjections) .timeDimensions(timeDimensions) diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java index 74dc06a008..ff7a1a6b68 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -7,9 +7,18 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; +import com.google.common.base.Functions; + +import lombok.Getter; + +import java.util.Map; +import java.util.stream.Collectors; + /** * A {@link QueryEngine} is an abstraction that an AggregationDataStore leverages to run analytic queries (OLAP style) * against an underlying persistence layer. @@ -51,7 +60,49 @@ *

    * This is a {@link java.util.function functional interface} whose functional method is {@link #executeQuery(Query)}. */ -public interface QueryEngine { +public abstract class QueryEngine { + @Getter + private final MetaDataStore metaDataStore; + + @Getter + private final EntityDictionary metadataDictionary; + + @Getter + private final Map tables; + + /** + * QueryEngine is constructed with a metadata store and is responsible for constructing all Tables and Entities + * metadata in this metadata store. + * + * @param metaDataStore a metadata store + */ + public QueryEngine(MetaDataStore metaDataStore) { + this.metaDataStore = metaDataStore; + this.metadataDictionary = metaDataStore.getDictionary(); + populateMetaData(metaDataStore); + this.tables = metaDataStore.getMetaData(Table.class).stream() + .collect(Collectors.toMap(Table::getName, Functions.identity())); + } + + /** + * Construct Table metadata for an entity. + * + * @param entityClass entity class + * @param metaDataDictionary metadata dictionary + * @return constructed Table + */ + protected abstract Table constructTable(Class entityClass, EntityDictionary metaDataDictionary); + + /** + * Query is responsible for constructing all Tables and Entities metadata in this metadata store. + * + * @param metaDataStore metadata store to populate + */ + private void populateMetaData(MetaDataStore metaDataStore) { + metaDataStore.getModelsToBind().stream() + .map(model -> constructTable(model, metadataDictionary)) + .forEach(metaDataStore::addTable); + } /** * Executes the specified {@link Query} against a specific persistent storage, which understand the provided @@ -61,12 +112,14 @@ public interface QueryEngine { * * @return query results */ - Iterable executeQuery(Query query); + public abstract Iterable executeQuery(Query query); /** * Returns the schema for a given entity class. - * @param entityClass The class to map to a schema. + * @param classAlias json type alias for that class * @return The schema that represents the provided entity. */ - Table getTable(Class entityClass); + public Table getTable(String classAlias) { + return tables.get(classAlias); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java deleted file mode 100644 index 7a77dfdf6d..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngineFactory.java +++ /dev/null @@ -1,15 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation; - -import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; - -/** - * Interface that constructs {@link QueryEngine} based on given entityDictionary. - */ -public interface QueryEngineFactory { - QueryEngine buildQueryEngine(MetaDataStore metaDataStore); -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java index 9a3dc6b960..678d7e306a 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java @@ -14,7 +14,7 @@ import com.yahoo.elide.core.filter.expression.NotFilterExpression; import com.yahoo.elide.core.filter.expression.OrFilterExpression; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.request.Sorting; @@ -31,7 +31,7 @@ public class QueryValidator { private Query query; private Set allFields; private EntityDictionary dictionary; - private AnalyticView queriedTable; + private Table queriedTable; private List metrics; private Set dimensionProjections; @@ -39,7 +39,7 @@ public QueryValidator(Query query, Set allFields, EntityDictionary dicti this.query = query; this.allFields = allFields; this.dictionary = dictionary; - this.queriedTable = query.getAnalyticView(); + this.queriedTable = query.getTable(); this.metrics = query.getMetrics(); this.dimensionProjections = query.getDimensions(); } @@ -71,12 +71,14 @@ private void validateHavingClause(FilterExpression havingClause) { Class cls = last.getType(); String fieldName = last.getFieldName(); - if (cls != queriedTable.getCls()) { + Class tableClass = dictionary.getEntityClass(queriedTable.getName()); + + if (cls != tableClass) { throw new InvalidOperationException( String.format( "Can't filter on relationship field %s in HAVING clause when querying table %s.", path.toString(), - queriedTable.getCls().getSimpleName())); + tableClass.getSimpleName())); } if (queriedTable.isMetric(fieldName)) { diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index a1aa10587d..fe0e886b99 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -10,7 +10,6 @@ import com.yahoo.elide.core.exceptions.DuplicateMappingException; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.metadata.models.Column; import com.yahoo.elide.datastores.aggregation.metadata.models.DataType; import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; @@ -25,7 +24,12 @@ import org.hibernate.annotations.Subselect; +import lombok.Getter; + +import java.lang.annotation.Annotation; +import java.util.Arrays; import java.util.HashMap; +import java.util.List; import java.util.Set; import java.util.stream.Collectors; @@ -33,38 +37,30 @@ * MetaDataStore is a in-memory data store that manage data models for an {@link AggregationDataStore}. */ public class MetaDataStore extends HashMapDataStore { - public static final Package META_DATA_PACKAGE = Table.class.getPackage(); + private static final Package META_DATA_PACKAGE = Table.class.getPackage(); + + private static final List> METADATA_STORE_ANNOTATIONS = + Arrays.asList(FromTable.class, FromSubquery.class, Subselect.class, javax.persistence.Table.class); - private static final Class[] METADATA_STORE_ANNOTATIONS = { - FromTable.class, FromSubquery.class, Subselect.class, javax.persistence.Table.class}; + @Getter + private final Set> modelsToBind; public MetaDataStore() { super(META_DATA_PACKAGE); this.dictionary = new EntityDictionary(new HashMap<>()); - ClassScanner.getAllClasses(Table.class.getPackage().getName()).forEach(cls -> dictionary.bindEntity(cls)); - - Set> modelsToBind = ClassScanner.getAnnotatedClasses(METADATA_STORE_ANNOTATIONS); + // bind meta data models to dictionary + ClassScanner.getAllClasses(Table.class.getPackage().getName()).forEach(dictionary::bindEntity); - // bind data models in the package - modelsToBind.forEach(modelClass -> { - dictionary.bindEntity(modelClass); - }); - - // resolve meta data from the bound models - modelsToBind.forEach(modelClass -> { - addTable(isAnalyticView(modelClass) - ? new AnalyticView(modelClass, dictionary) - : new Table(modelClass, dictionary)); - }); + // bind external data models in the package + this.modelsToBind = ClassScanner.getAnnotatedClasses(METADATA_STORE_ANNOTATIONS); + modelsToBind.forEach(dictionary::bindEntity); } @Override public void populateEntityDictionary(EntityDictionary dictionary) { - ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()).stream().forEach(cls -> { - dictionary.bindEntity(cls); - }); + ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()).forEach(dictionary::bindEntity); } /** @@ -72,7 +68,7 @@ public void populateEntityDictionary(EntityDictionary dictionary) { * * @param table table metadata */ - private void addTable(Table table) { + public void addTable(Table table) { addMetaData(table); table.getColumns().forEach(this::addColumn); } @@ -169,14 +165,4 @@ public Set getMetaData(Class cls) { public static boolean isMetricField(EntityDictionary dictionary, Class cls, String fieldName) { return dictionary.attributeOrRelationAnnotationExists(cls, fieldName, MetricAggregation.class); } - - /** - * Returns whether an entity class is analytic view. - * - * @param cls entity class - * @return True if {@link FromTable} or {@link FromSubquery} is presented. - */ - private static boolean isAnalyticView(Class cls) { - return cls.isAnnotationPresent(FromTable.class) || cls.isAnnotationPresent(FromSubquery.class); - } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java deleted file mode 100644 index 65b03048bb..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/AnalyticView.java +++ /dev/null @@ -1,48 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.models; - -import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.core.EntityDictionary; - -import lombok.Data; -import lombok.EqualsAndHashCode; -import lombok.ToString; - -import java.util.Set; -import java.util.stream.Collectors; -import javax.persistence.OneToMany; - -/** - * AnalyticViews are logical tables that support aggregation functionality, but don't support join or relationship. - */ -@EqualsAndHashCode(callSuper = true) -@Include(rootLevel = true, type = "analyticView") -@Data -public class AnalyticView extends Table { - - @OneToMany - @ToString.Exclude - private Set metrics; - - @OneToMany - @ToString.Exclude - private Set dimensions; - - public AnalyticView(Class cls, EntityDictionary dictionary) { - super(cls, dictionary); - - metrics = getColumns().stream() - .filter(col -> col instanceof Metric) - .map(Metric.class::cast) - .collect(Collectors.toSet()); - - dimensions = getColumns().stream() - .filter(col -> !(col instanceof Metric)) - .map(Dimension.class::cast) - .collect(Collectors.toSet()); - } -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java index 3ed5f7f56a..1eabdd3674 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java @@ -12,7 +12,7 @@ import lombok.EqualsAndHashCode; /** - * Regular field in tables, can be grouped by if the table is an AnalyticView. + * Regular field in tables, can be grouped by. */ @EqualsAndHashCode(callSuper = true) @Include(type = "dimension") diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java index 8dcfaf3d16..26f2fd9831 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java @@ -7,6 +7,7 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.metadata.enums.Format; @@ -14,10 +15,11 @@ import lombok.EqualsAndHashCode; import lombok.ToString; +import java.util.Set; import javax.persistence.ManyToOne; /** - * Special column for AnalyticView which supports aggregation. + * Column which supports aggregation. */ @EqualsAndHashCode(callSuper = true) @Include(type = "metric") @@ -37,14 +39,32 @@ public Metric(Class tableClass, String fieldName, EntityDictionary dictionary MetricAggregation.class, fieldName); + Meta meta = dictionary.getAttributeOrRelationAnnotation( + tableClass, + Meta.class, + fieldName); + try { this.metricFunction = metric.function().newInstance(); metricFunction.setName(getId() + "[" + metricFunction.getName() + "]"); metricFunction.setExpression(String.format( metricFunction.getExpression(), dictionary.getAnnotatedColumnName(tableClass, fieldName))); + + if (meta != null) { + metricFunction.setLongName(meta.longName()); + metricFunction.setDescription(meta.description()); + } } catch (InstantiationException | IllegalAccessException e) { throw new IllegalArgumentException("Can't initialize function for metric " + getId()); } } + + protected MetricFunction constructMetricFunction(String id, + String longName, + String description, + String expression, + Set arguments) { + return new MetricFunction(id, longName, description, expression, arguments); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index b31af3e474..8d36e694c4 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -7,6 +7,7 @@ import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; +import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; @@ -17,11 +18,12 @@ import lombok.Data; import lombok.ToString; +import java.util.Map; import java.util.Set; +import java.util.function.Function; import java.util.stream.Collectors; import javax.persistence.Id; import javax.persistence.OneToMany; -import javax.persistence.Transient; /** * Super class of all logical or physical tables. @@ -30,9 +32,6 @@ @Data @ToString public class Table { - @Transient - private Class cls; - @Id private String name; @@ -48,16 +47,37 @@ public class Table { @ToString.Exclude private Set columns; + @OneToMany + @ToString.Exclude + private Set metrics; + + @OneToMany + @ToString.Exclude + private Set dimensions; + + @Exclude + @ToString.Exclude + private final Map columnMap; + public Table(Class cls, EntityDictionary dictionary) { if (!dictionary.getBindings().contains(cls)) { throw new IllegalArgumentException( String.format("Table class {%s} is not defined in dictionary.", cls)); } - this.cls = cls; this.name = dictionary.getJsonAliasFor(cls); - this.columns = resolveColumns(cls, dictionary); + this.columns = constructColumns(cls, dictionary); + this.columnMap = this.columns.stream().collect(Collectors.toMap(Column::getName, Function.identity())); + + this.metrics = this.columns.stream() + .filter(col -> col instanceof Metric) + .map(Metric.class::cast) + .collect(Collectors.toSet()); + this.dimensions = this.columns.stream() + .filter(col -> !(col instanceof Metric)) + .map(Dimension.class::cast) + .collect(Collectors.toSet()); Meta meta = cls.getAnnotation(Meta.class); if (meta != null) { @@ -72,32 +92,68 @@ public Table(Class cls, EntityDictionary dictionary) { } /** - * Add all columns of this table. + * Construct all columns of this table. * * @param cls table class * @param dictionary dictionary contains the table class * @return all resolved column metadata */ - private static Set resolveColumns(Class cls, EntityDictionary dictionary) { - Set fields = dictionary.getAllFields(cls).stream() + private Set constructColumns(Class cls, EntityDictionary dictionary) { + Set columns = dictionary.getAllFields(cls).stream() .filter((field) -> Column.getDataType(cls, field, dictionary) != null) .map(field -> { if (isMetricField(dictionary, cls, field)) { - return new Metric(cls, field, dictionary); + return constructMetric(cls, field, dictionary); } else if (dictionary.attributeOrRelationAnnotationExists(cls, field, Temporal.class)) { - return new TimeDimension(cls, field, dictionary); + return constructTimeDimension(cls, field, dictionary); } else { - return new Dimension(cls, field, dictionary); + return constructDimension(cls, field, dictionary); } }) .collect(Collectors.toSet()); // add id field if exists if (dictionary.getIdFieldName(cls) != null) { - fields.add(new Dimension(cls, dictionary.getIdFieldName(cls), dictionary)); + columns.add(constructDimension(cls, dictionary.getIdFieldName(cls), dictionary)); } - return fields; + return columns; + } + + /** + * Construct a Metric instance. + * + * @param cls table class + * @param fieldName field name + * @param dictionary dictionary contains the table class + * @return Metric metadata instance + */ + protected Metric constructMetric(Class cls, String fieldName, EntityDictionary dictionary) { + return new Metric(cls, fieldName, dictionary); + } + + /** + * Construct a Dimension instance. + * + * @param cls table class + * @param fieldName field name + * @param dictionary dictionary contains the table class + * @return Dimension metadata instance + */ + protected TimeDimension constructTimeDimension(Class cls, String fieldName, EntityDictionary dictionary) { + return new TimeDimension(cls, fieldName, dictionary); + } + + /** + * Construct a TimeDimension instance. + * + * @param cls table class + * @param fieldName field name + * @param dictionary dictionary contains the table class + * @return TimeDimension metadata instance + */ + protected Dimension constructDimension(Class cls, String fieldName, EntityDictionary dictionary) { + return new Dimension(cls, fieldName, dictionary); } /** @@ -122,12 +178,9 @@ public Set getColumns(Class cls) { * @param metadata class * @return column as requested type if found */ - private T getColumn(Class cls, String fieldName) { - return columns.stream() - .filter(col -> cls.isAssignableFrom(col.getClass()) && (col.getName().equals(fieldName))) - .map(cls::cast) - .findFirst() - .orElse(null); + protected final T getColumn(Class cls, String fieldName) { + Column column = columnMap.get(fieldName); + return column != null && cls.isAssignableFrom(column.getClass()) ? cls.cast(column) : null; } public Metric getMetric(String fieldName) { diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java index d7786de7df..0d2f9367d7 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/Query.java @@ -9,7 +9,7 @@ import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; import lombok.Builder; @@ -28,7 +28,7 @@ @Data @Builder public class Query { - private final AnalyticView analyticView; + private final Table table; @Singular private final List metrics; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java index b8e2151a1c..caec8cd31e 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java @@ -151,7 +151,7 @@ public Iterable hydrate() { * @return A hydrated entity object. */ protected Object coerceObjectToEntity(Map result, MutableInt counter) { - Class entityClass = query.getAnalyticView().getCls(); + Class entityClass = entityDictionary.getEntityClass(query.getTable().getName()); //Construct the object. Object entityInstance; @@ -162,7 +162,7 @@ protected Object coerceObjectToEntity(Map result, MutableInt cou } result.forEach((fieldName, value) -> { - Dimension dim = query.getAnalyticView().getDimension(fieldName); + Dimension dim = query.getTable().getDimension(fieldName); if (dim != null && dim.getDataType() instanceof RelationshipType) { getStitchList().todo(entityInstance, fieldName, value); // We don't hydrate relationships here. @@ -194,7 +194,7 @@ private void populateObjectLookupTable() { String joinField = entry.getKey(); List joinFieldIds = entry.getValue(); Class relationshipType = getEntityDictionary().getParameterizedType( - getQuery().getAnalyticView().getCls(), + entityDictionary.getEntityClass(getQuery().getTable().getName()), joinField); getStitchList().populateLookup(relationshipType, getRelationshipValues(relationshipType, joinFieldIds)); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java index a90d708e5c..92201fe476 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -27,11 +27,11 @@ import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQueryTemplate; - import com.yahoo.elide.request.Sorting; + import org.hibernate.annotations.Subselect; import java.util.Collection; @@ -69,8 +69,8 @@ public SQLQuery resolveTemplate(Query clientQuery, Sorting sorting, FilterExpression whereClause, FilterExpression havingClause) { - SQLAnalyticView queriedTable = (SQLAnalyticView) clientQuery.getAnalyticView(); - Class tableCls = clientQuery.getAnalyticView().getCls(); + SQLTable table = (SQLTable) clientQuery.getTable(); + Class tableCls = dictionary.getEntityClass(clientQuery.getTable().getName()); String tableAlias = getClassAlias(tableCls); SQLQuery.SQLQueryBuilder builder = SQLQuery.builder().clientQuery(clientQuery); @@ -81,20 +81,20 @@ public SQLQuery resolveTemplate(Query clientQuery, ? "(" + tableCls.getAnnotation(FromSubquery.class).sql() + ")" : tableCls.isAnnotationPresent(FromTable.class) ? tableCls.getAnnotation(FromTable.class).name() - : queriedTable.getName(); + : table.getName(); builder.fromClause(String.format("%s AS %s", tableStatement, tableAlias)); - builder.projectionClause(constructProjectionWithReference(template, queriedTable)); + builder.projectionClause(constructProjectionWithReference(template, table)); Set groupByDimensions = template.getGroupByDimensions(); if (!groupByDimensions.isEmpty()) { if (!clientQuery.getMetrics().isEmpty()) { - builder.groupByClause(constructGroupByWithReference(groupByDimensions, queriedTable)); + builder.groupByClause(constructGroupByWithReference(groupByDimensions, table)); } - joinPaths.addAll(extractJoinPaths(groupByDimensions, queriedTable)); + joinPaths.addAll(extractJoinPaths(groupByDimensions, table)); } if (whereClause != null) { @@ -106,7 +106,7 @@ public SQLQuery resolveTemplate(Query clientQuery, if (havingClause != null) { builder.havingClause("HAVING " + translateFilterExpression( havingClause, - (predicate) -> constructHavingClauseWithReference(predicate, queriedTable, template))); + (predicate) -> constructHavingClauseWithReference(predicate, table, template))); joinPaths.addAll(extractJoinPaths(havingClause)); } @@ -127,13 +127,13 @@ public SQLQuery resolveTemplate(Query clientQuery, * Construct directly projection GROUP BY clause using column reference. * * @param groupByDimensions columns to project out - * @param queriedTable queried analytic view + * @param table queried table * @return GROUP BY tb1.col1, tb2.col2, ... */ private String constructGroupByWithReference(Set groupByDimensions, - SQLAnalyticView queriedTable) { + SQLTable table) { return "GROUP BY " + groupByDimensions.stream() - .map(dimension -> resolveSQLColumnReference(dimension, queriedTable)) + .map(dimension -> resolveSQLColumnReference(dimension, table)) .collect(Collectors.joining(", ")); } @@ -152,7 +152,7 @@ private String constructHavingClauseWithReference(FilterPredicate predicate, Class lastClass = last.getType(); String fieldName = last.getFieldName(); - if (!lastClass.equals(table.getCls())) { + if (!lastClass.equals(dictionary.getEntityClass(table.getName()))) { throw new InvalidPredicateException("The having clause can only reference fact table aggregations."); } @@ -174,16 +174,16 @@ private String constructHavingClauseWithReference(FilterPredicate predicate, * references. * * @param template query template with nested subquery - * @param queriedTable queried analytic view + * @param table queried table * @return SELECT function(metric1) AS alias1, tb1.dimension1 AS alias2 */ - private String constructProjectionWithReference(SQLQueryTemplate template, SQLAnalyticView queriedTable) { + private String constructProjectionWithReference(SQLQueryTemplate template, SQLTable table) { // TODO: project metric field using table column reference List metricProjections = template.getMetrics().stream() .map(invocation -> invocation.getFunctionExpression() + " AS " + invocation.getAlias()) .collect(Collectors.toList()); - Class tableClass = queriedTable.getCls(); + Class tableClass = dictionary.getEntityClass(table.getName()); List dimensionProjections = template.getGroupByDimensions().stream() .map(dimension -> { @@ -198,7 +198,7 @@ private String constructProjectionWithReference(SQLQueryTemplate template, SQLAn } } - return resolveSQLColumnReference(dimension, queriedTable) + " AS " + dimension.getAlias(); + return resolveSQLColumnReference(dimension, table) + " AS " + dimension.getAlias(); }) .collect(Collectors.toList()); @@ -398,15 +398,15 @@ private Set extractJoinPaths(Map sortClauses) { /** * Given the set of group by dimensions, extract any entity relationship traversals that require joins. - * This method takes in a {@link SQLAnalyticView} because the sql join path meta data is stored in it. + * This method takes in a {@link SQLTable} because the sql join path meta data is stored in it. * * @param groupByDimensions The list of dimensions we are grouping on. - * @param queriedTable queried analytic view + * @param table queried table * @return A set of path elements that capture a relationship traversal. */ private Set extractJoinPaths(Set groupByDimensions, - SQLAnalyticView queriedTable) { - return resolveSQLColumns(groupByDimensions, queriedTable).stream() + SQLTable table) { + return resolveSQLColumns(groupByDimensions, table).stream() .filter((dim) -> dim.getJoinPath() != null) .map(SQLColumn::getJoinPath) .collect(Collectors.toCollection(LinkedHashSet::new)); @@ -440,12 +440,12 @@ private String generatePredicateReference(FilterPredicate predicate) { * Resolve all projected sql column from a queried table. * * @param columnProjections projections - * @param queriedTable sql analytic view + * @param table sql table * @return projected columns */ - private Set resolveSQLColumns(Set columnProjections, SQLAnalyticView queriedTable) { + private Set resolveSQLColumns(Set columnProjections, SQLTable table) { return columnProjections.stream() - .map(colProjection -> queriedTable.getColumn(colProjection.getColumn().getName())) + .map(colProjection -> table.getSQLColumn(colProjection.getColumn().getName())) .collect(Collectors.toSet()); } @@ -454,11 +454,11 @@ private Set resolveSQLColumns(Set columnProjections * If the projection is {@link TimeDimensionProjection}, the correct time grain expression would be used. * * @param columnProjection projection - * @param queriedTable sql analytic view + * @param table sql table * @return projected columns */ - private String resolveSQLColumnReference(ColumnProjection columnProjection, SQLAnalyticView queriedTable) { - SQLColumn sqlColumn = queriedTable.getColumn(columnProjection.getColumn().getName()); + private String resolveSQLColumnReference(ColumnProjection columnProjection, SQLTable table) { + SQLColumn sqlColumn = table.getSQLColumn(columnProjection.getColumn().getName()); if (columnProjection instanceof TimeDimensionProjection) { TimeDimension timeDimension = ((TimeDimensionProjection) columnProjection).getTimeDimension(); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java index ab9c9b0176..b342ef905d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -16,14 +16,12 @@ import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLColumn; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; @@ -33,7 +31,6 @@ import org.hibernate.jpa.QueryHints; -import lombok.Getter; import lombok.extern.slf4j.Slf4j; import java.util.ArrayList; @@ -41,9 +38,7 @@ import java.util.Collections; import java.util.LinkedHashSet; import java.util.List; -import java.util.Map; import java.util.Set; -import java.util.function.Function; import java.util.stream.Collectors; import javax.persistence.EntityManager; import javax.persistence.EntityManagerFactory; @@ -53,31 +48,17 @@ * QueryEngine for SQL backed stores. */ @Slf4j -public class SQLQueryEngine implements QueryEngine { +public class SQLQueryEngine extends QueryEngine { + private final EntityManagerFactory entityManagerFactory; - private final EntityManagerFactory emf; - private final EntityDictionary metadataDictionary; - - @Getter - private Map, Table> tables; - - public SQLQueryEngine(EntityManagerFactory emf, MetaDataStore metaDataStore) { - this.emf = emf; - this.metadataDictionary = metaDataStore.getDictionary(); - - Set

    • tables = metaDataStore.getMetaData(Table.class); - tables.addAll(metaDataStore.getMetaData(AnalyticView.class)); - - this.tables = tables.stream() - .map(table -> table instanceof AnalyticView - ? new SQLAnalyticView(table.getCls(), metadataDictionary) - : new SQLTable(table.getCls(), metadataDictionary)) - .collect(Collectors.toMap(Table::getCls, Function.identity())); + public SQLQueryEngine(MetaDataStore metaDataStore, EntityManagerFactory entityManagerFactory) { + super(metaDataStore); + this.entityManagerFactory = entityManagerFactory; } @Override - public Table getTable(Class entityClass) { - return tables.get(entityClass); + protected Table constructTable(Class entityClass, EntityDictionary metaDataDictionary) { + return new SQLTable(entityClass, metaDataDictionary); } @Override @@ -85,7 +66,7 @@ public Iterable executeQuery(Query query) { EntityManager entityManager = null; EntityTransaction transaction = null; try { - entityManager = emf.createEntityManager(); + entityManager = entityManagerFactory.createEntityManager(); // manually begin the transaction transaction = entityManager.getTransaction(); @@ -132,7 +113,7 @@ public Iterable executeQuery(Query query) { () -> jpaQuery.setHint(QueryHints.HINT_READONLY, true).getResultList(), "Running Query: " + sql).get(); - return new SQLEntityHydrator(results, query, metadataDictionary, entityManager).hydrate(); + return new SQLEntityHydrator(results, query, getMetadataDictionary(), entityManager).hydrate(); } finally { if (transaction != null && transaction.isActive()) { transaction.commit(); @@ -188,7 +169,7 @@ public TimeDimensionProjection getTimeDimension() { } }); - return new SQLQueryConstructor(metadataDictionary).resolveTemplate( + return new SQLQueryConstructor(getMetadataDictionary()).resolveTemplate( query, queryTemplate, query.getSorting(), @@ -235,7 +216,7 @@ private void supplyFilterQueryParameters(Query query, private SQLQuery toPageTotalSQL(SQLQuery sql) { // TODO: refactor this method String groupByDimensions = - extractSQLDimensions(sql.getClientQuery(), (SQLAnalyticView) sql.getClientQuery().getAnalyticView()) + extractSQLDimensions(sql.getClientQuery(), (SQLTable) sql.getClientQuery().getTable()) .stream() .map(SQLColumn::getReference) .collect(Collectors.joining(", ")); @@ -255,12 +236,12 @@ private SQLQuery toPageTotalSQL(SQLQuery sql) { * Extract dimension projects in a query to sql dimensions. * * @param query requested query - * @param queriedTable queried analytic view + * @param table queried table * @return sql dimensions in this query */ - private List extractSQLDimensions(Query query, SQLAnalyticView queriedTable) { + private List extractSQLDimensions(Query query, SQLTable table) { return query.getDimensions().stream() - .map(projection -> queriedTable.getColumn(projection.getColumn().getName())) + .map(projection -> table.getSQLColumn(projection.getColumn().getName())) .collect(Collectors.toList()); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java deleted file mode 100644 index c7834b0bc3..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngineFactory.java +++ /dev/null @@ -1,30 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.queryengines.sql; - -import com.yahoo.elide.datastores.aggregation.QueryEngine; -import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; -import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; -import lombok.Getter; - -import javax.persistence.EntityManagerFactory; - -/** - * Object that constructs {@link QueryEngine} based on given entityDictionary and entityManagerFactory. - */ -public class SQLQueryEngineFactory implements QueryEngineFactory { - @Getter - private EntityManagerFactory emf; - - public SQLQueryEngineFactory(EntityManagerFactory emf) { - this.emf = emf; - } - - @Override - public QueryEngine buildQueryEngine(MetaDataStore metaDataStore) { - return new SQLQueryEngine(emf, metaDataStore); - } -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java deleted file mode 100644 index 89d5edfbad..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLAnalyticView.java +++ /dev/null @@ -1,34 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; - -import static com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable.resolveSQLDimensions; - -import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; - -import lombok.Data; -import lombok.EqualsAndHashCode; - -import java.util.Map; - -/** - * SQL extension of {@link AnalyticView} which also contains sql column meta data. - */ -@EqualsAndHashCode(callSuper = true) -@Data -public class SQLAnalyticView extends AnalyticView { - private Map sqlColumns; - - public SQLAnalyticView(Class cls, EntityDictionary dictionary) { - super(cls, dictionary); - this.sqlColumns = resolveSQLDimensions(cls, dictionary); - } - - public SQLColumn getColumn(String fieldName) { - return sqlColumns.get(fieldName); - } -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java index 2eac1ea9a2..e0e7df088d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java @@ -1,42 +1,17 @@ /* - * Copyright 2019, Yahoo Inc. + * Copyright 2020, Yahoo Inc. * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; -import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; -import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; - -import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; -import com.yahoo.elide.datastores.aggregation.metadata.models.Column; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; - -import lombok.Getter; /** - * SQLColumn contains meta data about underlying physical table. + * Column with physical SQL information like reference and join to path. */ -public class SQLColumn extends Column { - @Getter - private final String reference; - - @Getter - private final Path joinPath; - - protected SQLColumn(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); - - JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); +public interface SQLColumn { + String getReference(); - if (joinTo == null || joinTo.path().equals("")) { - this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); - this.joinPath = null; - } else { - Path path = new Path(tableClass, dictionary, joinTo.path()); - this.reference = generateColumnReference(path, dictionary); - this.joinPath = path; - } - } + Path getJoinPath(); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java new file mode 100644 index 0000000000..e56225c1c1 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java @@ -0,0 +1,42 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Getter; + +/** + * SQLDimension are dimension columns with extra physical information. + */ +public class SQLDimension extends Dimension implements SQLColumn { + @Getter + private final String reference; + + @Getter + private final Path joinPath; + + public SQLDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); + this.joinPath = null; + } else { + Path path = new Path(tableClass, dictionary, joinTo.path()); + this.reference = generateColumnReference(path, dictionary); + this.joinPath = path; + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java new file mode 100644 index 0000000000..9adb42c333 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java @@ -0,0 +1,32 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; +import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; + +import java.util.Set; + +/** + * SQLMetric would contain {@link SQLMetricFunction} instead of {@link MetricFunction}. + */ +public class SQLMetric extends Metric { + public SQLMetric(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + } + + @Override + protected SQLMetricFunction constructMetricFunction(String id, + String longName, + String description, + String expression, + Set arguments) { + return new SQLMetricFunction(id, longName, description, expression, arguments); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java index f0e230cfc7..c787232dec 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java @@ -5,47 +5,39 @@ */ package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; -import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; - import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.datastores.aggregation.metadata.models.Column; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import lombok.Data; import lombok.EqualsAndHashCode; -import java.util.Map; -import java.util.function.Function; -import java.util.stream.Collectors; - /** * SQL extension of {@link Table} which also contains sql column meta data. */ @EqualsAndHashCode(callSuper = true) @Data public class SQLTable extends Table { - private Map sqlColumns; - public SQLTable(Class cls, EntityDictionary dictionary) { super(cls, dictionary); - this.sqlColumns = resolveSQLDimensions(cls, dictionary); } - /** - * Resolve all sql columns of a table. - * - * @param cls table class - * @param dictionary dictionary contains the table class - * @return all resolved sql column metadata - */ - public static Map resolveSQLDimensions(Class cls, EntityDictionary dictionary) { - return dictionary.getAllFields(cls).stream() - .filter(field -> Column.getDataType(cls, field, dictionary) != null) - .filter(field -> !isMetricField(dictionary, cls, field)) - .collect(Collectors.toMap(Function.identity(), field -> new SQLColumn(cls, field, dictionary))); + public final SQLColumn getSQLColumn(String fieldName) { + SQLDimension dimension = getColumn(SQLDimension.class, fieldName); + return dimension == null ? getColumn(SQLTimeDimension.class, fieldName) : dimension; + } + + @Override + protected SQLMetric constructMetric(Class cls, String fieldName, EntityDictionary dictionary) { + return new SQLMetric(cls, fieldName, dictionary); + } + + @Override + protected SQLTimeDimension constructTimeDimension(Class cls, String fieldName, EntityDictionary dictionary) { + return new SQLTimeDimension(cls, fieldName, dictionary); } - public SQLColumn getColumn(String fieldName) { - return sqlColumns.get(fieldName); + @Override + protected SQLDimension constructDimension(Class cls, String fieldName, EntityDictionary dictionary) { + return new SQLDimension(cls, fieldName, dictionary); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java new file mode 100644 index 0000000000..a1c5a80dea --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java @@ -0,0 +1,42 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; + +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; +import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; + +import lombok.Getter; + +/** + * SQLTimeDimension are time dimension columns with extra physical information. + */ +public class SQLTimeDimension extends TimeDimension implements SQLColumn { + @Getter + private final String reference; + + @Getter + private final Path joinPath; + + public SQLTimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { + super(tableClass, fieldName, dictionary); + + JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); + + if (joinTo == null || joinTo.path().equals("")) { + this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); + this.joinPath = null; + } else { + Path path = new Path(tableClass, dictionary, joinTo.path()); + this.reference = generateColumnReference(path, dictionary); + this.joinPath = path; + } + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java index 6078874b1e..d909d883d0 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java @@ -16,7 +16,6 @@ import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; -import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; @@ -71,7 +70,7 @@ public void testBasicTranslation() { Query query = translator.getQuery(); - assertEquals(playerStatsTable, query.getAnalyticView()); + assertEquals(playerStatsTable, query.getTable()); assertEquals(1, query.getMetrics().size()); assertEquals("lowScore", query.getMetrics().get(0).getAlias()); assertEquals(2, query.getGroupByDimensions().size()); @@ -106,19 +105,6 @@ public void testWherePromotion() throws ParseException { assertEquals(havingFilter, query.getHavingFilter()); } - @Test - public void testInvalidQueriedTable() { - EntityProjection projection = EntityProjection.builder() - .type(Country.class) - .build(); - - assertThrows(InvalidOperationException.class, () -> new EntityProjectionTranslator( - new Table(Country.class, dictionary), - projection, - dictionary - )); - } - @Test public void testTimeDimension() { EntityProjection projection = basicProjection.copyOf() diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java index 9e63cbf3f6..5d69ce4747 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java @@ -41,7 +41,7 @@ public void testNoMetricQuery() { sortMap.put("country.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); @@ -58,7 +58,7 @@ public void testSortingOnId() { sortMap.put("id", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("id"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) @@ -78,7 +78,7 @@ public void testSortingOnNotQueriedDimension() { sortMap.put("country.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) @@ -97,7 +97,7 @@ public void testSortingOnNotQueriedMetric() { sortMap.put("highScore", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) @@ -116,7 +116,7 @@ public void testSortingOnNestedDimensionField() { sortMap.put("country.continent.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) @@ -139,7 +139,7 @@ public void testHavingFilterPromotionUngroupedDimension() throws ParseException FilterExpression havingFilter = constraints.getHavingExpression(); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .whereFilter(whereFilter) .havingFilter(havingFilter) @@ -163,7 +163,7 @@ public void testHavingFilterNoAggregatedMetric() throws ParseException { FilterExpression havingFilter = constraints.getHavingExpression(); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .whereFilter(whereFilter) .havingFilter(havingFilter) @@ -188,7 +188,7 @@ public void testHavingFilterOnDimensionTable() throws ParseException { FilterExpression havingFilter = constraints.getHavingExpression(); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .whereFilter(whereFilter) .havingFilter(havingFilter) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java index 486dde0269..b7798e6a4a 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java @@ -8,31 +8,35 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.jpa.JpaDataStore; import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; import com.yahoo.elide.datastores.multiplex.MultiplexManager; +import javax.persistence.EntityManagerFactory; + public class AggregationDataStoreTestHarness implements DataStoreTestHarness { - private SQLQueryEngineFactory queryEngineFactory; + private EntityManagerFactory entityManagerFactory; - public AggregationDataStoreTestHarness(SQLQueryEngineFactory queryEngineFactory) { - this.queryEngineFactory = queryEngineFactory; + public AggregationDataStoreTestHarness(EntityManagerFactory entityManagerFactory) { + this.entityManagerFactory = entityManagerFactory; } @Override public DataStore getDataStore() { MetaDataStore metaDataStore = new MetaDataStore(); - AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + QueryEngine sqlQueryEngine = new SQLQueryEngine(metaDataStore, entityManagerFactory); + + AggregationDataStore aggregationDataStore = new AggregationDataStore(sqlQueryEngine); DataStore jpaStore = new JpaDataStore( - () -> queryEngineFactory.getEmf().createEntityManager(), + () -> entityManagerFactory.createEntityManager(), NonJtaTransaction::new ); - // meta data store needs to be put at first to populate meta data models return new MultiplexManager(jpaStore, metaDataStore, aggregationDataStore); } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java index 0ad213230b..b95a6c5c60 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java @@ -19,14 +19,14 @@ import com.yahoo.elide.datastores.aggregation.example.SubCountry; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import java.util.Collections; @@ -36,7 +36,7 @@ public abstract class SQLUnitTest { protected static EntityManagerFactory emf; - protected static AnalyticView playerStatsTable; + protected static Table playerStatsTable; protected static EntityDictionary dictionary; protected static RSQLFilterDialect filterParser; protected static MetaDataStore metaDataStore = new MetaDataStore(); @@ -62,11 +62,11 @@ public static void init() { dictionary.bindEntity(Continent.class); filterParser = new RSQLFilterDialect(dictionary); - playerStatsTable = new SQLAnalyticView(PlayerStats.class, dictionary); + playerStatsTable = new SQLTable(PlayerStats.class, dictionary); metaDataStore.populateEntityDictionary(dictionary); - engine = new SQLQueryEngine(emf, metaDataStore); + engine = new SQLQueryEngine(metaDataStore, emf); ASIA.setName("Asia"); ASIA.setId("1"); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java index aa439d571e..25b7026cf5 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -21,7 +21,6 @@ import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; import com.yahoo.elide.datastores.aggregation.framework.AggregationDataStoreTestHarness; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; import com.yahoo.elide.initialization.IntegrationTest; import com.fasterxml.jackson.databind.JsonNode; @@ -36,7 +35,6 @@ import java.io.IOException; import java.util.Map; -import javax.persistence.EntityManagerFactory; import javax.persistence.Persistence; import javax.ws.rs.core.MediaType; @@ -44,15 +42,11 @@ * Integration tests for {@link AggregationDataStore}. */ public class AggregationDataStoreIntegrationTest extends IntegrationTest { - SQLQueryEngineFactory queryEngineFactory; - private static final ObjectMapper JSON_MAPPER = new ObjectMapper(); @Override protected DataStoreTestHarness createHarness() { - EntityManagerFactory emf = Persistence.createEntityManagerFactory("aggregationStore"); - queryEngineFactory = new SQLQueryEngineFactory(emf); - return new AggregationDataStoreTestHarness(queryEngineFactory); + return new AggregationDataStoreTestHarness(Persistence.createEntityManagerFactory("aggregationStore")); } @Test @@ -806,7 +800,7 @@ public void metaDataTest() { given() .accept("application/vnd.api+json") - .get("/analyticView/playerStats") + .get("/table/playerStats") .then() .statusCode(HttpStatus.SC_OK) .body("data.attributes.cardinality", equalTo("LARGE")) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java index 6043dfa550..1237c42763 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -17,9 +17,9 @@ import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import com.yahoo.elide.request.Sorting; @@ -35,13 +35,13 @@ import java.util.stream.StreamSupport; public class QueryEngineTest extends SQLUnitTest { - private static AnalyticView playerStatsViewTable; + private static Table playerStatsViewTable; @BeforeAll public static void init() { SQLUnitTest.init(); - playerStatsViewTable = new SQLAnalyticView(PlayerStatsView.class, dictionary); + playerStatsViewTable = new SQLTable(PlayerStatsView.class, dictionary); } /** @@ -50,7 +50,7 @@ public static void init() { @Test public void testFullTableLoad() { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .metric(invoke(playerStatsTable.getMetric("highScore"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) @@ -91,7 +91,7 @@ public void testFullTableLoad() { @Test public void testDegenerateDimensionFilter() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) @@ -120,7 +120,7 @@ public void testDegenerateDimensionFilter() throws Exception { @Test public void testFilterJoin() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) .whereFilter(filterParser.parseFilterExpression("country.name=='United States'", @@ -151,7 +151,7 @@ public void testFilterJoin() throws Exception { @Test public void testSubqueryFilterJoin() throws Exception { Query query = Query.builder() - .analyticView(playerStatsViewTable) + .table(playerStatsViewTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", PlayerStatsView.class, false)) @@ -174,7 +174,7 @@ public void testSubqueryFilterJoin() throws Exception { @Test public void testSubqueryLoad() { Query query = Query.builder() - .analyticView(playerStatsViewTable) + .table(playerStatsViewTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .build(); @@ -198,7 +198,7 @@ public void testSortJoin() { sortMap.put("player.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) @@ -248,7 +248,7 @@ public void testPagination() { ); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) @@ -278,7 +278,7 @@ public void testPagination() { @Test public void testHavingClause() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .havingFilter(filterParser.parseFilterExpression("highScore < 2400", @@ -306,7 +306,7 @@ public void testHavingClause() throws Exception { @Test public void testHavingClauseJoin() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) @@ -345,7 +345,7 @@ public void testEdgeCasesQuery() throws Exception { sortMap.put("player.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsViewTable) + .table(playerStatsViewTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsViewTable.getDimension("countryName"))) .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", @@ -377,7 +377,7 @@ public void testSortByMultipleColumns() { sortMap.put("player.name", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) @@ -421,7 +421,7 @@ public void testRelationshipHydration() { sortMap.put("overallRating", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) @@ -465,7 +465,7 @@ public void testRelationshipHydration() { @Test public void testJoinToGroupBy() { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) .build(); @@ -496,7 +496,7 @@ public void testJoinToGroupBy() { @Test public void testJoinToFilter() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .whereFilter(filterParser.parseFilterExpression("countryIsoCode==USA", @@ -530,7 +530,7 @@ public void testJoinToSort() { sortMap.put("countryIsoCode", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) @@ -570,7 +570,7 @@ public void testJoinToSort() { @Test public void testTotalScoreByMonth() { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.MONTH)) .build(); @@ -598,7 +598,7 @@ public void testFilterByTemporalDimension() { Lists.newArrayList(Timestamp.valueOf("2019-07-11 00:00:00"))); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) .whereFilter(predicate) @@ -622,7 +622,7 @@ public void testSortAggregatedMetric() { sortMap.put("lowScore", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) @@ -652,7 +652,7 @@ public void testAmbiguousFields() { sortMap.put("lowScore", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .groupByDimension(toProjection(playerStatsTable.getDimension("playerName"))) .groupByDimension(toProjection(playerStatsTable.getDimension("player2Name"))) .metric(invoke(playerStatsTable.getMetric("lowScore"))) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java index 77adb647e8..e0cbd633c7 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -52,7 +52,7 @@ public static void init() { @Test public void testFilterJoin() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) @@ -104,7 +104,7 @@ public void testRelationshipHydration() { sortMap.put("subCountry.name", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) @@ -163,7 +163,7 @@ public void testRelationshipHydration() { @Test public void testJoinToGroupBy() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("subCountryIsoCode"))) .build(); @@ -194,7 +194,7 @@ public void testJoinToGroupBy() throws Exception { @Test public void testJoinToFilter() throws Exception { Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .whereFilter(filterParser.parseFilterExpression("subCountryIsoCode==USA", @@ -230,7 +230,7 @@ public void testJoinToSort() throws Exception { sortMap.put("subCountryIsoCode", Sorting.SortOrder.asc); Query query = Query.builder() - .analyticView(playerStatsTable) + .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java index fb5ca13f71..7eb3f82be0 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -14,11 +14,11 @@ import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; -import com.yahoo.elide.datastores.aggregation.metadata.models.AnalyticView; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLAnalyticView; - +import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.request.Sorting; + import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -29,12 +29,12 @@ import java.util.stream.StreamSupport; public class ViewTest extends SQLUnitTest { - protected static AnalyticView playerStatsWithViewSchema; + protected static Table playerStatsWithViewSchema; @BeforeAll public static void init() { SQLUnitTest.init(); - playerStatsWithViewSchema = new SQLAnalyticView(PlayerStatsWithView.class, dictionary); + playerStatsWithViewSchema = new SQLTable(PlayerStatsWithView.class, dictionary); } @Test @@ -43,7 +43,7 @@ public void testViewRelationFailure() { sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryView"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) @@ -58,7 +58,7 @@ public void testViewAttribute() { sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) @@ -92,7 +92,7 @@ public void testNestedViewAttribute() throws Exception { sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) @@ -126,7 +126,7 @@ public void testNestedRelationshipAttribute() throws Exception { sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) @@ -161,7 +161,7 @@ public void testSortingViewAttribute() throws Exception { sortMap.put("countryView.isoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) @@ -196,7 +196,7 @@ public void testSortingNestedViewAttribute() throws Exception { sortMap.put("countryView.nestedView.isoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) @@ -231,7 +231,7 @@ public void testSortingNestedRelationshipAttribute() throws Exception { sortMap.put("countryView.nestedRelationship.isoCode", Sorting.SortOrder.desc); Query query = Query.builder() - .analyticView(playerStatsWithViewSchema) + .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java index 29a10bf969..c60193d289 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java @@ -14,9 +14,9 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.dialect.RSQLFilterDialect; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; -import com.yahoo.elide.datastores.aggregation.QueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; -import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngineFactory; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.jpa.JpaDataStore; import com.yahoo.elide.datastores.jpa.transaction.NonJtaTransaction; import com.yahoo.elide.datastores.multiplex.MultiplexManager; @@ -52,7 +52,8 @@ public class ElideAutoConfiguration { @Bean @ConditionalOnMissingBean public Elide initializeElide(EntityDictionary dictionary, - DataStore dataStore, ElideConfigProperties settings) { + DataStore dataStore, + ElideConfigProperties settings) { ElideSettingsBuilder builder = new ElideSettingsBuilder(dataStore) .withEntityDictionary(dictionary) @@ -95,25 +96,35 @@ public T instantiate(Class cls) { } /** - * Creates the DataStore Elide. Override to use a different store. + * Create a QueryEngine instance for aggregation data store to use. * @param entityManagerFactory The JPA factory which creates entity managers. - * @return An instance of a JPA DataStore. + * @return An instance of a QueryEngine */ @Bean @ConditionalOnMissingBean - public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, - QueryEngineFactory queryEngineFactory, - ElideConfigProperties settings) throws ClassNotFoundException { + public QueryEngine buildQueryEngine(EntityManagerFactory entityManagerFactory) { MetaDataStore metaDataStore = new MetaDataStore(); - AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngineFactory, metaDataStore); + return new SQLQueryEngine(metaDataStore, entityManagerFactory); + } + + /** + * Creates the DataStore Elide. Override to use a different store. + * @param entityManagerFactory The JPA factory which creates entity managers. + * @param queryEngine QueryEngine instance for aggregation data store + * @return An instance of a JPA DataStore. + */ + @Bean + @ConditionalOnMissingBean + public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, QueryEngine queryEngine) { + AggregationDataStore aggregationDataStore = new AggregationDataStore(queryEngine); JpaDataStore jpaDataStore = new JpaDataStore( () -> { return entityManagerFactory.createEntityManager(); }, (em -> { return new NonJtaTransaction(em); })); // meta data store needs to be put at first to populate meta data models - return new MultiplexManager(jpaDataStore, metaDataStore, aggregationDataStore); + return new MultiplexManager(jpaDataStore, queryEngine.getMetaDataStore(), aggregationDataStore); } /** @@ -135,15 +146,4 @@ public Swagger buildSwagger(EntityDictionary dictionary, ElideConfigProperties s return swagger; } - - /** - * Configure the QueryEngineFactory that the Aggregation Data Store uses. - * @param entityManagerFactory Needed by the SQLQueryEngine - * @return a SQLQueryEngineFactory - */ - @Bean - @ConditionalOnMissingBean - public QueryEngineFactory buildQueryEngineFactory(EntityManagerFactory entityManagerFactory) { - return new SQLQueryEngineFactory(entityManagerFactory); - } } From d205cfe3ca87bbccc6b912b8dcc946f0a61fe318 Mon Sep 17 00:00:00 2001 From: hchen04 Date: Tue, 11 Feb 2020 14:17:04 -0600 Subject: [PATCH 06/16] @Join and JoinPath --- .../yahoo/elide/core/EntityDictionary.java | 16 ---- .../aggregation/annotation/Join.java | 25 ++++++ .../aggregation/annotation/Temporal.java | 2 +- .../annotation/TimeGrainDefinition.java | 2 +- .../datastores/aggregation/core/JoinPath.java | 54 +++++++++++++ .../aggregation/metadata/MetaDataStore.java | 5 ++ .../aggregation/metadata/enums/ColumnTag.java | 14 ++++ .../enums/{Tag.java => TableTag.java} | 4 +- .../{time => metadata/enums}/TimeGrain.java | 2 +- .../aggregation/metadata/models/Column.java | 4 +- .../aggregation/metadata/models/Table.java | 4 + .../metadata/models/TimeDimensionGrain.java | 2 +- .../aggregation/query/ColumnProjection.java | 2 +- .../query/TimeDimensionProjection.java | 2 +- .../queryengines/sql/SQLQueryConstructor.java | 79 ++++++++----------- .../queryengines/sql/SQLQueryEngine.java | 3 +- .../queryengines/sql/annotation/JoinTo.java | 9 --- .../queryengines/sql/metadata/SQLColumn.java | 4 +- .../sql/metadata/SQLDimension.java | 6 +- .../sql/metadata/SQLTimeDimension.java | 6 +- .../sql/query/SQLQueryTemplate.java | 2 +- .../EntityProjectionTranslatorTest.java | 2 +- .../aggregation/example/CountryView.java | 6 +- .../aggregation/example/PlayerStats.java | 2 +- .../example/PlayerStatsWithView.java | 7 +- .../aggregation/framework/SQLUnitTest.java | 2 +- .../queryengines/sql/QueryEngineTest.java | 2 +- .../queryengines/sql/SubselectTest.java | 4 +- .../queryengines/sql/ViewTest.java | 23 +----- 29 files changed, 172 insertions(+), 123 deletions(-) create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Join.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java rename elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/{Tag.java => TableTag.java} (80%) rename elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/{time => metadata/enums}/TimeGrain.java (89%) diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index def633e65c..5c5f089ecb 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -1144,22 +1144,6 @@ private boolean isClassBound(Class objClass) { return (entityBindings.getOrDefault(objClass, EMPTY_BINDING) != EMPTY_BINDING); } - - /** - * Check whether a class is a JPA entity - * - * @param objClass class - * @return True if it is a JPA entity - */ - public final boolean isJPAEntity(Class objClass) { - try { - lookupEntityClass(objClass); - return true; - } catch (IllegalArgumentException e) { - return false; - } - } - /** * Retrieve the accessible object for a field from a target object. * diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Join.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Join.java new file mode 100644 index 0000000000..0d5eb08f31 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Join.java @@ -0,0 +1,25 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.annotation; + +import java.lang.annotation.Documented; +import java.lang.annotation.ElementType; +import java.lang.annotation.Retention; +import java.lang.annotation.RetentionPolicy; +import java.lang.annotation.Target; + +@Documented +@Target({ElementType.FIELD, ElementType.METHOD}) +@Retention(RetentionPolicy.RUNTIME) +public @interface Join { + /** + * Join on clause constraint for customizing relationship joins as a plain sql string. Provided in the model. + * Use "%from" and "%join% to represent the two sides of join. + * + * @return join constraint like %from.col1 = %join.col2 + */ + String value(); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java index 0a9e84ea93..5836c19ed6 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/Temporal.java @@ -5,7 +5,7 @@ */ package com.yahoo.elide.datastores.aggregation.annotation; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import java.lang.annotation.Documented; import java.lang.annotation.ElementType; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java index 6dac86360a..f1f5743f76 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/annotation/TimeGrainDefinition.java @@ -5,7 +5,7 @@ */ package com.yahoo.elide.datastores.aggregation.annotation; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; /** * A time grain that a time based dimension can be converted to. diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java new file mode 100644 index 0000000000..448aa5efac --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java @@ -0,0 +1,54 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.core; + +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isTableJoin; + +import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; +import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.annotation.Join; + +import com.google.common.collect.ImmutableList; + +import java.util.ArrayList; +import java.util.List; + +/** + * JoinPath extends {@link Path} to allow navigation through {@link Join} annotation. + */ +public class JoinPath extends Path { + public JoinPath(Class entityClass, EntityDictionary dictionary, String dotSeparatedPath) { + super(resolveJoinPathElements(entityClass, dictionary, dotSeparatedPath)); + } + + private static List resolveJoinPathElements(Class entityClass, + EntityDictionary dictionary, + String dotSeparatedPath) { + List elements = new ArrayList<>(); + String[] fieldNames = dotSeparatedPath.split("\\."); + + Class currentClass = entityClass; + for (String fieldName : fieldNames) { + if (dictionary.isRelation(currentClass, fieldName) || isTableJoin(currentClass, fieldName, dictionary)) { + Class joinClass = dictionary.getParameterizedType(currentClass, fieldName); + elements.add(new PathElement(currentClass, joinClass, fieldName)); + currentClass = joinClass; + } else if (dictionary.isAttribute(currentClass, fieldName) + || fieldName.equals(dictionary.getIdFieldName(entityClass))) { + Class attributeClass = dictionary.getType(currentClass, fieldName); + elements.add(new PathElement(currentClass, attributeClass, fieldName)); + } else if ("this".equals(fieldName)) { + elements.add(new PathElement(currentClass, null, fieldName)); + } else { + String alias = dictionary.getJsonAliasFor(currentClass); + throw new InvalidValueException(alias + " doesn't contain the field " + fieldName); + } + } + + return ImmutableList.copyOf(elements); + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index fe0e886b99..f230aee384 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.datastore.inmemory.HashMapDataStore; import com.yahoo.elide.core.exceptions.DuplicateMappingException; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.metadata.models.Column; import com.yahoo.elide.datastores.aggregation.metadata.models.DataType; @@ -165,4 +166,8 @@ public Set getMetaData(Class cls) { public static boolean isMetricField(EntityDictionary dictionary, Class cls, String fieldName) { return dictionary.attributeOrRelationAnnotationExists(cls, fieldName, MetricAggregation.class); } + + public static boolean isTableJoin(Class cls, String fieldName, EntityDictionary dictionary) { + return dictionary.getAttributeOrRelationAnnotation(cls, Join.class, fieldName) != null; + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java new file mode 100644 index 0000000000..2e8f22193d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java @@ -0,0 +1,14 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.metadata.enums; + +/** + * Tag attached to fields. + */ +public enum ColumnTag { + DISPLAY, + GROUPABLE +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java similarity index 80% rename from elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java rename to elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java index 92065a06d3..66f55a6b5f 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Tag.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java @@ -1,5 +1,5 @@ /* - * Copyright 2019, Yahoo Inc. + * Copyright 2020, Yahoo Inc. * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ @@ -8,6 +8,6 @@ /** * Tag attached to fields. */ -public enum Tag { +public enum TableTag { DISPLAY } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TimeGrain.java similarity index 89% rename from elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java rename to elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TimeGrain.java index c45102a3ac..0eb79d9aa0 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/time/TimeGrain.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TimeGrain.java @@ -3,7 +3,7 @@ * Licensed under the Apache License, Version 2.0 * See LICENSE file in project root for terms. */ -package com.yahoo.elide.datastores.aggregation.time; +package com.yahoo.elide.datastores.aggregation.metadata.enums; import java.time.Period; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java index 8bc9c02baf..5cfe80224b 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -8,7 +8,7 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.annotation.Meta; -import com.yahoo.elide.datastores.aggregation.metadata.enums.Tag; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ColumnTag; import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; import lombok.Data; @@ -45,7 +45,7 @@ public abstract class Column { private DataType dataType; @ToString.Exclude - private Set columnTags; + private Set columnTags; protected Column(Class tableClass, String fieldName, EntityDictionary dictionary) { this.tableName = dictionary.getJsonAliasFor(tableClass); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index 8d36e694c4..c07155e14b 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -14,6 +14,7 @@ import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TableTag; import lombok.Data; import lombok.ToString; @@ -55,6 +56,9 @@ public class Table { @ToString.Exclude private Set dimensions; + @ToString.Exclude + private Set tableTags; + @Exclude @ToString.Exclude private final Map columnMap; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java index 7ff87e8380..cdedc8dcff 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimensionGrain.java @@ -7,7 +7,7 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import lombok.Data; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java index 2228751f72..fd4bc4ca17 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/ColumnProjection.java @@ -6,10 +6,10 @@ package com.yahoo.elide.datastores.aggregation.query; import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.models.Column; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import java.io.Serializable; import java.util.TimeZone; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java index bab8385207..f7a9f95ba1 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/query/TimeDimensionProjection.java @@ -7,9 +7,9 @@ package com.yahoo.elide.datastores.aggregation.query; import com.yahoo.elide.core.exceptions.InvalidValueException; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.models.Column; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import java.util.TimeZone; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java index 92201fe476..21e2ce81ef 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -7,6 +7,7 @@ import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; +import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isTableJoin; import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; @@ -17,6 +18,8 @@ import com.yahoo.elide.core.filter.FilterTranslator; import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; +import com.yahoo.elide.datastores.aggregation.annotation.Join; +import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; @@ -186,20 +189,7 @@ private String constructProjectionWithReference(SQLQueryTemplate template, SQLTa Class tableClass = dictionary.getEntityClass(table.getName()); List dimensionProjections = template.getGroupByDimensions().stream() - .map(dimension -> { - String fieldName = dimension.getColumn().getName(); - - // relation to Non-JPA Entities object can't be projected - if (dictionary.isRelation(tableClass, fieldName)) { - Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); - if (!dictionary.isJPAEntity(relationshipClass)) { - throw new InvalidPredicateException( - "Can't query on non-JPA relationship field: " + dimension.getColumn().getName()); - } - } - - return resolveSQLColumnReference(dimension, table) + " AS " + dimension.getAlias(); - }) + .map(dimension -> resolveSQLColumnReference(dimension, table) + " AS " + dimension.getAlias()) .collect(Collectors.toList()); if (metricProjections.isEmpty()) { @@ -238,7 +228,7 @@ private void addJoinClauses(Path joinPath, Set alreadyJoined) { Class parentClass = pathElement.getType(); // Nothing left to join. - if (! dictionary.isRelation(parentClass, fieldName)) { + if (!dictionary.isRelation(parentClass, fieldName) && !isTableJoin(parentClass, fieldName, dictionary)) { return; } @@ -255,47 +245,48 @@ private void addJoinClauses(Path joinPath, Set alreadyJoined) { } /** - * Build a single dimension join clause for joining a relationship table to the parent table. + * Build a single dimension join clause for joining a relationship/join table to the parent table. * - * @param parentClass parent class - * @param parentAlias parent table alias - * @param relationshipClass relationship class - * @param relationshipName relationship field name + * @param fromClass parent class + * @param fromAlias parent table alias + * @param joinClass relationship/join class + * @param joinField relationship/join field name * @return built join clause i.e. LEFT JOIN table1 AS dimension1 ON table0.dim_id = dimension1.id */ - private String extractJoinClause(Class parentClass, - String parentAlias, - Class relationshipClass, - String relationshipName) { + private String extractJoinClause(Class fromClass, + String fromAlias, + Class joinClass, + String joinField) { //TODO - support composite join keys. //TODO - support joins where either side owns the relationship. //TODO - Support INNER and RIGHT joins. //TODO - Support toMany joins. - String relationshipAlias = appendAlias(parentAlias, relationshipName); - String relationshipColumnName = dictionary.getAnnotatedColumnName(parentClass, relationshipName); + String joinAlias = appendAlias(fromAlias, joinField); + String joinColumnName = dictionary.getAnnotatedColumnName(fromClass, joinField); // resolve the right hand side of JOIN - String joinSource = constructTableOrSubselect(relationshipClass); - - JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation( - parentClass, - JoinTo.class, - relationshipColumnName); - - String joinClause = joinTo == null - ? String.format("%s.%s = %s.%s", - parentAlias, - relationshipColumnName, - relationshipAlias, - dictionary.getAnnotatedColumnName( - relationshipClass, - dictionary.getIdFieldName(relationshipClass))) - : extractJoinExpression(joinTo.joinClause(), parentAlias, relationshipAlias); + String joinSource = constructTableOrSubselect(joinClass); + + Join join = dictionary.getAttributeOrRelationAnnotation( + fromClass, + Join.class, + joinField); + + String joinClause = join == null + ? String.format( + "%s.%s = %s.%s", + fromAlias, + joinColumnName, + joinAlias, + dictionary.getAnnotatedColumnName( + joinClass, + dictionary.getIdFieldName(joinClass))) + : extractJoinExpression(join.value(), fromAlias, joinAlias); return String.format("LEFT JOIN %s AS %s ON %s", joinSource, - relationshipAlias, + joinAlias, joinClause); } @@ -365,7 +356,7 @@ private Path expandJoinToPath(Path path) { return path; } - return new Path(entityClass, dictionary, joinTo.path()); + return new JoinPath(entityClass, dictionary, joinTo.path()); } /** diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java index b342ef905d..168ef1007f 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -14,6 +14,7 @@ import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.expression.PredicateExtractionVisitor; import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; @@ -264,7 +265,7 @@ public static String generateColumnReference(Path path, EntityDictionary diction if (joinTo == null) { return getPathAlias(path) + "." + dictionary.getAnnotatedColumnName(lastClass, last.getFieldName()); } else { - return generateColumnReference(new Path(lastClass, dictionary, joinTo.path()), dictionary); + return generateColumnReference(new JoinPath(lastClass, dictionary, joinTo.path()), dictionary); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java index 299135aea0..f72b05a01f 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/annotation/JoinTo.java @@ -19,19 +19,10 @@ @Target({ElementType.FIELD, ElementType.METHOD}) @Retention(RetentionPolicy.RUNTIME) public @interface JoinTo { - /** * Dot separated path through the entity relationship graph to an attribute. * If the current entity is author, then a path would be "book.publisher.name". * @return The path */ String path() default ""; - - /** - * Join on clause constraint for customizing relationship joins as a plain sql string. Provided in the model. - * Use "%from" and "%join% to represent the two sides of join. - * - * @return join constraint like %from.col1 = %join.col2 - */ - String joinClause() default ""; } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java index e0e7df088d..6e5f734cc3 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLColumn.java @@ -5,7 +5,7 @@ */ package com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata; -import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.core.JoinPath; /** * Column with physical SQL information like reference and join to path. @@ -13,5 +13,5 @@ public interface SQLColumn { String getReference(); - Path getJoinPath(); + JoinPath getJoinPath(); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java index e56225c1c1..c5d46f4156 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java @@ -9,7 +9,7 @@ import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; @@ -23,7 +23,7 @@ public class SQLDimension extends Dimension implements SQLColumn { private final String reference; @Getter - private final Path joinPath; + private final JoinPath joinPath; public SQLDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { super(tableClass, fieldName, dictionary); @@ -34,7 +34,7 @@ public SQLDimension(Class tableClass, String fieldName, EntityDictionary dict this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); this.joinPath = null; } else { - Path path = new Path(tableClass, dictionary, joinTo.path()); + JoinPath path = new JoinPath(tableClass, dictionary, joinTo.path()); this.reference = generateColumnReference(path, dictionary); this.joinPath = path; } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java index a1c5a80dea..17241405c2 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java @@ -9,7 +9,7 @@ import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; import com.yahoo.elide.core.EntityDictionary; -import com.yahoo.elide.core.Path; +import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; @@ -23,7 +23,7 @@ public class SQLTimeDimension extends TimeDimension implements SQLColumn { private final String reference; @Getter - private final Path joinPath; + private final JoinPath joinPath; public SQLTimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { super(tableClass, fieldName, dictionary); @@ -34,7 +34,7 @@ public SQLTimeDimension(Class tableClass, String fieldName, EntityDictionary this.reference = getClassAlias(tableClass) + "." + dictionary.getAnnotatedColumnName(tableClass, fieldName); this.joinPath = null; } else { - Path path = new Path(tableClass, dictionary, joinTo.path()); + JoinPath path = new JoinPath(tableClass, dictionary, joinTo.path()); this.reference = generateColumnReference(path, dictionary); this.joinPath = path; } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java index efc06caecf..b1f0d50003 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLQueryTemplate.java @@ -5,10 +5,10 @@ */ package com.yahoo.elide.datastores.aggregation.queryengines.sql.query; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import com.google.common.collect.Sets; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java index d909d883d0..661e378edb 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java @@ -16,10 +16,10 @@ import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import com.yahoo.elide.request.Argument; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java index 0f983ff1ef..96fcc23fad 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -8,6 +8,7 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; import lombok.Data; @@ -31,10 +32,7 @@ public class CountryView { private CountryViewNested nestedView; - @ToOne - @JoinTo( - joinClause = "%from.id = %join.id" - ) + @Join("%from.id = %join.id") public CountryViewNested getNestedView() { return nestedView; } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java index e5157c49fe..60f394bafc 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java @@ -13,11 +13,11 @@ import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import lombok.EqualsAndHashCode; import lombok.Setter; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java index 7541c38775..c6d437e919 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -6,19 +6,19 @@ package com.yahoo.elide.datastores.aggregation.example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; import com.yahoo.elide.datastores.aggregation.annotation.TimeGrainDefinition; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMin; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import lombok.EqualsAndHashCode; import lombok.Setter; @@ -200,8 +200,7 @@ public void setSubCountryIsoCode(String isoCode) { this.subCountryIsoCode = isoCode; } - @ToOne - @JoinTo(joinClause = "%from.country_id = %join.id") + @Join("%from.country_id = %join.id") public CountryView getCountryView() { return countryView; } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java index b95a6c5c60..6fc0b28798 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java @@ -18,6 +18,7 @@ import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; import com.yahoo.elide.datastores.aggregation.example.SubCountry; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; @@ -27,7 +28,6 @@ import com.yahoo.elide.datastores.aggregation.query.TimeDimensionProjection; import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import java.util.Collections; import java.util.HashMap; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java index 1237c42763..5d870e1c3e 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -17,10 +17,10 @@ import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; import com.yahoo.elide.request.Sorting; import com.google.common.collect.Lists; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java index e0cbd633c7..bed32e8a87 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -13,10 +13,10 @@ import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.SubCountry; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; +import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.query.Query; -import com.yahoo.elide.datastores.aggregation.time.TimeGrain; - import com.yahoo.elide.request.Sorting; + import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java index 7eb3f82be0..b4424f2db9 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -8,9 +8,7 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.junit.jupiter.api.Assertions.assertNull; -import static org.junit.jupiter.api.Assertions.assertThrows; -import com.yahoo.elide.core.exceptions.InvalidPredicateException; import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStatsWithView; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; @@ -37,21 +35,6 @@ public static void init() { playerStatsWithViewSchema = new SQLTable(PlayerStatsWithView.class, dictionary); } - @Test - public void testViewRelationFailure() { - Map sortMap = new TreeMap<>(); - sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); - - Query query = Query.builder() - .table(playerStatsWithViewSchema) - .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) - .groupByDimension(toProjection(playerStatsWithViewSchema.getDimension("countryView"))) - .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) - .build(); - - assertThrows(InvalidPredicateException.class, () -> engine.executeQuery(query)); - } - @Test public void testViewAttribute() { Map sortMap = new TreeMap<>(); @@ -158,7 +141,7 @@ public void testNestedRelationshipAttribute() throws Exception { @Test public void testSortingViewAttribute() throws Exception { Map sortMap = new TreeMap<>(); - sortMap.put("countryView.isoCode", Sorting.SortOrder.desc); + sortMap.put("countryViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() .table(playerStatsWithViewSchema) @@ -193,7 +176,7 @@ public void testSortingViewAttribute() throws Exception { @Test public void testSortingNestedViewAttribute() throws Exception { Map sortMap = new TreeMap<>(); - sortMap.put("countryView.nestedView.isoCode", Sorting.SortOrder.desc); + sortMap.put("countryViewRelationshipIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() .table(playerStatsWithViewSchema) @@ -228,7 +211,7 @@ public void testSortingNestedViewAttribute() throws Exception { @Test public void testSortingNestedRelationshipAttribute() throws Exception { Map sortMap = new TreeMap<>(); - sortMap.put("countryView.nestedRelationship.isoCode", Sorting.SortOrder.desc); + sortMap.put("countryViewRelationshipIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() .table(playerStatsWithViewSchema) From 15db2f1ce0d049042548e800c8d95be650e7bb32 Mon Sep 17 00:00:00 2001 From: hchen04 Date: Tue, 11 Feb 2020 15:53:12 -0600 Subject: [PATCH 07/16] add comment --- .../datastores/aggregation/metadata/MetaDataStore.java | 8 ++++++++ .../datastores/aggregation/metadata/models/Table.java | 4 +++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index f230aee384..a25a4c1662 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -167,6 +167,14 @@ public static boolean isMetricField(EntityDictionary dictionary, Class cls, S return dictionary.attributeOrRelationAnnotationExists(cls, fieldName, MetricAggregation.class); } + /** + * Returns whether a field in a table/entity is actually a JOIN to other table/entity. + * + * @param cls table/entity class + * @param fieldName field name + * @param dictionary metadata dictionary + * @return True if this field is a table join + */ public static boolean isTableJoin(Class cls, String fieldName, EntityDictionary dictionary) { return dictionary.getAttributeOrRelationAnnotation(cls, Join.class, fieldName) != null; } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index c07155e14b..8bac19a3dd 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -14,6 +14,7 @@ import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.enums.TableTag; import lombok.Data; @@ -104,7 +105,8 @@ public Table(Class cls, EntityDictionary dictionary) { */ private Set constructColumns(Class cls, EntityDictionary dictionary) { Set columns = dictionary.getAllFields(cls).stream() - .filter((field) -> Column.getDataType(cls, field, dictionary) != null) + .filter(field -> !MetaDataStore.isTableJoin(cls, field, dictionary)) + .filter(field -> Column.getDataType(cls, field, dictionary) != null) .map(field -> { if (isMetricField(dictionary, cls, field)) { return constructMetric(cls, field, dictionary); From 2db981f91fdc6bf023f43ebf7ca54dc6c1269df1 Mon Sep 17 00:00:00 2001 From: hchen04 Date: Mon, 13 Jan 2020 11:04:01 -0600 Subject: [PATCH 08/16] hide non-jpd entities in grpahql --- .../elide-datastore-aggregation/pom.xml | 7 + .../AggregationDataStoreIntegrationTest.java | 37 +++++ .../graphql/responses/testGraphQLSchema.json | 129 ++++++++++++++++++ .../com/yahoo/elide/graphql/ModelBuilder.java | 7 +- 4 files changed, 179 insertions(+), 1 deletion(-) create mode 100644 elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml index c2da5362a9..ec275df729 100644 --- a/elide-datastore/elide-datastore-aggregation/pom.xml +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -178,6 +178,13 @@ 2.5.0 test + + + org.apache.ant + ant + 1.10.7 + test + diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java index 25b7026cf5..9486c3c9c3 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -28,12 +28,15 @@ import com.fasterxml.jackson.databind.node.ArrayNode; import com.fasterxml.jackson.databind.node.JsonNodeFactory; import com.fasterxml.jackson.databind.node.ObjectNode; +import org.apache.tools.ant.util.FileUtils; import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; import io.restassured.response.ValidatableResponse; import java.io.IOException; +import java.io.InputStream; +import java.io.InputStreamReader; import java.util.Map; import javax.persistence.Persistence; import javax.ws.rs.core.MediaType; @@ -49,6 +52,34 @@ protected DataStoreTestHarness createHarness() { return new AggregationDataStoreTestHarness(Persistence.createEntityManagerFactory("aggregationStore")); } + @Test + public void testGraphQLSchema() throws IOException { + String graphQLRequest = "{" + + "__type(name: \"_edges__playerStatsWithView\") {" + + " name " + + " fields {" + + " name " + + " type {" + + " name" + + " fields {" + + " name " + + " type {" + + " name " + + " fields {" + + " name" + + " }" + + " }" + + " }" + + " }" + + " }" + + "}" + + "}"; + + String expected = loadGraphQLResponse("testGraphQLSchema.json"); + + runQueryWithExpectedResult(graphQLRequest, expected); + } + @Test public void basicAggregationTest() throws Exception { String graphQLRequest = document( @@ -938,4 +969,10 @@ private JsonNode toJsonNode(String query, Map variables) { } return graphqlNode; } + + public String loadGraphQLResponse(String fileName) throws IOException { + try (InputStream in = AggregationDataStoreIntegrationTest.class.getResourceAsStream("/graphql/responses/" + fileName)) { + return FileUtils.readFully(new InputStreamReader(in)); + } + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json b/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json new file mode 100644 index 0000000000..b1d12f5f94 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json @@ -0,0 +1,129 @@ +{ + "data": { + "__type": { + "name": "_edges__playerStatsWithView", + "fields": [ + { + "name": "node", + "type": { + "name": "_node__playerStatsWithView", + "fields": [ + { + "name": "id", + "type": { + "name": "DeferredID", + "fields": null + } + }, + { + "name": "countryIsoCode", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "countryViewIsoCode", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "countryViewRelationshipIsoCode", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "countryViewViewIsoCode", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "highScore", + "type": { + "name": "Long", + "fields": null + } + }, + { + "name": "lowScore", + "type": { + "name": "Long", + "fields": null + } + }, + { + "name": "overallRating", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "recordedDate", + "type": { + "name": "Date", + "fields": null + } + }, + { + "name": "subCountryIsoCode", + "type": { + "name": "String", + "fields": null + } + }, + { + "name": "country", + "type": { + "name": "country", + "fields": [ + { + "name": "edges" + }, + { + "name": "pageInfo" + } + ] + } + }, + { + "name": "player", + "type": { + "name": "player", + "fields": [ + { + "name": "edges" + }, + { + "name": "pageInfo" + } + ] + } + }, + { + "name": "subCountry", + "type": { + "name": "subCountry", + "fields": [ + { + "name": "edges" + }, + { + "name": "pageInfo" + } + ] + } + } + ] + } + } + ] + } + } +} diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java index 640aa98c6a..e62a6d33c7 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java @@ -131,7 +131,12 @@ public ModelBuilder(EntityDictionary dictionary, DataFetcher dataFetcher) { inputObjectRegistry = new HashMap<>(); queryObjectRegistry = new HashMap<>(); connectionObjectRegistry = new HashMap<>(); - excludedEntities = new HashSet<>(); + + // non-JPA entities can't be exposed as relationship as they don't have unique id field, + // but can still be queried as root classes. + excludedEntities = dictionary.getBindings().stream() + .filter(cls -> !dictionary.isJPAEntity(cls)) + .collect(Collectors.toSet()); } public void withExcludedEntities(Set> excludedEntities) { From a111864985e703dc709b035ac66a73a250902caf Mon Sep 17 00:00:00 2001 From: hchen04 Date: Tue, 11 Feb 2020 16:41:45 -0600 Subject: [PATCH 09/16] hide joins --- .../com/yahoo/elide/annotation/Hidden.java | 21 ++++++++++++++++++ .../yahoo/elide/core/EntityDictionary.java | 22 ++++++++++++++++--- .../aggregation/metadata/models/Table.java | 5 ++--- .../aggregation/example/CountryView.java | 2 ++ .../example/PlayerStatsWithView.java | 2 ++ .../com/yahoo/elide/graphql/ModelBuilder.java | 1 + 6 files changed, 47 insertions(+), 6 deletions(-) create mode 100644 elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java new file mode 100644 index 0000000000..0e32cce036 --- /dev/null +++ b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java @@ -0,0 +1,21 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.annotation; + +import static java.lang.annotation.ElementType.FIELD; +import static java.lang.annotation.ElementType.METHOD; +import static java.lang.annotation.RetentionPolicy.RUNTIME; + +import java.lang.annotation.Retention; +import java.lang.annotation.Target; + +/** + * Hide a field from Elide Apis but bind in dictionary. + */ +@Target({METHOD, FIELD}) +@Retention(RUNTIME) +public @interface Hidden { +} diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index 5c5f089ecb..4aa073f183 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -12,6 +12,7 @@ import com.yahoo.elide.annotation.ComputedAttribute; import com.yahoo.elide.annotation.ComputedRelationship; import com.yahoo.elide.annotation.Exclude; +import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.MappedInterface; import com.yahoo.elide.annotation.NonTransferable; @@ -32,7 +33,6 @@ import com.google.common.collect.HashBiMap; import com.google.common.collect.Maps; import com.google.common.collect.Sets; - import org.antlr.v4.runtime.tree.ParseTree; import org.apache.commons.lang3.StringUtils; @@ -62,7 +62,6 @@ import java.util.concurrent.CopyOnWriteArrayList; import java.util.function.Function; import java.util.stream.Collectors; - import javax.persistence.AccessType; import javax.persistence.CascadeType; import javax.persistence.Column; @@ -482,7 +481,9 @@ public Map> getCheckMappings() { * @return List of attribute names for entity */ public List getAttributes(Class entityClass) { - return getEntityBinding(entityClass).attributes; + return getEntityBinding(entityClass).attributes.stream() + .filter(attr -> getAttributeOrRelationAnnotation(entityClass, Hidden.class, attr) == null) + .collect(Collectors.toList()); } /** @@ -1144,6 +1145,21 @@ private boolean isClassBound(Class objClass) { return (entityBindings.getOrDefault(objClass, EMPTY_BINDING) != EMPTY_BINDING); } + /** + * Check whether a class is a JPA entity + * + * @param objClass class + * @return True if it is a JPA entity + */ + public final boolean isJPAEntity(Class objClass) { + try { + lookupEntityClass(objClass); + return true; + } catch (IllegalArgumentException e) { + return false; + } + } + /** * Retrieve the accessible object for a field from a target object. * diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index 8bac19a3dd..cf7ca18789 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -6,6 +6,7 @@ package com.yahoo.elide.datastores.aggregation.metadata.models; import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; +import static com.yahoo.elide.datastores.aggregation.metadata.models.Column.getDataType; import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; @@ -14,7 +15,6 @@ import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; -import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.enums.TableTag; import lombok.Data; @@ -105,8 +105,7 @@ public Table(Class cls, EntityDictionary dictionary) { */ private Set constructColumns(Class cls, EntityDictionary dictionary) { Set columns = dictionary.getAllFields(cls).stream() - .filter(field -> !MetaDataStore.isTableJoin(cls, field, dictionary)) - .filter(field -> Column.getDataType(cls, field, dictionary) != null) + .filter(field -> getDataType(cls, field, dictionary) != null) .map(field -> { if (isMetricField(dictionary, cls, field)) { return constructMetric(cls, field, dictionary); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java index 96fcc23fad..211d624c5d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -5,6 +5,7 @@ */ package com.yahoo.elide.datastores.aggregation.example; +import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; @@ -32,6 +33,7 @@ public class CountryView { private CountryViewNested nestedView; + @Hidden @Join("%from.id = %join.id") public CountryViewNested getNestedView() { return nestedView; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java index c6d437e919..922701967c 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -5,6 +5,7 @@ */ package com.yahoo.elide.datastores.aggregation.example; +import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; @@ -200,6 +201,7 @@ public void setSubCountryIsoCode(String isoCode) { this.subCountryIsoCode = isoCode; } + @Hidden @Join("%from.country_id = %join.id") public CountryView getCountryView() { return countryView; diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java index e62a6d33c7..fcf7411926 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java @@ -11,6 +11,7 @@ import static graphql.schema.GraphQLInputObjectField.newInputObjectField; import static graphql.schema.GraphQLObjectType.newObject; +import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RelationshipType; From 391858568adc1a52f7f343fecfbfebffd0c51a6a Mon Sep 17 00:00:00 2001 From: hchen04 Date: Wed, 12 Feb 2020 11:08:52 -0600 Subject: [PATCH 10/16] refactor hidden --- .../com/yahoo/elide/annotation/Hidden.java | 21 ----- .../elide/contrib/swagger/SwaggerBuilder.java | 4 +- .../com/yahoo/elide/core/EntityBinding.java | 88 +++++++++++++++---- .../yahoo/elide/core/EntityDictionary.java | 60 ++++++++++--- .../main/java/com/yahoo/elide/core/Path.java | 55 ++++++++---- .../elide/core/filter/FilterPredicate.java | 43 +-------- .../com/yahoo/elide/utils/TypeHelper.java | 48 +++++++++- .../aggregation/AggregationDataStore.java | 7 +- .../datastores/aggregation/core/JoinPath.java | 40 ++------- .../aggregation/metadata/MetaDataStore.java | 6 +- .../aggregation/metadata/enums/ColumnTag.java | 14 --- .../aggregation/metadata/enums/TableTag.java | 13 --- .../aggregation/metadata/models/Column.java | 3 +- .../aggregation/metadata/models/Table.java | 7 +- .../queryengines/sql/SQLQueryConstructor.java | 4 +- .../queryengines/sql/SQLQueryEngine.java | 5 +- .../aggregation/example/CountryView.java | 2 - .../example/PlayerStatsWithView.java | 2 - .../elide/core/filter/FilterTranslator.java | 3 +- .../hql/AbstractHQLQueryBuilder.java | 19 ++-- .../hql/RootCollectionFetchQueryBuilder.java | 4 +- .../RootCollectionPageTotalsQueryBuilder.java | 4 +- .../hql/SubCollectionFetchQueryBuilder.java | 6 +- .../SubCollectionPageTotalsQueryBuilder.java | 7 +- .../multiplex/MultiplexManager.java | 15 ++-- .../datastores/search/SearchDataStore.java | 2 +- .../com/yahoo/elide/graphql/ModelBuilder.java | 10 +-- 27 files changed, 266 insertions(+), 226 deletions(-) delete mode 100644 elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java diff --git a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java b/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java deleted file mode 100644 index 0e32cce036..0000000000 --- a/elide-annotations/src/main/java/com/yahoo/elide/annotation/Hidden.java +++ /dev/null @@ -1,21 +0,0 @@ -/* - * Copyright 2020, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.annotation; - -import static java.lang.annotation.ElementType.FIELD; -import static java.lang.annotation.ElementType.METHOD; -import static java.lang.annotation.RetentionPolicy.RUNTIME; - -import java.lang.annotation.Retention; -import java.lang.annotation.Target; - -/** - * Hide a field from Elide Apis but bind in dictionary. - */ -@Target({METHOD, FIELD}) -@Retention(RUNTIME) -public @interface Hidden { -} diff --git a/elide-contrib/elide-swagger/src/main/java/com/yahoo/elide/contrib/swagger/SwaggerBuilder.java b/elide-contrib/elide-swagger/src/main/java/com/yahoo/elide/contrib/swagger/SwaggerBuilder.java index ca8f1798ed..643f85fb98 100644 --- a/elide-contrib/elide-swagger/src/main/java/com/yahoo/elide/contrib/swagger/SwaggerBuilder.java +++ b/elide-contrib/elide-swagger/src/main/java/com/yahoo/elide/contrib/swagger/SwaggerBuilder.java @@ -677,9 +677,9 @@ public Swagger build() { converters.addConverter(new JsonApiModelResolver(dictionary)); if (allClasses.isEmpty()) { - allClasses = dictionary.getBindings(); + allClasses = dictionary.getBoundClasses(); } else { - allClasses = Sets.intersection(dictionary.getBindings(), allClasses); + allClasses = Sets.intersection(dictionary.getBoundClasses(), allClasses); if (allClasses.isEmpty()) { throw new IllegalArgumentException("None of the provided classes are exported by Elide"); } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java index 79ae2a9f43..ec33b58492 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityBinding.java @@ -98,8 +98,8 @@ public class EntityBinding { private AccessType accessType; public final EntityPermissions entityPermissions; - public final List attributes; - public final List relationships; + public final List apiAttributes; + public final List apiRelationships; public final List> inheritedTypes; public final ConcurrentLinkedDeque attributesDeque = new ConcurrentLinkedDeque<>(); public final ConcurrentLinkedDeque relationshipsDeque = new ConcurrentLinkedDeque<>(); @@ -125,8 +125,8 @@ public class EntityBinding { private EntityBinding() { jsonApiType = null; entityName = null; - attributes = new ArrayList<>(); - relationships = new ArrayList<>(); + apiAttributes = new ArrayList<>(); + apiRelationships = new ArrayList<>(); inheritedTypes = new ArrayList<>(); idField = null; idType = null; @@ -135,7 +135,35 @@ private EntityBinding() { idGenerated = false; } - public EntityBinding(EntityDictionary dictionary, Class cls, String type, String name) { + /** + * Constructor + * + * @param dictionary Dictionary to use + * @param cls Entity class + * @param type Declared Elide type name + * @param name Declared Entity name + */ + public EntityBinding(EntityDictionary dictionary, + Class cls, + String type, + String name) { + this(dictionary, cls, type, name, new HashSet<>()); + } + + /** + * Constructor + * + * @param dictionary Dictionary to use + * @param cls Entity class + * @param type Declared Elide type name + * @param name Declared Entity name + * @param hiddenAnnotations Annotations for hiding a field in API + */ + public EntityBinding(EntityDictionary dictionary, + Class cls, + String type, + String name, + Set> hiddenAnnotations) { entityClass = cls; jsonApiType = type; entityName = name; @@ -182,10 +210,10 @@ public EntityBinding(EntityDictionary dictionary, Class cls, String type, Str fieldOrMethodList.addAll(getInstanceMembers(cls.getMethods())); } - bindEntityFields(cls, type, fieldOrMethodList); + bindEntityFields(cls, type, fieldOrMethodList, hiddenAnnotations); - attributes = dequeToList(attributesDeque); - relationships = dequeToList(relationshipsDeque); + apiAttributes = dequeToList(attributesDeque); + apiRelationships = dequeToList(relationshipsDeque); entityPermissions = new EntityPermissions(dictionary, cls, fieldOrMethodList); } @@ -236,8 +264,11 @@ public List getAllFields() { * @param cls Class type to bind fields * @param type JSON API type identifier * @param fieldOrMethodList List of fields and methods on entity + * @param hiddenAnnotations Annotations for hiding a field in API */ - private void bindEntityFields(Class cls, String type, Collection fieldOrMethodList) { + private void bindEntityFields(Class cls, String type, + Collection fieldOrMethodList, + Set> hiddenAnnotations) { for (AccessibleObject fieldOrMethod : fieldOrMethodList) { bindTriggerIfPresent(OnCreatePreSecurity.class, fieldOrMethod); bindTriggerIfPresent(OnDeletePreSecurity.class, fieldOrMethod); @@ -270,7 +301,9 @@ private void bindEntityFields(Class cls, String type, Collection dequeToList(final Deque deque) { * Bind an attribute or relationship. * * @param fieldOrMethod Field or method to bind + * @param isHidden Whether this field is hidden from API */ - private void bindAttrOrRelation(AccessibleObject fieldOrMethod) { + private void bindAttrOrRelation(AccessibleObject fieldOrMethod, boolean isHidden) { boolean isRelation = RELATIONSHIP_TYPES.stream().anyMatch(fieldOrMethod::isAnnotationPresent); String fieldName = getFieldName(fieldOrMethod); @@ -339,13 +373,21 @@ private void bindAttrOrRelation(AccessibleObject fieldOrMethod) { } if (isRelation) { - bindRelation(fieldOrMethod, fieldName, fieldType); + bindRelation(fieldOrMethod, fieldName, fieldType, isHidden); } else { - bindAttr(fieldOrMethod, fieldName, fieldType); + bindAttr(fieldOrMethod, fieldName, fieldType, isHidden); } } - private void bindRelation(AccessibleObject fieldOrMethod, String fieldName, Class fieldType) { + /** + * Bind a relationship to current class + * + * @param fieldOrMethod Field or method to bind + * @param fieldName Field name + * @param fieldType Field type + * @param isHidden Whether this field is hidden from API + */ + private void bindRelation(AccessibleObject fieldOrMethod, String fieldName, Class fieldType, boolean isHidden) { boolean manyToMany = fieldOrMethod.isAnnotationPresent(ManyToMany.class); boolean manyToOne = fieldOrMethod.isAnnotationPresent(ManyToOne.class); boolean oneToMany = fieldOrMethod.isAnnotationPresent(OneToMany.class); @@ -387,13 +429,25 @@ private void bindRelation(AccessibleObject fieldOrMethod, String fieldName, Clas relationshipToInverse.put(fieldName, mappedBy); relationshipToCascadeTypes.put(fieldName, cascadeTypes); - relationshipsDeque.push(fieldName); + if (!isHidden) { + relationshipsDeque.push(fieldName); + } fieldsToValues.put(fieldName, fieldOrMethod); fieldsToTypes.put(fieldName, fieldType); } - private void bindAttr(AccessibleObject fieldOrMethod, String fieldName, Class fieldType) { - attributesDeque.push(fieldName); + /** + * Bind an attribute to current class + * + * @param fieldOrMethod Field or method to bind + * @param fieldName Field name + * @param fieldType Field type + * @param isHidden Whether this field is hidden from API + */ + private void bindAttr(AccessibleObject fieldOrMethod, String fieldName, Class fieldType, boolean isHidden) { + if (!isHidden) { + attributesDeque.push(fieldName); + } fieldsToValues.put(fieldName, fieldOrMethod); fieldsToTypes.put(fieldName, fieldType); } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java index 4aa073f183..c2306c5554 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java @@ -12,7 +12,6 @@ import com.yahoo.elide.annotation.ComputedAttribute; import com.yahoo.elide.annotation.ComputedRelationship; import com.yahoo.elide.annotation.Exclude; -import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.MappedInterface; import com.yahoo.elide.annotation.NonTransferable; @@ -457,13 +456,22 @@ public AccessType getAccessType(Class entityClass) { return getEntityBinding(entityClass).getAccessType(); } + /** + * Get all bound classes. + * + * @return the bound classes + */ + public Set> getBoundClasses() { + return entityBindings.keySet(); + } + /** * Get all bindings. * * @return the bindings */ - public Set> getBindings() { - return entityBindings.keySet(); + public Set getBindings() { + return new HashSet<>(entityBindings.values()); } /** @@ -481,9 +489,7 @@ public Map> getCheckMappings() { * @return List of attribute names for entity */ public List getAttributes(Class entityClass) { - return getEntityBinding(entityClass).attributes.stream() - .filter(attr -> getAttributeOrRelationAnnotation(entityClass, Hidden.class, attr) == null) - .collect(Collectors.toList()); + return getEntityBinding(entityClass).apiAttributes; } /** @@ -503,7 +509,7 @@ public List getAttributes(Object entity) { * @return List of relationship names for entity */ public List getRelationships(Class entityClass) { - return getEntityBinding(entityClass).relationships; + return getEntityBinding(entityClass).apiRelationships; } /** @@ -524,7 +530,7 @@ public List getRelationships(Object entity) { */ public List getElideBoundRelationships(Class entityClass) { return getRelationships(entityClass).stream() - .filter(relationName -> getBindings().contains(getParameterizedType(entityClass, relationName))) + .filter(relationName -> getBoundClasses().contains(getParameterizedType(entityClass, relationName))) .collect(Collectors.toList()); } @@ -843,6 +849,16 @@ public boolean isTransferable(Class entityClass) { * @param cls Entity bean class */ public void bindEntity(Class cls) { + bindEntity(cls, new HashSet<>()); + } + + /** + * Add given Entity bean to dictionary. + * + * @param cls Entity bean class + * @param hiddenAnnotations Annotations for hiding a field in API + */ + public void bindEntity(Class cls, Set> hiddenAnnotations) { Class declaredClass = lookupIncludeClass(cls); if (declaredClass == null) { @@ -873,7 +889,29 @@ public void bindEntity(Class cls) { } bindJsonApiToEntity.put(type, declaredClass); - entityBindings.put(declaredClass, new EntityBinding(this, declaredClass, type, name)); + entityBindings.put(declaredClass, new EntityBinding(this, declaredClass, type, name, hiddenAnnotations)); + if (include.rootLevel()) { + bindEntityRoots.add(declaredClass); + } + } + + /** + * Add an EntityBinding instance to dictionary. + * + * @param entityBinding EntityBinding instance + */ + public void bindEntity(EntityBinding entityBinding) { + Class declaredClass = entityBinding.entityClass; + + if (isClassBound(declaredClass)) { + //Ignore duplicate bindings. + return; + } + + Include include = (Include) getFirstAnnotation(declaredClass, Collections.singletonList(Include.class)); + + bindJsonApiToEntity.put(entityBinding.jsonApiType, declaredClass); + entityBindings.put(declaredClass, entityBinding); if (include.rootLevel()) { bindEntityRoots.add(declaredClass); } @@ -1211,11 +1249,11 @@ public Set getFieldsOfType(Class targetClass, Class targetType) { } public boolean isRelation(Class entityClass, String relationName) { - return getEntityBinding(entityClass).relationships.contains(relationName); + return getEntityBinding(entityClass).apiRelationships.contains(relationName); } public boolean isAttribute(Class entityClass, String attributeName) { - return getEntityBinding(entityClass).attributes.contains(attributeName); + return getEntityBinding(entityClass).apiAttributes.contains(attributeName); } /** diff --git a/elide-core/src/main/java/com/yahoo/elide/core/Path.java b/elide-core/src/main/java/com/yahoo/elide/core/Path.java index bea0a157f1..e7bd2ed405 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/Path.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/Path.java @@ -5,6 +5,10 @@ */ package com.yahoo.elide.core; +import static com.yahoo.elide.core.EntityDictionary.getSimpleName; +import static com.yahoo.elide.utils.TypeHelper.appendAlias; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; + import com.yahoo.elide.core.exceptions.InvalidValueException; import com.google.common.collect.ImmutableList; @@ -25,7 +29,6 @@ @EqualsAndHashCode public class Path { private static final String PERIOD = "."; - private static final String UNDERSCORE = "_"; @Getter private List pathElements; /** @@ -50,15 +53,29 @@ public Path(List pathElements) { } public Path(Class entityClass, EntityDictionary dictionary, String dotSeparatedPath) { + pathElements = resolvePathElements(entityClass, dictionary, dotSeparatedPath); + } + + /** + * Resolve a dot separated path into list of path elements. + * + * @param entityClass root class e.g. "foo" + * @param dictionary dictionary + * @param dotSeparatedPath path e.g. "bar.baz" + * @return list of path elements e.g. ["foo.bar", "bar.baz"] + */ + private List resolvePathElements(Class entityClass, + EntityDictionary dictionary, + String dotSeparatedPath) { List elements = new ArrayList<>(); String[] fieldNames = dotSeparatedPath.split("\\."); Class currentClass = entityClass; for (String fieldName : fieldNames) { - if (dictionary.isRelation(currentClass, fieldName)) { - Class relationClass = dictionary.getParameterizedType(currentClass, fieldName); - elements.add(new PathElement(currentClass, relationClass, fieldName)); - currentClass = relationClass; + if (needNavigation(currentClass, fieldName, dictionary)) { + Class joinClass = dictionary.getParameterizedType(currentClass, fieldName); + elements.add(new PathElement(currentClass, joinClass, fieldName)); + currentClass = joinClass; } else if (dictionary.isAttribute(currentClass, fieldName) || fieldName.equals(dictionary.getIdFieldName(entityClass))) { Class attributeClass = dictionary.getType(currentClass, fieldName); @@ -70,7 +87,20 @@ public Path(Class entityClass, EntityDictionary dictionary, String dotSeparat throw new InvalidValueException(alias + " doesn't contain the field " + fieldName); } } - pathElements = ImmutableList.copyOf(elements); + + return ImmutableList.copyOf(elements); + } + + /** + * Check whether a field need navigation to another entity. + * + * @param entityClass entity class + * @param fieldName field name + * @param dictionary dictionary + * @return True if the field requires navigation. + */ + protected boolean needNavigation(Class entityClass, String fieldName, EntityDictionary dictionary) { + return dictionary.isRelation(entityClass, fieldName); } public Optional lastElement() { @@ -95,23 +125,14 @@ public String getAlias() { } PathElement previous = pathElements.get(pathElements.size() - 2); - return getTypeAlias(previous.getType()) + UNDERSCORE + previous.getFieldName(); + return appendAlias(getTypeAlias(previous.getType()), previous.getFieldName()); } @Override public String toString() { return pathElements.size() == 0 ? "EMPTY" : pathElements.stream() - .map(e -> '[' + EntityDictionary.getSimpleName(e.getType()) + ']' + PERIOD + e.getFieldName()) + .map(e -> '[' + getSimpleName(e.getType()) + ']' + PERIOD + e.getFieldName()) .collect(Collectors.joining("/")); } - - /** - * Convert a class name into a hibernate friendly name. - * @param type The type to alias - * @return type name alias that will likely not conflict with other types or with reserved keywords. - */ - public static String getTypeAlias(Class type) { - return type.getCanonicalName().replace(PERIOD, UNDERSCORE); - } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java index 70446b5f19..7c7c835809 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/filter/FilterPredicate.java @@ -104,47 +104,6 @@ public FilterPredicate scopedBy(PathElement scope) { return new FilterPredicate(new Path(pathElements), operator, values); } - /** - * Generate alias for representing a relationship path which dose not include the last field name. - * The path would start with the class alias of the first element, and then each field would append "_fieldName" to - * the result. - * The last field would not be included as that's not a part of the relationship path. - * - * @param path path that represents a relationship chain - * @return relationship path alias, i.e. foo.bar.baz would be foo_bar - */ - public static String getPathAlias(Path path) { - List elements = path.getPathElements(); - String alias = getTypeAlias(elements.get(0).getType()); - - for (int i = 0; i < elements.size() - 1; i++) { - alias = appendAlias(alias, elements.get(i).getFieldName()); - } - - return alias; - } - - /** - * Append a new field to a parent alias to get new alias. - * - * @param parentAlias parent path alias - * @param fieldName field name - * @return alias for the field - */ - public static String appendAlias(String parentAlias, String fieldName) { - return parentAlias + "_" + fieldName; - } - - /** - * Build an HQL friendly alias for a class. - * - * @param type The type to alias - * @return type name alias that will likely not conflict with other types or with reserved keywords. - */ - public static String getTypeAlias(Class type) { - return type.getCanonicalName().replace(PERIOD, UNDERSCORE); - } - public Class getEntityType() { List elements = path.getPathElements(); PathElement first = elements.get(0); @@ -180,8 +139,8 @@ public String toString() { for (PathElement element : elements) { formattedPath.append(PERIOD).append(element.getFieldName()); - } + return formattedPath.append(' ').append(operator).append(' ').append(values).toString(); } diff --git a/elide-core/src/main/java/com/yahoo/elide/utils/TypeHelper.java b/elide-core/src/main/java/com/yahoo/elide/utils/TypeHelper.java index 478fa0f334..55fdeb4751 100644 --- a/elide-core/src/main/java/com/yahoo/elide/utils/TypeHelper.java +++ b/elide-core/src/main/java/com/yahoo/elide/utils/TypeHelper.java @@ -6,14 +6,19 @@ package com.yahoo.elide.utils; +import com.yahoo.elide.core.Path; + import com.google.common.collect.Sets; +import java.util.List; import java.util.Set; /** - * Utilities for checking classes and primitive types. + * Utilities for handling types and aliases. */ public class TypeHelper { + private static final String UNDERSCORE = "_"; + private static final String PERIOD = "."; private static final Set> PRIMITIVE_NUMBER_TYPES = Sets .newHashSet(short.class, int.class, long.class, float.class, double.class); @@ -26,4 +31,45 @@ public class TypeHelper { public static boolean isPrimitiveNumberType(Class type) { return PRIMITIVE_NUMBER_TYPES.contains(type); } + + /** + * Generate alias for representing a relationship path which dose not include the last field name. + * The path would start with the class alias of the first element, and then each field would append "_fieldName" to + * the result. + * The last field would not be included as that's not a part of the relationship path. + * + * @param path path that represents a relationship chain + * @return relationship path alias, i.e. foo.bar.baz would be foo_bar + */ + public static String getPathAlias(Path path) { + List elements = path.getPathElements(); + String alias = getTypeAlias(elements.get(0).getType()); + + for (int i = 0; i < elements.size() - 1; i++) { + alias = appendAlias(alias, elements.get(i).getFieldName()); + } + + return alias; + } + + /** + * Append a new field to a parent alias to get new alias. + * + * @param parentAlias parent path alias + * @param fieldName field name + * @return alias for the field + */ + public static String appendAlias(String parentAlias, String fieldName) { + return parentAlias + UNDERSCORE + fieldName; + } + + /** + * Build an query friendly alias for a class. + * + * @param type The type to alias + * @return type name alias that will likely not conflict with other types or with reserved keywords. + */ + public static String getTypeAlias(Class type) { + return type.getCanonicalName().replace(PERIOD, UNDERSCORE); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java index b4c43fc6f6..11cdf8b02b 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -9,6 +9,7 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; @@ -17,6 +18,7 @@ import java.lang.annotation.Annotation; import java.util.Arrays; +import java.util.Collections; import java.util.List; /** @@ -41,9 +43,10 @@ public AggregationDataStore(QueryEngine queryEngine) { */ @Override public void populateEntityDictionary(EntityDictionary dictionary) { - for (Class cls : AGGREGATION_STORE_CLASSES) { + for (Class annotation : AGGREGATION_STORE_CLASSES) { // bind non-jpa entity tables - ClassScanner.getAnnotatedClasses(cls).forEach(dictionary::bindEntity); + ClassScanner.getAnnotatedClasses(annotation) + .forEach(cls -> dictionary.bindEntity(cls, Collections.singleton(Join.class))); } /* Add 'grain' argument to each TimeDimensionColumn */ diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java index 448aa5efac..15b2f0de19 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/JoinPath.java @@ -5,50 +5,22 @@ */ package com.yahoo.elide.datastores.aggregation.core; -import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isTableJoin; - import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; -import com.yahoo.elide.core.exceptions.InvalidValueException; import com.yahoo.elide.datastores.aggregation.annotation.Join; - -import com.google.common.collect.ImmutableList; - -import java.util.ArrayList; -import java.util.List; +import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; /** * JoinPath extends {@link Path} to allow navigation through {@link Join} annotation. */ public class JoinPath extends Path { public JoinPath(Class entityClass, EntityDictionary dictionary, String dotSeparatedPath) { - super(resolveJoinPathElements(entityClass, dictionary, dotSeparatedPath)); + super(entityClass, dictionary, dotSeparatedPath); } - private static List resolveJoinPathElements(Class entityClass, - EntityDictionary dictionary, - String dotSeparatedPath) { - List elements = new ArrayList<>(); - String[] fieldNames = dotSeparatedPath.split("\\."); - - Class currentClass = entityClass; - for (String fieldName : fieldNames) { - if (dictionary.isRelation(currentClass, fieldName) || isTableJoin(currentClass, fieldName, dictionary)) { - Class joinClass = dictionary.getParameterizedType(currentClass, fieldName); - elements.add(new PathElement(currentClass, joinClass, fieldName)); - currentClass = joinClass; - } else if (dictionary.isAttribute(currentClass, fieldName) - || fieldName.equals(dictionary.getIdFieldName(entityClass))) { - Class attributeClass = dictionary.getType(currentClass, fieldName); - elements.add(new PathElement(currentClass, attributeClass, fieldName)); - } else if ("this".equals(fieldName)) { - elements.add(new PathElement(currentClass, null, fieldName)); - } else { - String alias = dictionary.getJsonAliasFor(currentClass); - throw new InvalidValueException(alias + " doesn't contain the field " + fieldName); - } - } - - return ImmutableList.copyOf(elements); + @Override + protected boolean needNavigation(Class entityClass, String fieldName, EntityDictionary dictionary) { + return dictionary.isRelation(entityClass, fieldName) + || MetaDataStore.isTableJoin(entityClass, fieldName, dictionary); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index a25a4c1662..2db399d844 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -29,6 +29,7 @@ import java.lang.annotation.Annotation; import java.util.Arrays; +import java.util.Collections; import java.util.HashMap; import java.util.List; import java.util.Set; @@ -56,12 +57,13 @@ public MetaDataStore() { // bind external data models in the package this.modelsToBind = ClassScanner.getAnnotatedClasses(METADATA_STORE_ANNOTATIONS); - modelsToBind.forEach(dictionary::bindEntity); + modelsToBind.forEach(cls -> dictionary.bindEntity(cls, Collections.singleton(Join.class))); } @Override public void populateEntityDictionary(EntityDictionary dictionary) { - ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()).forEach(dictionary::bindEntity); + ClassScanner.getAllClasses(META_DATA_PACKAGE.getName()) + .forEach(cls -> dictionary.bindEntity(cls, Collections.singleton(Join.class))); } /** diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java deleted file mode 100644 index 2e8f22193d..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ColumnTag.java +++ /dev/null @@ -1,14 +0,0 @@ -/* - * Copyright 2020, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.enums; - -/** - * Tag attached to fields. - */ -public enum ColumnTag { - DISPLAY, - GROUPABLE -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java deleted file mode 100644 index 66f55a6b5f..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/TableTag.java +++ /dev/null @@ -1,13 +0,0 @@ -/* - * Copyright 2020, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.enums; - -/** - * Tag attached to fields. - */ -public enum TableTag { - DISPLAY -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java index 5cfe80224b..b1b1fc27a3 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -8,7 +8,6 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.annotation.Meta; -import com.yahoo.elide.datastores.aggregation.metadata.enums.ColumnTag; import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; import lombok.Data; @@ -45,7 +44,7 @@ public abstract class Column { private DataType dataType; @ToString.Exclude - private Set columnTags; + private Set columnTags; protected Column(Class tableClass, String fieldName, EntityDictionary dictionary) { this.tableName = dictionary.getJsonAliasFor(tableClass); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index cf7ca18789..17247e5828 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -15,11 +15,11 @@ import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; -import com.yahoo.elide.datastores.aggregation.metadata.enums.TableTag; import lombok.Data; import lombok.ToString; +import java.util.HashSet; import java.util.Map; import java.util.Set; import java.util.function.Function; @@ -58,19 +58,20 @@ public class Table { private Set dimensions; @ToString.Exclude - private Set tableTags; + private Set tableTags; @Exclude @ToString.Exclude private final Map columnMap; public Table(Class cls, EntityDictionary dictionary) { - if (!dictionary.getBindings().contains(cls)) { + if (!dictionary.getBoundClasses().contains(cls)) { throw new IllegalArgumentException( String.format("Table class {%s} is not defined in dictionary.", cls)); } this.name = dictionary.getJsonAliasFor(cls); + this.tableTags = new HashSet<>(); this.columns = constructColumns(cls, dictionary); this.columnMap = this.columns.stream().collect(Collectors.toMap(Column::getName, Function.identity())); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java index 21e2ce81ef..4d1fcf9107 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -5,11 +5,11 @@ */ package com.yahoo.elide.datastores.aggregation.queryengines.sql; -import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; -import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isTableJoin; import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.generateColumnReference; import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; +import static com.yahoo.elide.utils.TypeHelper.appendAlias; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java index 168ef1007f..8ececad924 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -5,7 +5,8 @@ */ package com.yahoo.elide.datastores.aggregation.queryengines.sql; -import static com.yahoo.elide.core.filter.FilterPredicate.getPathAlias; +import static com.yahoo.elide.utils.TypeHelper.getPathAlias; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; @@ -276,6 +277,6 @@ public static String generateColumnReference(Path path, EntityDictionary diction * @return alias */ public static String getClassAlias(Class entityClass) { - return FilterPredicate.getTypeAlias(entityClass); + return getTypeAlias(entityClass); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java index 211d624c5d..96fcc23fad 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -5,7 +5,6 @@ */ package com.yahoo.elide.datastores.aggregation.example; -import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; @@ -33,7 +32,6 @@ public class CountryView { private CountryViewNested nestedView; - @Hidden @Join("%from.id = %join.id") public CountryViewNested getNestedView() { return nestedView; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java index 922701967c..c6d437e919 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -5,7 +5,6 @@ */ package com.yahoo.elide.datastores.aggregation.example; -import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; @@ -201,7 +200,6 @@ public void setSubCountryIsoCode(String isoCode) { this.subCountryIsoCode = isoCode; } - @Hidden @Join("%from.country_id = %join.id") public CountryView getCountryView() { return countryView; diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java index 5609d4369e..72dc6798c1 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/filter/FilterTranslator.java @@ -25,6 +25,7 @@ import static com.yahoo.elide.core.filter.Operator.PREFIX; import static com.yahoo.elide.core.filter.Operator.PREFIX_CASE_INSENSITIVE; import static com.yahoo.elide.core.filter.Operator.TRUE; +import static com.yahoo.elide.utils.TypeHelper.getPathAlias; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.exceptions.InvalidPredicateException; @@ -56,7 +57,7 @@ public class FilterTranslator implements FilterOperation { public static final Function GENERATE_HQL_COLUMN_NO_ALIAS = FilterPredicate::getFieldPath; public static final Function GENERATE_HQL_COLUMN_WITH_ALIAS = - (predicate) -> FilterPredicate.getPathAlias(predicate.getPath()) + "." + predicate.getField(); + (predicate) -> getPathAlias(predicate.getPath()) + "." + predicate.getField(); static { predicateOverrides = new HashMap<>(); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java index e03e29b795..60762e6ba5 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/AbstractHQLQueryBuilder.java @@ -5,8 +5,8 @@ */ package com.yahoo.elide.core.hibernate.hql; -import static com.yahoo.elide.core.filter.FilterPredicate.appendAlias; -import static com.yahoo.elide.core.filter.FilterPredicate.getTypeAlias; +import static com.yahoo.elide.utils.TypeHelper.appendAlias; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; @@ -18,6 +18,7 @@ import com.yahoo.elide.core.hibernate.Session; import com.yahoo.elide.request.Pagination; import com.yahoo.elide.request.Sorting; + import org.apache.commons.lang3.StringUtils; import java.util.ArrayList; @@ -28,7 +29,6 @@ import java.util.Optional; import java.util.Set; import java.util.stream.Collectors; - import javax.persistence.OneToOne; /** @@ -42,7 +42,6 @@ public abstract class AbstractHQLQueryBuilder { protected Optional pagination; protected Optional filterExpression; protected static final String SPACE = " "; - protected static final String UNDERSCORE = "_"; protected static final String PERIOD = "."; protected static final String COMMA = ","; protected static final String FROM = " FROM "; @@ -227,15 +226,13 @@ protected String getSortClause(final Optional sorting, Class sortCla if (!validSortingRules.isEmpty()) { final List ordering = new ArrayList<>(); // pass over the sorting rules - validSortingRules.entrySet().stream().forEachOrdered(entry -> { - Path path = entry.getKey(); + validSortingRules.forEach((path, order) -> { - String prefix = (prefixWithAlias) ? Path.getTypeAlias(sortClass) + PERIOD : ""; + String prefix = (prefixWithAlias) ? getTypeAlias(sortClass) + PERIOD : ""; - ordering.add(prefix + path.getFieldPath() + SPACE - + (entry.getValue().equals(Sorting.SortOrder.desc) ? "desc" : "asc")); - } - ); + ordering.add(prefix + path.getFieldPath() + SPACE + + (order.equals(Sorting.SortOrder.desc) ? "desc" : "asc")); + }); sortingRules = " order by " + StringUtils.join(ordering, COMMA); } } diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java index 9978721da6..c1f01f3e64 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionFetchQueryBuilder.java @@ -5,6 +5,8 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.FilterTranslator; @@ -36,7 +38,7 @@ public RootCollectionFetchQueryBuilder(Class entityClass, @Override public Query build() { String entityName = entityClass.getCanonicalName(); - String entityAlias = FilterPredicate.getTypeAlias(entityClass); + String entityAlias = getTypeAlias(entityClass); Query query; if (filterExpression.isPresent()) { diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java index 84795e0389..9f9d9ff589 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/RootCollectionPageTotalsQueryBuilder.java @@ -5,6 +5,8 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.FilterTranslator; @@ -55,7 +57,7 @@ public AbstractHQLQueryBuilder withPossibleSorting(Optional ignored) { @Override public Query build() { String entityName = entityClass.getCanonicalName(); - String entityAlias = FilterPredicate.getTypeAlias(entityClass); + String entityAlias = getTypeAlias(entityClass); Collection predicates; diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java index 75a9ba681f..8ed38bc510 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionFetchQueryBuilder.java @@ -5,6 +5,8 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.filter.FilterPredicate; import com.yahoo.elide.core.filter.FilterTranslator; @@ -42,8 +44,8 @@ public Query build() { return null; } - String childAlias = FilterPredicate.getTypeAlias(relationship.getChildType()); - String parentAlias = FilterPredicate.getTypeAlias(relationship.getParentType()) + "__fetch"; + String childAlias = getTypeAlias(relationship.getChildType()); + String parentAlias = getTypeAlias(relationship.getParentType()) + "__fetch"; String parentName = relationship.getParentType().getCanonicalName(); String relationshipName = relationship.getRelationshipName(); diff --git a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java index 8d96cf4b20..fa586f2153 100644 --- a/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java +++ b/elide-datastore/elide-datastore-hibernate/src/main/java/com/yahoo/elide/core/hibernate/hql/SubCollectionPageTotalsQueryBuilder.java @@ -5,6 +5,9 @@ */ package com.yahoo.elide.core.hibernate.hql; +import static com.yahoo.elide.utils.TypeHelper.appendAlias; +import static com.yahoo.elide.utils.TypeHelper.getTypeAlias; + import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path.PathElement; import com.yahoo.elide.core.filter.FilterPredicate; @@ -80,8 +83,8 @@ public Query build() { String relationshipName = relationship.getRelationshipName(); //Relationship alias is Author_books - String parentAlias = FilterPredicate.getTypeAlias(parentType); - String relationshipAlias = parentAlias + UNDERSCORE + relationshipName; + String parentAlias = getTypeAlias(parentType); + String relationshipAlias = appendAlias(parentAlias, relationshipName); if (filterExpression.isPresent()) { // Copy and scope the filter expression for the join clause diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java index f9f39803d8..5978ffed11 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexManager.java @@ -7,6 +7,7 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.EntityBinding; import com.yahoo.elide.core.EntityDictionary; import lombok.AccessLevel; @@ -55,18 +56,12 @@ public void populateEntityDictionary(EntityDictionary dictionary) { EntityDictionary subordinateDictionary = new EntityDictionary(dictionary.getCheckMappings()); dataStore.populateEntityDictionary(subordinateDictionary); - for (Class cls : subordinateDictionary.getBindings()) { + for (EntityBinding binding : subordinateDictionary.getBindings()) { // route class to this database manager - this.dataStoreMap.put(cls, dataStore); + this.dataStoreMap.put(binding.entityClass, dataStore); + // bind to multiplex dictionary - dictionary.bindEntity(cls); - // copy attribute arguments - subordinateDictionary.getAttributes(cls).forEach( - attribute -> dictionary.addArgumentsToAttribute( - cls, - attribute, - subordinateDictionary.getAttributeArguments(cls, attribute)) - ); + dictionary.bindEntity(binding); } } } diff --git a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataStore.java b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataStore.java index 8e8de17ec8..402cfc907f 100644 --- a/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataStore.java +++ b/elide-datastore/elide-datastore-search/src/main/java/com/yahoo/elide/datastores/search/SearchDataStore.java @@ -57,7 +57,7 @@ public void populateEntityDictionary(EntityDictionary entityDictionary) { FullTextEntityManager em = Search.getFullTextEntityManager(entityManagerFactory.createEntityManager()); try { - for (Class entityClass : entityDictionary.getBindings()) { + for (Class entityClass : entityDictionary.getBoundClasses()) { if (entityDictionary.getAnnotation(entityClass, Indexed.class) != null) { em.createIndexer(entityClass).startAndWait(); } diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java index fcf7411926..da4d1e9883 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/ModelBuilder.java @@ -11,7 +11,6 @@ import static graphql.schema.GraphQLInputObjectField.newInputObjectField; import static graphql.schema.GraphQLObjectType.newObject; -import com.yahoo.elide.annotation.Hidden; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.RelationshipType; @@ -132,12 +131,7 @@ public ModelBuilder(EntityDictionary dictionary, DataFetcher dataFetcher) { inputObjectRegistry = new HashMap<>(); queryObjectRegistry = new HashMap<>(); connectionObjectRegistry = new HashMap<>(); - - // non-JPA entities can't be exposed as relationship as they don't have unique id field, - // but can still be queried as root classes. - excludedEntities = dictionary.getBindings().stream() - .filter(cls -> !dictionary.isJPAEntity(cls)) - .collect(Collectors.toSet()); + excludedEntities = new HashSet<>(); } public void withExcludedEntities(Set> excludedEntities) { @@ -149,7 +143,7 @@ public void withExcludedEntities(Set> excludedEntities) { * @return The built schema. */ public GraphQLSchema build() { - Set> allClasses = dictionary.getBindings(); + Set> allClasses = dictionary.getBoundClasses(); if (allClasses.isEmpty()) { throw new IllegalArgumentException("None of the provided classes are exported by Elide"); From 65fd06fad8afb5d3960e835fee1c87c6b5fca37c Mon Sep 17 00:00:00 2001 From: hchen04 Date: Fri, 14 Feb 2020 14:43:37 -0600 Subject: [PATCH 11/16] remove ant --- elide-datastore/elide-datastore-aggregation/pom.xml | 7 ------- .../integration/AggregationDataStoreIntegrationTest.java | 5 +++-- 2 files changed, 3 insertions(+), 9 deletions(-) diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml index ec275df729..c2da5362a9 100644 --- a/elide-datastore/elide-datastore-aggregation/pom.xml +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -178,13 +178,6 @@ 2.5.0 test - - - org.apache.ant - ant - 1.10.7 - test - diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java index 9486c3c9c3..6c64bafd1a 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -28,16 +28,17 @@ import com.fasterxml.jackson.databind.node.ArrayNode; import com.fasterxml.jackson.databind.node.JsonNodeFactory; import com.fasterxml.jackson.databind.node.ObjectNode; -import org.apache.tools.ant.util.FileUtils; import org.junit.jupiter.api.Disabled; import org.junit.jupiter.api.Test; import io.restassured.response.ValidatableResponse; +import java.io.BufferedReader; import java.io.IOException; import java.io.InputStream; import java.io.InputStreamReader; import java.util.Map; +import java.util.stream.Collectors; import javax.persistence.Persistence; import javax.ws.rs.core.MediaType; @@ -972,7 +973,7 @@ private JsonNode toJsonNode(String query, Map variables) { public String loadGraphQLResponse(String fileName) throws IOException { try (InputStream in = AggregationDataStoreIntegrationTest.class.getResourceAsStream("/graphql/responses/" + fileName)) { - return FileUtils.readFully(new InputStreamReader(in)); + return new BufferedReader(new InputStreamReader(in)).lines().collect(Collectors.joining("\n")); } } } From 95b1a67c0af3815b1091b4ca64330bf64f77c996 Mon Sep 17 00:00:00 2001 From: Han Chen Date: Mon, 17 Feb 2020 10:34:26 -0600 Subject: [PATCH 12/16] remove relationshp, update model (#1186) --- .../aggregation/AggregationDataStore.java | 2 +- .../datastores/aggregation/QueryEngine.java | 13 +- .../aggregation/QueryValidator.java | 13 +- .../aggregation/metadata/MetaDataStore.java | 11 - .../metadata/enums/Aggregation.java | 15 - .../aggregation/metadata/enums/ValueType.java | 34 ++- .../aggregation/metadata/models/Column.java | 40 +-- .../aggregation/metadata/models/DataType.java | 58 ---- .../metadata/models/Dimension.java | 4 +- .../metadata/models/FunctionArgument.java | 10 +- .../aggregation/metadata/models/Metric.java | 5 +- .../metadata/models/RelationshipType.java | 22 -- .../aggregation/metadata/models/Table.java | 35 ++- .../metadata/models/TimeDimension.java | 11 +- .../queryengines/AbstractEntityHydrator.java | 8 +- .../queryengines/sql/SQLQueryConstructor.java | 8 +- .../sql/metadata/SQLDimension.java | 6 +- .../queryengines/sql/metadata/SQLMetric.java | 5 +- .../queryengines/sql/metadata/SQLTable.java | 14 +- .../sql/metadata/SQLTimeDimension.java | 6 +- .../EntityProjectionTranslatorTest.java | 15 +- .../aggregation/QueryValidatorTest.java | 40 +-- .../aggregation/example/CountryView.java | 16 +- .../example/CountryViewNested.java | 4 +- .../aggregation/example/PlayerStats.java | 80 +++--- .../aggregation/example/PlayerStatsView.java | 7 +- .../example/PlayerStatsWithView.java | 19 +- .../AggregationDataStoreIntegrationTest.java | 272 ++++-------------- .../queryengines/sql/QueryEngineTest.java | 238 ++++++--------- .../queryengines/sql/SubselectTest.java | 123 +------- .../queryengines/sql/ViewTest.java | 28 +- .../graphql/responses/testGraphQLSchema.json | 49 ---- 32 files changed, 326 insertions(+), 885 deletions(-) delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java delete mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java index 11cdf8b02b..2238a89762 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -53,7 +53,7 @@ public void populateEntityDictionary(EntityDictionary dictionary) { for (Table table : queryEngine.getMetaDataStore().getMetaData(Table.class)) { for (TimeDimension timeDim : table.getColumns(TimeDimension.class)) { dictionary.addArgumentToAttribute( - dictionary.getEntityClass(table.getName()), + dictionary.getEntityClass(table.getId()), timeDim.getName(), new ArgumentType("grain", String.class)); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java index ff7a1a6b68..43e8be5cee 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -8,6 +8,7 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.DataStoreTransaction; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.exceptions.InvalidPredicateException; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; @@ -67,7 +68,6 @@ public abstract class QueryEngine { @Getter private final EntityDictionary metadataDictionary; - @Getter private final Map tables; /** @@ -81,7 +81,7 @@ public QueryEngine(MetaDataStore metaDataStore) { this.metadataDictionary = metaDataStore.getDictionary(); populateMetaData(metaDataStore); this.tables = metaDataStore.getMetaData(Table.class).stream() - .collect(Collectors.toMap(Table::getName, Functions.identity())); + .collect(Collectors.toMap(Table::getId, Functions.identity())); } /** @@ -99,6 +99,15 @@ public QueryEngine(MetaDataStore metaDataStore) { * @param metaDataStore metadata store to populate */ private void populateMetaData(MetaDataStore metaDataStore) { + metaDataStore.getModelsToBind() + .forEach(model -> { + if (!metadataDictionary.isJPAEntity(model) + && !metadataDictionary.getRelationships(model).isEmpty()) { + throw new InvalidPredicateException( + "Non-JPA entities " + model.getSimpleName() + " is not allowed to have relationship."); + } + }); + metaDataStore.getModelsToBind().stream() .map(model -> constructTable(model, metadataDictionary)) .forEach(metaDataStore::addTable); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java index 678d7e306a..1b17a944cc 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryValidator.java @@ -71,7 +71,7 @@ private void validateHavingClause(FilterExpression havingClause) { Class cls = last.getType(); String fieldName = last.getFieldName(); - Class tableClass = dictionary.getEntityClass(queriedTable.getName()); + Class tableClass = dictionary.getEntityClass(queriedTable.getId()); if (cls != tableClass) { throw new InvalidOperationException( @@ -128,17 +128,6 @@ public void validateSorting() { private void validateSortingPath(Path path, Set allFields) { List pathElements = path.getPathElements(); - // TODO: add support for double nested sorting - if (pathElements.size() > 2) { - throw new UnsupportedOperationException( - "Currently sorting on double nested fields is not supported"); - } - - if (metrics.isEmpty() && pathElements.size() > 1) { - throw new UnsupportedOperationException( - "Query with no metric can't sort on nested field."); - } - Path.PathElement currentElement = pathElements.get(0); String currentField = currentElement.getFieldName(); Class currentClass = currentElement.getType(); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index 2db399d844..6b9c3294d8 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -12,7 +12,6 @@ import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.metadata.models.Column; -import com.yahoo.elide.datastores.aggregation.metadata.models.DataType; import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; @@ -83,7 +82,6 @@ public void addTable(Table table) { */ private void addColumn(Column column) { addMetaData(column); - addDataType(column.getDataType()); if (column instanceof TimeDimension) { ((TimeDimension) column).getSupportedGrains().forEach(this::addTimeDimensionGrain); @@ -102,15 +100,6 @@ private void addMetricFunction(MetricFunction metricFunction) { metricFunction.getArguments().forEach(this::addFunctionArgument); } - /** - * Add a datatype metadata object. - * - * @param dataType datatype metadata - */ - private void addDataType(DataType dataType) { - addMetaData(dataType); - } - /** * Add a function argument metadata object. * diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java deleted file mode 100644 index 828f00b523..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/Aggregation.java +++ /dev/null @@ -1,15 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.enums; - -/** - * Aggregation functions. - */ -public enum Aggregation { - SUM, - MIN, - MAX; -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java index 6df6643e9e..f80dc2c814 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/enums/ValueType.java @@ -5,15 +5,43 @@ */ package com.yahoo.elide.datastores.aggregation.metadata.enums; +import java.math.BigDecimal; +import java.util.HashMap; +import java.util.Map; + /** * Actual value type of a data type. */ public enum ValueType { - DATE, - NUMBER, + TIME, + INTEGER, + DECIMAL, + MONEY, TEXT, COORDINATE, BOOLEAN, RELATIONSHIP, - ID + ID; + + private static final Map, ValueType> SCALAR_TYPES = new HashMap, ValueType>() {{ + put(short.class, INTEGER); + put(Short.class, INTEGER); + put(int.class, INTEGER); + put(Integer.class, INTEGER); + put(long.class, INTEGER); + put(Long.class, INTEGER); + put(BigDecimal.class, DECIMAL); + put(float.class, DECIMAL); + put(Float.class, DECIMAL); + put(double.class, DECIMAL); + put(Double.class, DECIMAL); + put(boolean.class, BOOLEAN); + put(Boolean.class, BOOLEAN); + put(char.class, TEXT); + put(String.class, TEXT); + }}; + + public static ValueType getScalarType(Class fieldClass) { + return SCALAR_TYPES.get(fieldClass); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java index b1b1fc27a3..8b34693590 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -6,19 +6,19 @@ package com.yahoo.elide.datastores.aggregation.metadata.models; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; import lombok.Data; +import lombok.EqualsAndHashCode; import lombok.ToString; import java.util.Date; import java.util.HashSet; -import java.util.Locale; import java.util.Set; import javax.persistence.Id; -import javax.persistence.ManyToOne; /** * Column is the super class of a field in a table, it can be either dimension or metric. @@ -34,21 +34,25 @@ public abstract class Column { private String longName; - private String tableName; - private String description; private String category; - @ManyToOne - private DataType dataType; + @ToOne + @ToString.Exclude + @EqualsAndHashCode.Exclude + private Table table; + + private ValueType valueType; @ToString.Exclude private Set columnTags; - protected Column(Class tableClass, String fieldName, EntityDictionary dictionary) { - this.tableName = dictionary.getJsonAliasFor(tableClass); - this.id = tableName + "." + fieldName; + protected Column(Table table, String fieldName, EntityDictionary dictionary) { + this.table = table; + Class tableClass = dictionary.getEntityClass(table.getId()); + + this.id = table.getId() + "." + fieldName; this.name = fieldName; this.columnTags = new HashSet<>(); @@ -58,29 +62,25 @@ protected Column(Class tableClass, String fieldName, EntityDictionary diction this.description = meta.description(); } - dataType = getDataType(tableClass, fieldName, dictionary); - if (dataType == null) { + valueType = getValueType(tableClass, fieldName, dictionary); + if (valueType == null) { throw new IllegalArgumentException("Unknown data type for " + this.id); } } - public static DataType getDataType(Class tableClass, String fieldName, EntityDictionary dictionary) { - String tableName = dictionary.getJsonAliasFor(tableClass); - DataType dataType; + public static ValueType getValueType(Class tableClass, String fieldName, EntityDictionary dictionary) { if (dictionary.isRelation(tableClass, fieldName)) { - Class relationshipClass = dictionary.getParameterizedType(tableClass, fieldName); - dataType = new RelationshipType(dictionary.getJsonAliasFor(relationshipClass)); + return ValueType.RELATIONSHIP; } else { Class fieldClass = dictionary.getType(tableClass, fieldName); if (fieldName.equals(dictionary.getIdFieldName(tableClass))) { - dataType = new DataType(tableName + "." + fieldName, ValueType.ID); + return ValueType.ID; } else if (Date.class.isAssignableFrom(fieldClass)) { - dataType = new DataType(fieldClass.getSimpleName().toLowerCase(Locale.ENGLISH), ValueType.DATE); + return ValueType.TIME; } else { - dataType = DataType.getScalarType(fieldClass); + return ValueType.getScalarType(fieldClass); } } - return dataType; } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java deleted file mode 100644 index de8a0d2b81..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/DataType.java +++ /dev/null @@ -1,58 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.models; - -import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.BOOLEAN; -import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.NUMBER; -import static com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType.TEXT; - -import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; - -import lombok.AllArgsConstructor; -import lombok.Data; -import lombok.ToString; - -import java.math.BigDecimal; -import java.util.HashMap; -import java.util.Map; -import javax.persistence.Id; - -/** - * Data type of a column. - */ -@Include(rootLevel = true, type = "dataType") -@Data -@AllArgsConstructor -@ToString -public class DataType { - private static final Map, DataType> SCALAR_TYPES = new HashMap, DataType>() {{ - put(short.class, new DataType("p_short", NUMBER)); - put(Short.class, new DataType("short", NUMBER)); - put(int.class, new DataType("p_int", NUMBER)); - put(Integer.class, new DataType("int", NUMBER)); - put(long.class, new DataType("p_bigint", NUMBER)); - put(Long.class, new DataType("bigint", NUMBER)); - put(BigDecimal.class, new DataType("bigDecimal", NUMBER)); - put(float.class, new DataType("p_float", NUMBER)); - put(Float.class, new DataType("float", NUMBER)); - put(double.class, new DataType("p_double", NUMBER)); - put(Double.class, new DataType("double", NUMBER)); - put(boolean.class, new DataType("p_boolean", BOOLEAN)); - put(Boolean.class, new DataType("boolean", BOOLEAN)); - put(char.class, new DataType("p_char", TEXT)); - put(String.class, new DataType("string", TEXT)); - }}; - - @Id - private String name; - - private ValueType valueType; - - public static DataType getScalarType(Class valueClass) { - return SCALAR_TYPES.get(valueClass); - } -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java index 1eabdd3674..e27ab6d93d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Dimension.java @@ -18,7 +18,7 @@ @Include(type = "dimension") @Data public class Dimension extends Column { - public Dimension(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public Dimension(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java index d7b1de386b..73045eb4f0 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/FunctionArgument.java @@ -6,12 +6,12 @@ package com.yahoo.elide.datastores.aggregation.metadata.models; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; import lombok.Data; import lombok.ToString; import javax.persistence.Id; -import javax.persistence.ManyToOne; /** * Arguments that can be provided into a metric function. @@ -27,13 +27,15 @@ public class FunctionArgument { private String description; - @ManyToOne - private DataType dataType; + private ValueType type; + + private String subType; public FunctionArgument(String functionName, FunctionArgument argument) { this.id = functionName + "." + argument.getName(); this.name = argument.getName(); this.description = argument.getDescription(); - this.dataType = argument.getDataType(); + this.type = argument.getType(); + this.subType = argument.getSubType(); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java index 26f2fd9831..fd8b320122 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Metric.java @@ -31,8 +31,9 @@ public class Metric extends Column { @ToString.Exclude private MetricFunction metricFunction; - public Metric(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public Metric(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); + Class tableClass = dictionary.getEntityClass(table.getId()); MetricAggregation metric = dictionary.getAttributeOrRelationAnnotation( tableClass, diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java deleted file mode 100644 index c0e8f49587..0000000000 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/RelationshipType.java +++ /dev/null @@ -1,22 +0,0 @@ -/* - * Copyright 2019, Yahoo Inc. - * Licensed under the Apache License, Version 2.0 - * See LICENSE file in project root for terms. - */ -package com.yahoo.elide.datastores.aggregation.metadata.models; - -import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; - -import lombok.Data; -import lombok.EqualsAndHashCode; - -/** - * Special data type that represents a relationship between tables. - */ -@Data -@EqualsAndHashCode(callSuper = true) -public class RelationshipType extends DataType { - public RelationshipType(String name) { - super(name, ValueType.RELATIONSHIP); - } -} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java index 17247e5828..23f0ef244f 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Table.java @@ -6,7 +6,7 @@ package com.yahoo.elide.datastores.aggregation.metadata.models; import static com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore.isMetricField; -import static com.yahoo.elide.datastores.aggregation.metadata.models.Column.getDataType; +import static com.yahoo.elide.datastores.aggregation.metadata.models.Column.getValueType; import com.yahoo.elide.annotation.Exclude; import com.yahoo.elide.annotation.Include; @@ -35,9 +35,9 @@ @ToString public class Table { @Id - private String name; + private String id; - private String longName; + private String name; private String description; @@ -70,7 +70,7 @@ public Table(Class cls, EntityDictionary dictionary) { String.format("Table class {%s} is not defined in dictionary.", cls)); } - this.name = dictionary.getJsonAliasFor(cls); + this.id = dictionary.getJsonAliasFor(cls); this.tableTags = new HashSet<>(); this.columns = constructColumns(cls, dictionary); @@ -87,7 +87,7 @@ public Table(Class cls, EntityDictionary dictionary) { Meta meta = cls.getAnnotation(Meta.class); if (meta != null) { - this.longName = meta.longName(); + this.name = meta.longName(); this.description = meta.description(); } @@ -106,21 +106,21 @@ public Table(Class cls, EntityDictionary dictionary) { */ private Set constructColumns(Class cls, EntityDictionary dictionary) { Set columns = dictionary.getAllFields(cls).stream() - .filter(field -> getDataType(cls, field, dictionary) != null) + .filter(field -> getValueType(cls, field, dictionary) != null) .map(field -> { if (isMetricField(dictionary, cls, field)) { - return constructMetric(cls, field, dictionary); + return constructMetric(field, dictionary); } else if (dictionary.attributeOrRelationAnnotationExists(cls, field, Temporal.class)) { - return constructTimeDimension(cls, field, dictionary); + return constructTimeDimension(field, dictionary); } else { - return constructDimension(cls, field, dictionary); + return constructDimension(field, dictionary); } }) .collect(Collectors.toSet()); // add id field if exists if (dictionary.getIdFieldName(cls) != null) { - columns.add(constructDimension(cls, dictionary.getIdFieldName(cls), dictionary)); + columns.add(constructDimension(dictionary.getIdFieldName(cls), dictionary)); } return columns; @@ -129,37 +129,34 @@ private Set constructColumns(Class cls, EntityDictionary dictionary) /** * Construct a Metric instance. * - * @param cls table class * @param fieldName field name * @param dictionary dictionary contains the table class * @return Metric metadata instance */ - protected Metric constructMetric(Class cls, String fieldName, EntityDictionary dictionary) { - return new Metric(cls, fieldName, dictionary); + protected Metric constructMetric(String fieldName, EntityDictionary dictionary) { + return new Metric(this, fieldName, dictionary); } /** * Construct a Dimension instance. * - * @param cls table class * @param fieldName field name * @param dictionary dictionary contains the table class * @return Dimension metadata instance */ - protected TimeDimension constructTimeDimension(Class cls, String fieldName, EntityDictionary dictionary) { - return new TimeDimension(cls, fieldName, dictionary); + protected TimeDimension constructTimeDimension(String fieldName, EntityDictionary dictionary) { + return new TimeDimension(this, fieldName, dictionary); } /** * Construct a TimeDimension instance. * - * @param cls table class * @param fieldName field name * @param dictionary dictionary contains the table class * @return TimeDimension metadata instance */ - protected Dimension constructDimension(Class cls, String fieldName, EntityDictionary dictionary) { - return new Dimension(cls, fieldName, dictionary); + protected Dimension constructDimension(String fieldName, EntityDictionary dictionary) { + return new Dimension(this, fieldName, dictionary); } /** diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java index 22b06e3fd8..6174f3dfb2 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/TimeDimension.java @@ -11,6 +11,7 @@ import lombok.Data; import lombok.EqualsAndHashCode; +import lombok.ToString; import java.util.Arrays; import java.util.LinkedHashSet; @@ -28,14 +29,18 @@ @Data public class TimeDimension extends Dimension { @ManyToMany + @ToString.Exclude Set supportedGrains; private TimeZone timezone; - public TimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public TimeDimension(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); - Temporal temporal = dictionary.getAttributeOrRelationAnnotation(tableClass, Temporal.class, fieldName); + Temporal temporal = dictionary.getAttributeOrRelationAnnotation( + dictionary.getEntityClass(table.getId()), + Temporal.class, + fieldName); this.supportedGrains = Arrays.stream(temporal.grains()) .map(grain -> new TimeDimensionGrain(getId(), grain)) diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java index caec8cd31e..c23fe1870f 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/AbstractEntityHydrator.java @@ -7,9 +7,9 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.QueryEngine; +import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; import com.yahoo.elide.datastores.aggregation.metadata.metric.MetricFunctionInvocation; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; -import com.yahoo.elide.datastores.aggregation.metadata.models.RelationshipType; import com.yahoo.elide.datastores.aggregation.query.ColumnProjection; import com.yahoo.elide.datastores.aggregation.query.Query; @@ -151,7 +151,7 @@ public Iterable hydrate() { * @return A hydrated entity object. */ protected Object coerceObjectToEntity(Map result, MutableInt counter) { - Class entityClass = entityDictionary.getEntityClass(query.getTable().getName()); + Class entityClass = entityDictionary.getEntityClass(query.getTable().getId()); //Construct the object. Object entityInstance; @@ -164,7 +164,7 @@ protected Object coerceObjectToEntity(Map result, MutableInt cou result.forEach((fieldName, value) -> { Dimension dim = query.getTable().getDimension(fieldName); - if (dim != null && dim.getDataType() instanceof RelationshipType) { + if (dim != null && dim.getValueType().equals(ValueType.RELATIONSHIP)) { getStitchList().todo(entityInstance, fieldName, value); // We don't hydrate relationships here. } else { getEntityDictionary().setValue(entityInstance, fieldName, value); @@ -194,7 +194,7 @@ private void populateObjectLookupTable() { String joinField = entry.getKey(); List joinFieldIds = entry.getValue(); Class relationshipType = getEntityDictionary().getParameterizedType( - entityDictionary.getEntityClass(getQuery().getTable().getName()), + entityDictionary.getEntityClass(getQuery().getTable().getId()), joinField); getStitchList().populateLookup(relationshipType, getRelationshipValues(relationshipType, joinFieldIds)); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java index 4d1fcf9107..5f4572f552 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryConstructor.java @@ -73,7 +73,7 @@ public SQLQuery resolveTemplate(Query clientQuery, FilterExpression whereClause, FilterExpression havingClause) { SQLTable table = (SQLTable) clientQuery.getTable(); - Class tableCls = dictionary.getEntityClass(clientQuery.getTable().getName()); + Class tableCls = dictionary.getEntityClass(clientQuery.getTable().getId()); String tableAlias = getClassAlias(tableCls); SQLQuery.SQLQueryBuilder builder = SQLQuery.builder().clientQuery(clientQuery); @@ -84,7 +84,7 @@ public SQLQuery resolveTemplate(Query clientQuery, ? "(" + tableCls.getAnnotation(FromSubquery.class).sql() + ")" : tableCls.isAnnotationPresent(FromTable.class) ? tableCls.getAnnotation(FromTable.class).name() - : table.getName(); + : table.getId(); builder.fromClause(String.format("%s AS %s", tableStatement, tableAlias)); @@ -155,7 +155,7 @@ private String constructHavingClauseWithReference(FilterPredicate predicate, Class lastClass = last.getType(); String fieldName = last.getFieldName(); - if (!lastClass.equals(dictionary.getEntityClass(table.getName()))) { + if (!lastClass.equals(dictionary.getEntityClass(table.getId()))) { throw new InvalidPredicateException("The having clause can only reference fact table aggregations."); } @@ -186,7 +186,7 @@ private String constructProjectionWithReference(SQLQueryTemplate template, SQLTa .map(invocation -> invocation.getFunctionExpression() + " AS " + invocation.getAlias()) .collect(Collectors.toList()); - Class tableClass = dictionary.getEntityClass(table.getName()); + Class tableClass = dictionary.getEntityClass(table.getId()); List dimensionProjections = template.getGroupByDimensions().stream() .map(dimension -> resolveSQLColumnReference(dimension, table) + " AS " + dimension.getAlias()) diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java index c5d46f4156..7381063f82 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java @@ -11,6 +11,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; import lombok.Getter; @@ -25,8 +26,9 @@ public class SQLDimension extends Dimension implements SQLColumn { @Getter private final JoinPath joinPath; - public SQLDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public SQLDimension(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); + Class tableClass = dictionary.getEntityClass(table.getId()); JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java index 9adb42c333..00aa2e4dad 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLMetric.java @@ -9,6 +9,7 @@ import com.yahoo.elide.datastores.aggregation.metadata.models.FunctionArgument; import com.yahoo.elide.datastores.aggregation.metadata.models.Metric; import com.yahoo.elide.datastores.aggregation.metadata.models.MetricFunction; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.SQLMetricFunction; import java.util.Set; @@ -17,8 +18,8 @@ * SQLMetric would contain {@link SQLMetricFunction} instead of {@link MetricFunction}. */ public class SQLMetric extends Metric { - public SQLMetric(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public SQLMetric(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); } @Override diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java index c787232dec..6f15cf764d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTable.java @@ -8,14 +8,12 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; -import lombok.Data; import lombok.EqualsAndHashCode; /** * SQL extension of {@link Table} which also contains sql column meta data. */ @EqualsAndHashCode(callSuper = true) -@Data public class SQLTable extends Table { public SQLTable(Class cls, EntityDictionary dictionary) { super(cls, dictionary); @@ -27,17 +25,17 @@ public final SQLColumn getSQLColumn(String fieldName) { } @Override - protected SQLMetric constructMetric(Class cls, String fieldName, EntityDictionary dictionary) { - return new SQLMetric(cls, fieldName, dictionary); + protected SQLMetric constructMetric(String fieldName, EntityDictionary dictionary) { + return new SQLMetric(this, fieldName, dictionary); } @Override - protected SQLTimeDimension constructTimeDimension(Class cls, String fieldName, EntityDictionary dictionary) { - return new SQLTimeDimension(cls, fieldName, dictionary); + protected SQLTimeDimension constructTimeDimension(String fieldName, EntityDictionary dictionary) { + return new SQLTimeDimension(this, fieldName, dictionary); } @Override - protected SQLDimension constructDimension(Class cls, String fieldName, EntityDictionary dictionary) { - return new SQLDimension(cls, fieldName, dictionary); + protected SQLDimension constructDimension(String fieldName, EntityDictionary dictionary) { + return new SQLDimension(this, fieldName, dictionary); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java index 17241405c2..fff0a73be0 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java @@ -10,6 +10,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.core.JoinPath; +import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; @@ -25,8 +26,9 @@ public class SQLTimeDimension extends TimeDimension implements SQLColumn { @Getter private final JoinPath joinPath; - public SQLTimeDimension(Class tableClass, String fieldName, EntityDictionary dictionary) { - super(tableClass, fieldName, dictionary); + public SQLTimeDimension(Table table, String fieldName, EntityDictionary dictionary) { + super(table, fieldName, dictionary); + Class tableClass = dictionary.getEntityClass(table.getId()); JoinTo joinTo = dictionary.getAttributeOrRelationAnnotation(tableClass, JoinTo.class, fieldName); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java index 661e378edb..bfe356df5c 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/EntityProjectionTranslatorTest.java @@ -11,7 +11,6 @@ import com.yahoo.elide.core.exceptions.InvalidOperationException; import com.yahoo.elide.core.filter.dialect.ParseException; import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.datastores.aggregation.example.Country; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.filter.visitor.FilterConstraints; import com.yahoo.elide.datastores.aggregation.filter.visitor.SplitFilterExpressionVisitor; @@ -23,7 +22,6 @@ import com.yahoo.elide.request.Argument; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; -import com.yahoo.elide.request.Relationship; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; @@ -43,16 +41,6 @@ public class EntityProjectionTranslatorTest extends SQLUnitTest { .type(String.class) .name("overallRating") .build()) - .relationship(Relationship.builder() - .name("country") - .projection(EntityProjection.builder() - .type(Country.class) - .attribute(Attribute.builder() - .type(String.class) - .name("name") - .build()) - .build()) - .build()) .build(); @BeforeAll @@ -73,11 +61,10 @@ public void testBasicTranslation() { assertEquals(playerStatsTable, query.getTable()); assertEquals(1, query.getMetrics().size()); assertEquals("lowScore", query.getMetrics().get(0).getAlias()); - assertEquals(2, query.getGroupByDimensions().size()); + assertEquals(1, query.getGroupByDimensions().size()); List dimensions = new ArrayList<>(query.getGroupByDimensions()); assertEquals("overallRating", dimensions.get(0).getColumn().getName()); - assertEquals("country", dimensions.get(1).getColumn().getName()); } @Test diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java index 5d69ce4747..08583135f3 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/QueryValidatorTest.java @@ -35,23 +35,6 @@ public static void init() { SQLUnitTest.init(); } - @Test - public void testNoMetricQuery() { - Map sortMap = new TreeMap<>(); - sortMap.put("country.name", Sorting.SortOrder.asc); - - Query query = Query.builder() - .table(playerStatsTable) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) - .build(); - - QueryValidator validator = new QueryValidator(query, Collections.singleton("overallRating"), dictionary); - - UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); - assertEquals("Query with no metric can't sort on nested field.", exception.getMessage()); - } - @Test public void testSortingOnId() { Map sortMap = new TreeMap<>(); @@ -75,7 +58,7 @@ public void testSortingOnId() { @Test public void testSortingOnNotQueriedDimension() { Map sortMap = new TreeMap<>(); - sortMap.put("country.name", Sorting.SortOrder.asc); + sortMap.put("countryIsoCode", Sorting.SortOrder.asc); Query query = Query.builder() .table(playerStatsTable) @@ -88,7 +71,7 @@ public void testSortingOnNotQueriedDimension() { QueryValidator validator = new QueryValidator(query, allFields, dictionary); InvalidOperationException exception = assertThrows(InvalidOperationException.class, validator::validate); - assertEquals("Invalid operation: 'Can't sort on country as it is not present in query'", exception.getMessage()); + assertEquals("Invalid operation: 'Can't sort on countryIsoCode as it is not present in query'", exception.getMessage()); } @Test @@ -110,25 +93,6 @@ public void testSortingOnNotQueriedMetric() { assertEquals("Invalid operation: 'Can't sort on highScore as it is not present in query'", exception.getMessage()); } - @Test - public void testSortingOnNestedDimensionField() { - Map sortMap = new TreeMap<>(); - sortMap.put("country.continent.name", Sorting.SortOrder.asc); - - Query query = Query.builder() - .table(playerStatsTable) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) - .build(); - - Set allFields = new HashSet<>(Arrays.asList("country", "lowScore")); - QueryValidator validator = new QueryValidator(query, allFields, dictionary); - - UnsupportedOperationException exception = assertThrows(UnsupportedOperationException.class, validator::validate); - assertEquals("Currently sorting on double nested fields is not supported", exception.getMessage()); - } - @Test public void testHavingFilterPromotionUngroupedDimension() throws ParseException { FilterExpression originalFilter = filterParser.parseFilterExpression("countryIsoCode==USA,lowScore<45", diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java index 96fcc23fad..17ed251a29 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryView.java @@ -6,22 +6,21 @@ package com.yahoo.elide.datastores.aggregation.example; import com.yahoo.elide.annotation.Include; -import com.yahoo.elide.annotation.ToOne; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; import com.yahoo.elide.datastores.aggregation.annotation.Join; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; import lombok.Data; import javax.persistence.Column; -import javax.persistence.Table; /** * A view version of table countries. */ @Data @Include -@Table(name = "countries") +@FromTable(name = "countries") public class CountryView { @Column(name = "id") private String countryId; @@ -40,17 +39,6 @@ public CountryViewNested getNestedView() { @JoinTo(path = "nestedView.isoCode") private String nestedViewIsoCode; - private Country nestedRelationship; - - @ToOne - @Column(name = "id") - public Country getNestedRelationship() { - return nestedRelationship; - } - - @JoinTo(path = "nestedRelationship.isoCode") - private String nestedRelationshipIsoCode; - public String getCountryId() { return countryId; } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java index c77ef81a0f..87fd3fbb12 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/CountryViewNested.java @@ -7,18 +7,18 @@ import com.yahoo.elide.annotation.Include; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromTable; import lombok.Data; import javax.persistence.Id; -import javax.persistence.Table; /** * A nested view for testing. */ @Data @Include -@Table(name = "countries") +@FromTable(name = "countries") public class CountryViewNested { private String id; diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java index 60f394bafc..b6e5c99bac 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStats.java @@ -9,6 +9,7 @@ import com.yahoo.elide.datastores.aggregation.annotation.Cardinality; import com.yahoo.elide.datastores.aggregation.annotation.CardinalitySize; import com.yahoo.elide.datastores.aggregation.annotation.FriendlyName; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.annotation.Temporal; @@ -26,8 +27,6 @@ import java.util.Date; import javax.persistence.Column; import javax.persistence.Id; -import javax.persistence.JoinColumn; -import javax.persistence.ManyToOne; /** * A root level entity for testing AggregationDataStore. */ @@ -138,8 +137,7 @@ public void setOverallRating(final String overallRating) { this.overallRating = overallRating; } - @ManyToOne - @JoinColumn(name = "country_id") + @Join("%from.country_id = %join.id") public Country getCountry() { return country; } @@ -148,43 +146,6 @@ public void setCountry(final Country country) { this.country = country; } - @ManyToOne - @JoinColumn(name = "sub_country_id") - public SubCountry getSubCountry() { - return subCountry; - } - - public void setSubCountry(final SubCountry subCountry) { - this.subCountry = subCountry; - } - - @ManyToOne - @JoinColumn(name = "player_id") - public Player getPlayer() { - return player; - } - - public void setPlayer(final Player player) { - this.player = player; - } - - /** - * DO NOT put {@link Cardinality} annotation on this field. See - * - * @return the date of the player session. - */ - @Temporal(grains = { - @TimeGrainDefinition(grain = TimeGrain.DAY, expression = DAY_FORMAT), - @TimeGrainDefinition(grain = TimeGrain.MONTH, expression = MONTH_FORMAT) - }, timeZone = "UTC") - public Date getRecordedDate() { - return recordedDate; - } - - public void setRecordedDate(final Date recordedDate) { - this.recordedDate = recordedDate; - } - @JoinTo(path = "country.isoCode") public String getCountryIsoCode() { return countryIsoCode; @@ -194,6 +155,14 @@ public void setCountryIsoCode(String isoCode) { this.countryIsoCode = isoCode; } + @Join("%from.sub_country_id = %join.id") + public SubCountry getSubCountry() { + return subCountry; + } + + public void setSubCountry(final SubCountry subCountry) { + this.subCountry = subCountry; + } @JoinTo(path = "subCountry.isoCode") @Column(updatable = false, insertable = false) // subselect field should be read-only @@ -205,8 +174,16 @@ public void setSubCountryIsoCode(String isoCode) { this.subCountryIsoCode = isoCode; } - @JoinColumn(name = "player2_id") - @ManyToOne + @Join("%from.player_id = %join.id") + public Player getPlayer() { + return player; + } + + public void setPlayer(final Player player) { + this.player = player; + } + + @Join("%from.player2_id = %join.id") public Player getPlayer2() { return player2; } @@ -232,4 +209,21 @@ public String getPlayer2Name() { public void setPlayer2Name(String player2Name) { this.player2Name = player2Name; } + + /** + * DO NOT put {@link Cardinality} annotation on this field. See + * + * @return the date of the player session. + */ + @Temporal(grains = { + @TimeGrainDefinition(grain = TimeGrain.DAY, expression = DAY_FORMAT), + @TimeGrainDefinition(grain = TimeGrain.MONTH, expression = MONTH_FORMAT) + }, timeZone = "UTC") + public Date getRecordedDate() { + return recordedDate; + } + + public void setRecordedDate(final Date recordedDate) { + this.recordedDate = recordedDate; + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java index 22562b204a..90d0809277 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsView.java @@ -6,14 +6,14 @@ package com.yahoo.elide.datastores.aggregation.example; import com.yahoo.elide.annotation.Include; +import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.annotation.MetricAggregation; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metric.functions.SqlMax; + import lombok.Data; import javax.persistence.Id; -import javax.persistence.JoinColumn; -import javax.persistence.OneToOne; /** * A root level entity for testing AggregationDataStore. @@ -40,7 +40,6 @@ public class PlayerStatsView { */ private String countryName; - @OneToOne - @JoinColumn(name = "player_id") + @Join("%from.player_id = %join.id") private Player player; } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java index c6d437e919..e2cd45928a 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/example/PlayerStatsWithView.java @@ -27,8 +27,6 @@ import java.util.Date; import javax.persistence.Column; import javax.persistence.Id; -import javax.persistence.JoinColumn; -import javax.persistence.ManyToOne; /** * A root level entity for testing AggregationDataStore. @@ -78,9 +76,6 @@ public class PlayerStatsWithView { @Setter private String countryViewViewIsoCode; - @Setter - private String countryViewRelationshipIsoCode; - /** * A dimension field joined to this table. */ @@ -136,8 +131,7 @@ public void setOverallRating(final String overallRating) { this.overallRating = overallRating; } - @ManyToOne - @JoinColumn(name = "country_id") + @Join("%from.country_id = %join.id") public Country getCountry() { return country; } @@ -146,8 +140,7 @@ public void setCountry(final Country country) { this.country = country; } - @ManyToOne - @JoinColumn(name = "sub_country_id") + @Join("%from.sub_country_id = %join.id") public SubCountry getSubCountry() { return subCountry; } @@ -156,8 +149,7 @@ public void setSubCountry(final SubCountry subCountry) { this.subCountry = subCountry; } - @ManyToOne - @JoinColumn(name = "player_id") + @Join("%from.player_id = %join.id") public Player getPlayer() { return player; } @@ -214,9 +206,4 @@ public String getCountryViewIsoCode() { public String getCountryViewViewIsoCode() { return countryViewViewIsoCode; } - - @JoinTo(path = "countryView.nestedRelationship.isoCode") - public String getCountryViewRelationshipIsoCode() { - return countryViewRelationshipIsoCode; - } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java index 6c64bafd1a..0b4f981854 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -87,15 +87,13 @@ public void basicAggregationTest() throws Exception { selection( field( "playerStats", + arguments( + argument("sort", "\"highScore\"") + ), selections( field("highScore"), field("overallRating"), - field( - "country", - selections( - field("name") - ) - ) + field("countryIsoCode") ) ) ) @@ -105,35 +103,20 @@ public void basicAggregationTest() throws Exception { selections( field( "playerStats", + selections( + field("highScore", 1000), + field("overallRating", "Good"), + field("countryIsoCode", "HKG") + ), selections( field("highScore", 1234), field("overallRating", "Good"), - field( - "country", - selections( - field("name", "United States") - ) - ) + field("countryIsoCode", "USA") ), selections( field("highScore", 2412), field("overallRating", "Great"), - field( - "country", - selections( - field("name", "United States") - ) - ) - ), - selections( - field("highScore", 1000), - field("overallRating", "Good"), - field( - "country", - selections( - field("name", "Hong Kong") - ) - ) + field("countryIsoCode", "USA") ) ) ) @@ -149,17 +132,10 @@ public void noMetricQueryTest() throws Exception { field( "playerStatsWithView", arguments( - argument("sort", "\"countryViewRelationshipIsoCode\"") + argument("sort", "\"countryViewViewIsoCode\"") ), selections( - field( - "country", - selections( - field("name"), - field("isoCode") - ) - ), - field("countryViewRelationshipIsoCode") + field("countryViewViewIsoCode") ) ) ) @@ -170,24 +146,10 @@ public void noMetricQueryTest() throws Exception { field( "playerStatsWithView", selections( - field( - "country", - selections( - field("name", "Hong Kong"), - field("isoCode", "HKG") - ) - ), - field("countryViewRelationshipIsoCode", "HKG") + field("countryViewViewIsoCode", "HKG") ), selections( - field( - "country", - selections( - field("name", "United States"), - field("isoCode", "USA") - ) - ), - field("countryViewRelationshipIsoCode", "USA") + field("countryViewViewIsoCode", "USA") ) ) ) @@ -240,12 +202,7 @@ public void havingFilterTest() throws Exception { selections( field("lowScore"), field("overallRating"), - field( - "player", - selections( - field("name") - ) - ) + field("playerName") ) ) ) @@ -258,12 +215,7 @@ public void havingFilterTest() throws Exception { selections( field("lowScore", 35), field("overallRating", "Good"), - field( - "player", - selections( - field("name", "Jon Doe") - ) - ) + field("playerName", "Jon Doe") ) ) ) @@ -283,17 +235,13 @@ public void wherePromotionTest() throws Exception { field( "playerStats", arguments( - argument("filter", "\"overallRating==\\\"Good\\\",lowScore<\\\"45\\\"\"") + argument("filter", "\"overallRating==\\\"Good\\\",lowScore<\\\"45\\\"\""), + argument("sort", "\"lowScore\"") ), selections( field("lowScore"), field("overallRating"), - field( - "player", - selections( - field("name") - ) - ) + field("playerName") ) ) ) @@ -306,22 +254,12 @@ public void wherePromotionTest() throws Exception { selections( field("lowScore", 35), field("overallRating", "Good"), - field( - "player", - selections( - field("name", "Jon Doe") - ) - ) + field("playerName", "Jon Doe") ), selections( field("lowScore", 72), field("overallRating", "Good"), - field( - "player", - selections( - field("name", "Han") - ) - ) + field("playerName", "Han") ) ) ) @@ -347,12 +285,7 @@ public void havingClauseJoinTest() throws Exception { selections( field("lowScore"), field("countryIsoCode"), - field( - "player", - selections( - field("name") - ) - ) + field("playerName") ) ) ) @@ -365,22 +298,12 @@ public void havingClauseJoinTest() throws Exception { selections( field("lowScore", 35), field("countryIsoCode", "USA"), - field( - "player", - selections( - field("name", "Jon Doe") - ) - ) + field("playerName", "Jon Doe") ), selections( field("lowScore", 241), field("countryIsoCode", "USA"), - field( - "player", - selections( - field("name", "Jane Doe") - ) - ) + field("playerName", "Jane Doe") ) ) ) @@ -515,12 +438,7 @@ public void metricSortingTest() throws Exception { ), selections( field("highScore"), - field( - "country", - selections( - field("name") - ) - ) + field("countryIsoCode") ) ) ) @@ -532,21 +450,11 @@ public void metricSortingTest() throws Exception { "playerStats", selections( field("highScore", 2412), - field( - "country", - selections( - field("name", "United States") - ) - ) + field("countryIsoCode", "USA") ), selections( field("highScore", 1000), - field( - "country", - selections( - field("name", "Hong Kong") - ) - ) + field("countryIsoCode", "HKG") ) ) ) @@ -562,17 +470,12 @@ public void multipleColumnsSortingTest() throws Exception { field( "playerStats", arguments( - argument("sort", "\"overallRating,player.name\"") + argument("sort", "\"overallRating,playerName\"") ), selections( field("lowScore"), field("overallRating"), - field( - "player", - selections( - field("name") - ) - ) + field("playerName") ) ) ) @@ -585,32 +488,17 @@ public void multipleColumnsSortingTest() throws Exception { selections( field("lowScore", 72), field("overallRating", "Good"), - field( - "player", - selections( - field("name", "Han") - ) - ) + field("playerName", "Han") ), selections( field("lowScore", 35), field("overallRating", "Good"), - field( - "player", - selections( - field("name", "Jon Doe") - ) - ) + field("playerName", "Jon Doe") ), selections( field("lowScore", 241), field("overallRating", "Great"), - field( - "player", - selections( - field("name", "Jane Doe") - ) - ) + field("playerName", "Jane Doe") ) ) ) @@ -648,7 +536,7 @@ public void nestedDimensionNotInQuerySortingTest() throws Exception { field( "playerStats", arguments( - argument("sort", "\"-country.name,lowScore\"") + argument("sort", "\"-countryIsoCode,lowScore\"") ), selections( field("lowScore") @@ -657,7 +545,7 @@ public void nestedDimensionNotInQuerySortingTest() throws Exception { ) ).toQuery(); - String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on country as it is not present in query'\""; + String expected = "\"Exception while fetching data (/playerStats) : Invalid operation: 'Can't sort on countryIsoCode as it is not present in query'\""; runQueryWithExpectedError(graphQLRequest, expected); } @@ -673,12 +561,7 @@ public void sortingOnMetricNotInQueryTest() throws Exception { ), selections( field("lowScore"), - field( - "country", - selections( - field("name") - ) - ) + field("countryIsoCode") ) ) ) @@ -689,39 +572,6 @@ public void sortingOnMetricNotInQueryTest() throws Exception { runQueryWithExpectedError(graphQLRequest, expected); } - @Test - public void sortingMultipleLevelNesting() throws Exception { - String graphQLRequest = document( - selection( - field( - "playerStats", - arguments( - argument("sort", "\"country.continent.name\"") - ), - selections( - field("lowScore"), - field( - "country", - selections( - field("name"), - field( - "continent", - selections( - field("name") - ) - ) - ) - ) - ) - ) - ) - ).toQuery(); - - String expected = "\"Exception while fetching data (/playerStats) : Currently sorting on double nested fields is not supported\""; - - runQueryWithExpectedError(graphQLRequest, expected); - } - @Test @Disabled //FIXME Needs metric computation support for test case to be valid. @@ -763,14 +613,11 @@ public void basicViewAggregationTest() throws Exception { selection( field( "playerStatsWithView", + arguments( + argument("sort", "\"highScore\"") + ), selections( field("highScore"), - field( - "country", - selections( - field("name") - ) - ), field("countryViewIsoCode") ) ) @@ -783,22 +630,10 @@ public void basicViewAggregationTest() throws Exception { "playerStatsWithView", selections( field("highScore", 1000), - field( - "country", - selections( - field("name", "Hong Kong") - ) - ), field("countryViewIsoCode", "HKG") ), selections( field("highScore", 2412), - field( - "country", - selections( - field("name", "United States") - ) - ), field("countryViewIsoCode", "USA") ) ) @@ -817,7 +652,7 @@ public void jsonApiAggregationTest() { .statusCode(HttpStatus.SC_OK) .body("data.id", hasItems("0", "1", "2")) .body("data.attributes.highScore", hasItems(1000, 1234, 2412)) - .body("data.relationships.country.data.id", hasItems("840", "344")); + .body("data.attributes.countryIsoCode", hasItems("USA", "HKG")); } @Test @@ -840,23 +675,22 @@ public void metaDataTest() { "data.relationships.dimensions.data.id", hasItems( "playerStats.id", - "playerStats.player", - "playerStats.country", - "playerStats.subCountry", - "playerStats.recordedDate", - "playerStats.overallRating", + "playerStats.playerName", + "playerStats.player2Name", "playerStats.countryIsoCode", - "playerStats.subCountryIsoCode")) + "playerStats.subCountryIsoCode", + "playerStats.recordedDate", + "playerStats.overallRating")) .body("data.relationships.metrics.data.id", hasItems("playerStats.lowScore", "playerStats.highScore")); given() .accept("application/vnd.api+json") - .get("/dimension/playerStats.player") + .get("/dimension/playerStats.playerName") .then() .statusCode(HttpStatus.SC_OK) - .body("data.attributes.name", equalTo("player")) - .body("data.attributes.tableName", equalTo("playerStats")) - .body("data.relationships.dataType.data.id", equalTo("player")); + .body("data.attributes.name", equalTo("playerName")) + .body("data.attributes.valueType", equalTo("TEXT")) + .body("data.relationships.table.data.id", equalTo("playerStats")); given() .accept("application/vnd.api+json") @@ -864,8 +698,8 @@ public void metaDataTest() { .then() .statusCode(HttpStatus.SC_OK) .body("data.attributes.name", equalTo("lowScore")) - .body("data.attributes.tableName", equalTo("playerStats")) - .body("data.relationships.dataType.data.id", equalTo("p_bigint")) + .body("data.attributes.valueType", equalTo("INTEGER")) + .body("data.relationships.table.data.id", equalTo("playerStats")) .body("data.relationships.metricFunction.data.id", equalTo("playerStats.lowScore[min]")) .body("included.id", hasItem("playerStats.lowScore[min]")) .body("included.attributes.description", hasItem("sql min function")) @@ -878,8 +712,8 @@ public void metaDataTest() { .then() .statusCode(HttpStatus.SC_OK) .body("data.attributes.name", equalTo("recordedDate")) - .body("data.attributes.tableName", equalTo("playerStats")) - .body("data.relationships.dataType.data.id", equalTo("date")) + .body("data.attributes.valueType", equalTo("TIME")) + .body("data.relationships.table.data.id", equalTo("playerStats")) .body( "data.relationships.supportedGrains.data.id", hasItems("playerStats.recordedDate.day", "playerStats.recordedDate.month")) diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java index 5d870e1c3e..f5e5949215 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/QueryEngineTest.java @@ -7,7 +7,6 @@ package com.yahoo.elide.datastores.aggregation.queryengines.sql; import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertNotNull; import com.yahoo.elide.core.Path; import com.yahoo.elide.core.filter.FilterPredicate; @@ -20,6 +19,7 @@ import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; import com.yahoo.elide.datastores.aggregation.queryengines.sql.metadata.SQLTable; import com.yahoo.elide.request.Sorting; @@ -84,63 +84,86 @@ public void testFullTableLoad() { } /** - * Test group by a degenerate dimension with a filter applied. + * Test loading records using {@link FromSubquery} + */ + @Test + public void testFromSubQuery() { + Query query = Query.builder() + .table(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .build(); + + List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) + .collect(Collectors.toList()); + + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + + assertEquals(1, results.size()); + assertEquals(stats2, results.get(0)); + } + + /** + * Test group by, having, dimension, metric at the same time. * * @throws Exception exception */ @Test - public void testDegenerateDimensionFilter() throws Exception { + public void testAllArgumentQuery() throws Exception { + Map sortMap = new TreeMap<>(); + sortMap.put("countryName", Sorting.SortOrder.asc); + Query query = Query.builder() - .table(playerStatsTable) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .whereFilter(filterParser.parseFilterExpression("overallRating==Great", - PlayerStats.class, false)) + .table(playerStatsViewTable) + .metric(invoke(playerStatsTable.getMetric("highScore"))) + .groupByDimension(toProjection(playerStatsViewTable.getDimension("countryName"))) + .whereFilter(filterParser.parseFilterExpression("countryName=='United States'", + PlayerStatsView.class, false)) + .havingFilter(filterParser.parseFilterExpression("highScore > 300", + PlayerStatsView.class, false)) + .sorting(new SortingImpl(sortMap, PlayerStatsView.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) .collect(Collectors.toList()); - PlayerStats stats1 = new PlayerStats(); - stats1.setId("0"); - stats1.setLowScore(241); - stats1.setOverallRating("Great"); - stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); + PlayerStatsView stats2 = new PlayerStatsView(); + stats2.setId("0"); + stats2.setHighScore(2412); + stats2.setCountryName("United States"); assertEquals(1, results.size()); - assertEquals(stats1, results.get(0)); + assertEquals(stats2, results.get(0)); } /** - * Test filtering on a dimension attribute. + * Test group by a degenerate dimension with a filter applied. * * @throws Exception exception */ @Test - public void testFilterJoin() throws Exception { + public void testDegenerateDimensionFilter() throws Exception { Query query = Query.builder() .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .whereFilter(filterParser.parseFilterExpression("country.name=='United States'", + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) + .whereFilter(filterParser.parseFilterExpression("overallRating==Great", PlayerStats.class, false)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) .collect(Collectors.toList()); - PlayerStats usa0 = new PlayerStats(); - usa0.setId("0"); - usa0.setLowScore(35); - usa0.setCountry(USA); + PlayerStats stats1 = new PlayerStats(); + stats1.setId("0"); + stats1.setLowScore(241); + stats1.setOverallRating("Great"); + stats1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); assertEquals(1, results.size()); - assertEquals(usa0, results.get(0)); - - // test relationship hydration - PlayerStats actualStats1 = (PlayerStats) results.get(0); - assertNotNull(actualStats1.getCountry()); + assertEquals(stats1, results.get(0)); } /** @@ -149,11 +172,11 @@ public void testFilterJoin() throws Exception { * @throws Exception exception */ @Test - public void testSubqueryFilterJoin() throws Exception { + public void testNotProjectedFilter() throws Exception { Query query = Query.builder() .table(playerStatsViewTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) - .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", + .whereFilter(filterParser.parseFilterExpression("countryName=='United States'", PlayerStatsView.class, false)) .build(); @@ -168,25 +191,34 @@ public void testSubqueryFilterJoin() throws Exception { assertEquals(stats2, results.get(0)); } - /** - * Test a view which filters on "stats.overallRating = 'Great'". - */ @Test - public void testSubqueryLoad() { + public void testSortAggregatedMetric() { + Map sortMap = new TreeMap<>(); + sortMap.put("lowScore", Sorting.SortOrder.desc); + Query query = Query.builder() - .table(playerStatsViewTable) - .metric(invoke(playerStatsTable.getMetric("highScore"))) + .table(playerStatsTable) + .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) + .metric(invoke(playerStatsTable.getMetric("lowScore"))) + .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) .collect(Collectors.toList()); - PlayerStatsView stats2 = new PlayerStatsView(); - stats2.setId("0"); - stats2.setHighScore(2412); + PlayerStats stats0 = new PlayerStats(); + stats0.setId("0"); + stats0.setLowScore(241); + stats0.setOverallRating("Great"); - assertEquals(1, results.size()); - assertEquals(stats2, results.get(0)); + PlayerStats stats1 = new PlayerStats(); + stats1.setId("1"); + stats1.setLowScore(35); + stats1.setOverallRating("Good"); + + assertEquals(2, results.size()); + assertEquals(stats0, results.get(0)); + assertEquals(stats1, results.get(1)); } /** @@ -195,7 +227,7 @@ public void testSubqueryLoad() { @Test public void testSortJoin() { Map sortMap = new TreeMap<>(); - sortMap.put("player.name", Sorting.SortOrder.asc); + sortMap.put("playerName", Sorting.SortOrder.asc); Query query = Query.builder() .table(playerStatsTable) @@ -334,39 +366,6 @@ public void testHavingClauseJoin() throws Exception { assertEquals(stats1, results.get(1)); } - /** - * Test group by, having, dimension, metric at the same time. - * - * @throws Exception exception - */ - @Test - public void testEdgeCasesQuery() throws Exception { - Map sortMap = new TreeMap<>(); - sortMap.put("player.name", Sorting.SortOrder.asc); - - Query query = Query.builder() - .table(playerStatsViewTable) - .metric(invoke(playerStatsTable.getMetric("highScore"))) - .groupByDimension(toProjection(playerStatsViewTable.getDimension("countryName"))) - .whereFilter(filterParser.parseFilterExpression("player.name=='Jane Doe'", - PlayerStatsView.class, false)) - .havingFilter(filterParser.parseFilterExpression("highScore > 300", - PlayerStatsView.class, false)) - .sorting(new SortingImpl(sortMap, PlayerStatsView.class, dictionary)) - .build(); - - List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) - .collect(Collectors.toList()); - - PlayerStatsView stats2 = new PlayerStatsView(); - stats2.setId("0"); - stats2.setHighScore(2412); - stats2.setCountryName("United States"); - - assertEquals(1, results.size()); - assertEquals(stats2, results.get(0)); - } - /** * Test sorting by two different columns-one metric and one dimension. */ @@ -374,7 +373,7 @@ public void testEdgeCasesQuery() throws Exception { public void testSortByMultipleColumns() { Map sortMap = new TreeMap<>(); sortMap.put("lowScore", Sorting.SortOrder.desc); - sortMap.put("player.name", Sorting.SortOrder.asc); + sortMap.put("playerName", Sorting.SortOrder.asc); Query query = Query.builder() .table(playerStatsTable) @@ -411,54 +410,6 @@ public void testSortByMultipleColumns() { assertEquals(stats2, results.get(2)); } - /** - * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. - */ - @Test - public void testRelationshipHydration() { - Map sortMap = new TreeMap<>(); - sortMap.put("country.name", Sorting.SortOrder.desc); - sortMap.put("overallRating", Sorting.SortOrder.desc); - - Query query = Query.builder() - .table(playerStatsTable) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) - .build(); - - List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) - .collect(Collectors.toList()); - - PlayerStats usa0 = new PlayerStats(); - usa0.setId("0"); - usa0.setLowScore(241); - usa0.setOverallRating("Great"); - usa0.setCountry(USA); - - PlayerStats usa1 = new PlayerStats(); - usa1.setId("1"); - usa1.setLowScore(35); - usa1.setOverallRating("Good"); - usa1.setCountry(USA); - - PlayerStats hk2 = new PlayerStats(); - hk2.setId("2"); - hk2.setLowScore(72); - hk2.setOverallRating("Good"); - hk2.setCountry(HONG_KONG); - - assertEquals(3, results.size()); - assertEquals(usa0, results.get(0)); - assertEquals(usa1, results.get(1)); - assertEquals(hk2, results.get(2)); - - // test join - PlayerStats actualStats1 = (PlayerStats) results.get(0); - assertNotNull(actualStats1.getCountry()); - } - /** * Test grouping by a dimension with a JoinTo annotation. */ @@ -528,12 +479,13 @@ public void testJoinToFilter() throws Exception { public void testJoinToSort() { Map sortMap = new TreeMap<>(); sortMap.put("countryIsoCode", Sorting.SortOrder.asc); + sortMap.put("highScore", Sorting.SortOrder.asc); Query query = Query.builder() .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("countryIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); @@ -543,19 +495,19 @@ public void testJoinToSort() { PlayerStats stats1 = new PlayerStats(); stats1.setId("0"); stats1.setOverallRating("Good"); - stats1.setCountry(HONG_KONG); + stats1.setCountryIsoCode("HKG"); stats1.setHighScore(1000); PlayerStats stats2 = new PlayerStats(); stats2.setId("1"); stats2.setOverallRating("Good"); - stats2.setCountry(USA); + stats2.setCountryIsoCode("USA"); stats2.setHighScore(1234); PlayerStats stats3 = new PlayerStats(); stats3.setId("2"); stats3.setOverallRating("Great"); - stats3.setCountry(USA); + stats3.setCountryIsoCode("USA"); stats3.setHighScore(2412); assertEquals(3, results.size()); @@ -616,36 +568,6 @@ public void testFilterByTemporalDimension() { assertEquals(stats0, results.get(0)); } - @Test - public void testSortAggregatedMetric() { - Map sortMap = new TreeMap<>(); - sortMap.put("lowScore", Sorting.SortOrder.desc); - - Query query = Query.builder() - .table(playerStatsTable) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) - .build(); - - List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) - .collect(Collectors.toList()); - - PlayerStats stats0 = new PlayerStats(); - stats0.setId("0"); - stats0.setLowScore(241); - stats0.setOverallRating("Great"); - - PlayerStats stats1 = new PlayerStats(); - stats1.setId("1"); - stats1.setLowScore(35); - stats1.setOverallRating("Good"); - - assertEquals(2, results.size()); - assertEquals(stats0, results.get(0)); - assertEquals(stats1, results.get(1)); - } - @Test public void testAmbiguousFields() { Map sortMap = new TreeMap<>(); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java index bed32e8a87..d6fc403633 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SubselectTest.java @@ -7,20 +7,17 @@ package com.yahoo.elide.datastores.aggregation.queryengines.sql; import static org.junit.jupiter.api.Assertions.assertEquals; -import static org.junit.jupiter.api.Assertions.assertNotNull; import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.example.SubCountry; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; -import com.yahoo.elide.datastores.aggregation.metadata.enums.TimeGrain; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.request.Sorting; import org.junit.jupiter.api.BeforeAll; import org.junit.jupiter.api.Test; -import java.sql.Timestamp; import java.util.List; import java.util.Map; import java.util.TreeMap; @@ -44,117 +41,6 @@ public static void init() { SUB_USA.setId("840"); } - /** - * Test filtering on a dimension attribute. - * - * @throws Exception exception - */ - @Test - public void testFilterJoin() throws Exception { - Query query = Query.builder() - .table(playerStatsTable) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .metric(invoke(playerStatsTable.getMetric("highScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .whereFilter(filterParser.parseFilterExpression("subCountry.name=='United States'", - PlayerStats.class, false)) - .build(); - - - List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) - .collect(Collectors.toList()); - - PlayerStats usa0 = new PlayerStats(); - usa0.setId("0"); - usa0.setLowScore(35); - usa0.setHighScore(1234); - usa0.setOverallRating("Good"); - usa0.setCountry(USA); - usa0.setSubCountry(SUB_USA); - usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); - - PlayerStats usa1 = new PlayerStats(); - usa1.setId("1"); - usa1.setLowScore(241); - usa1.setHighScore(2412); - usa1.setOverallRating("Great"); - usa1.setCountry(USA); - usa1.setSubCountry(SUB_USA); - usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); - - assertEquals(2, results.size()); - assertEquals(usa0, results.get(0)); - assertEquals(usa1, results.get(1)); - - // test join - PlayerStats actualStats0 = (PlayerStats) results.get(0); - assertNotNull(actualStats0.getSubCountry()); - assertNotNull(actualStats0.getCountry()); - } - - /** - * Test hydrating multiple relationship values. Make sure the objects are constructed correctly. - */ - @Test - public void testRelationshipHydration() { - Map sortMap = new TreeMap<>(); - sortMap.put("subCountry.name", Sorting.SortOrder.desc); - - Query query = Query.builder() - .table(playerStatsTable) - .metric(invoke(playerStatsTable.getMetric("lowScore"))) - .metric(invoke(playerStatsTable.getMetric("highScore"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("country"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) - .timeDimension(toProjection(playerStatsTable.getTimeDimension("recordedDate"), TimeGrain.DAY)) - .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) - .build(); - - List results = StreamSupport.stream(engine.executeQuery(query).spliterator(), false) - .collect(Collectors.toList()); - - PlayerStats usa0 = new PlayerStats(); - usa0.setId("0"); - usa0.setLowScore(35); - usa0.setHighScore(1234); - usa0.setOverallRating("Good"); - usa0.setCountry(USA); - usa0.setSubCountry(SUB_USA); - usa0.setRecordedDate(Timestamp.valueOf("2019-07-12 00:00:00")); - - PlayerStats usa1 = new PlayerStats(); - usa1.setId("1"); - usa1.setLowScore(241); - usa1.setHighScore(2412); - usa1.setOverallRating("Great"); - usa1.setCountry(USA); - usa1.setSubCountry(SUB_USA); - usa1.setRecordedDate(Timestamp.valueOf("2019-07-11 00:00:00")); - - PlayerStats hk2 = new PlayerStats(); - hk2.setId("2"); - hk2.setLowScore(72); - hk2.setHighScore(1000); - hk2.setOverallRating("Good"); - hk2.setCountry(HONG_KONG); - hk2.setSubCountry(SUB_HONG_KONG); - hk2.setRecordedDate(Timestamp.valueOf("2019-07-13 00:00:00")); - - assertEquals(3, results.size()); - assertEquals(usa0, results.get(0)); - assertEquals(usa1, results.get(1)); - assertEquals(hk2, results.get(2)); - - // test join - PlayerStats actualStats0 = (PlayerStats) results.get(0); - assertNotNull(actualStats0.getSubCountry()); - assertNotNull(actualStats0.getCountry()); - } - /** * Test grouping by a dimension with a JoinTo annotation. * @@ -228,12 +114,13 @@ public void testJoinToFilter() throws Exception { public void testJoinToSort() throws Exception { Map sortMap = new TreeMap<>(); sortMap.put("subCountryIsoCode", Sorting.SortOrder.asc); + sortMap.put("highScore", Sorting.SortOrder.asc); Query query = Query.builder() .table(playerStatsTable) .metric(invoke(playerStatsTable.getMetric("highScore"))) .groupByDimension(toProjection(playerStatsTable.getDimension("overallRating"))) - .groupByDimension(toProjection(playerStatsTable.getDimension("subCountry"))) + .groupByDimension(toProjection(playerStatsTable.getDimension("subCountryIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStats.class, dictionary)) .build(); @@ -243,19 +130,19 @@ public void testJoinToSort() throws Exception { PlayerStats stats1 = new PlayerStats(); stats1.setId("0"); stats1.setOverallRating("Good"); - stats1.setSubCountry(SUB_HONG_KONG); + stats1.setSubCountryIsoCode("HKG"); stats1.setHighScore(1000); PlayerStats stats2 = new PlayerStats(); stats2.setId("1"); stats2.setOverallRating("Good"); - stats2.setSubCountry(SUB_USA); + stats2.setSubCountryIsoCode("USA"); stats2.setHighScore(1234); PlayerStats stats3 = new PlayerStats(); stats3.setId("2"); stats3.setOverallRating("Great"); - stats3.setSubCountry(SUB_USA); + stats3.setSubCountryIsoCode("USA"); stats3.setHighScore(2412); assertEquals(3, results.size()); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java index b4424f2db9..1cd56d7e53 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/ViewTest.java @@ -112,7 +112,7 @@ public void testNestedRelationshipAttribute() throws Exception { .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( - toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); @@ -122,12 +122,12 @@ public void testNestedRelationshipAttribute() throws Exception { PlayerStatsWithView usa0 = new PlayerStatsWithView(); usa0.setId("0"); usa0.setLowScore(35); - usa0.setCountryViewRelationshipIsoCode("USA"); + usa0.setCountryViewViewIsoCode("USA"); PlayerStatsWithView hk1 = new PlayerStatsWithView(); hk1.setId("1"); hk1.setLowScore(72); - hk1.setCountryViewRelationshipIsoCode("HKG"); + hk1.setCountryViewViewIsoCode("HKG"); assertEquals(2, results.size()); assertEquals(usa0, results.get(0)); @@ -147,7 +147,7 @@ public void testSortingViewAttribute() throws Exception { .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( - toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); @@ -157,12 +157,12 @@ public void testSortingViewAttribute() throws Exception { PlayerStatsWithView usa0 = new PlayerStatsWithView(); usa0.setId("0"); usa0.setLowScore(35); - usa0.setCountryViewRelationshipIsoCode("USA"); + usa0.setCountryViewViewIsoCode("USA"); PlayerStatsWithView hk1 = new PlayerStatsWithView(); hk1.setId("1"); hk1.setLowScore(72); - hk1.setCountryViewRelationshipIsoCode("HKG"); + hk1.setCountryViewViewIsoCode("HKG"); assertEquals(2, results.size()); assertEquals(usa0, results.get(0)); @@ -176,13 +176,13 @@ public void testSortingViewAttribute() throws Exception { @Test public void testSortingNestedViewAttribute() throws Exception { Map sortMap = new TreeMap<>(); - sortMap.put("countryViewRelationshipIsoCode", Sorting.SortOrder.desc); + sortMap.put("countryViewViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( - toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); @@ -192,12 +192,12 @@ public void testSortingNestedViewAttribute() throws Exception { PlayerStatsWithView usa0 = new PlayerStatsWithView(); usa0.setId("0"); usa0.setLowScore(35); - usa0.setCountryViewRelationshipIsoCode("USA"); + usa0.setCountryViewViewIsoCode("USA"); PlayerStatsWithView hk1 = new PlayerStatsWithView(); hk1.setId("1"); hk1.setLowScore(72); - hk1.setCountryViewRelationshipIsoCode("HKG"); + hk1.setCountryViewViewIsoCode("HKG"); assertEquals(2, results.size()); assertEquals(usa0, results.get(0)); @@ -211,13 +211,13 @@ public void testSortingNestedViewAttribute() throws Exception { @Test public void testSortingNestedRelationshipAttribute() throws Exception { Map sortMap = new TreeMap<>(); - sortMap.put("countryViewRelationshipIsoCode", Sorting.SortOrder.desc); + sortMap.put("countryViewViewIsoCode", Sorting.SortOrder.desc); Query query = Query.builder() .table(playerStatsWithViewSchema) .metric(invoke(playerStatsWithViewSchema.getMetric("lowScore"))) .groupByDimension( - toProjection(playerStatsWithViewSchema.getDimension("countryViewRelationshipIsoCode"))) + toProjection(playerStatsWithViewSchema.getDimension("countryViewViewIsoCode"))) .sorting(new SortingImpl(sortMap, PlayerStatsWithView.class, dictionary)) .build(); @@ -227,12 +227,12 @@ public void testSortingNestedRelationshipAttribute() throws Exception { PlayerStatsWithView usa0 = new PlayerStatsWithView(); usa0.setId("0"); usa0.setLowScore(35); - usa0.setCountryViewRelationshipIsoCode("USA"); + usa0.setCountryViewViewIsoCode("USA"); PlayerStatsWithView hk1 = new PlayerStatsWithView(); hk1.setId("1"); hk1.setLowScore(72); - hk1.setCountryViewRelationshipIsoCode("HKG"); + hk1.setCountryViewViewIsoCode("HKG"); assertEquals(2, results.size()); assertEquals(usa0, results.get(0)); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json b/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json index b1d12f5f94..e026183b0a 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json +++ b/elide-datastore/elide-datastore-aggregation/src/test/resources/graphql/responses/testGraphQLSchema.json @@ -29,13 +29,6 @@ "fields": null } }, - { - "name": "countryViewRelationshipIsoCode", - "type": { - "name": "String", - "fields": null - } - }, { "name": "countryViewViewIsoCode", "type": { @@ -77,48 +70,6 @@ "name": "String", "fields": null } - }, - { - "name": "country", - "type": { - "name": "country", - "fields": [ - { - "name": "edges" - }, - { - "name": "pageInfo" - } - ] - } - }, - { - "name": "player", - "type": { - "name": "player", - "fields": [ - { - "name": "edges" - }, - { - "name": "pageInfo" - } - ] - } - }, - { - "name": "subCountry", - "type": { - "name": "subCountry", - "fields": [ - { - "name": "edges" - }, - { - "name": "pageInfo" - } - ] - } } ] } From b5187a0410b520cac728bff087b30b817ef9c2c2 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Mon, 17 Feb 2020 18:41:02 -0600 Subject: [PATCH 13/16] Rebased on master --- .../inmemory/HashMapStoreTransaction.java | 1 - .../state/RelationshipTerminalState.java | 2 +- .../elide/core/DataStoreTransactionTest.java | 34 +- .../elide/core/EntityDictionaryTest.java | 33 +- .../com/yahoo/elide/core/LifeCycleTest.java | 5 +- .../core/PersistenceResourceTestSetup.java | 22 +- .../elide/core/PersistentResourceTest.java | 311 +----------------- .../elide/core/utils/ClassScannerTest.java | 2 +- .../jsonapi/EntityProjectionMakerTest.java | 68 +++- .../com/yahoo/elide/jsonapi/JsonApiTest.java | 2 + .../security/PermissionExecutorTest.java | 2 +- elide-datastore/elide-datastore-jpa/pom.xml | 4 - .../graphql/PersistentResourceFetcher.java | 2 - .../com/yahoo/elide/tests/TransferableIT.java | 1 + .../spring/tests/AggregationStoreTest.java | 2 +- 15 files changed, 110 insertions(+), 381 deletions(-) diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java index e8fa05d324..d4ce2ef85d 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java @@ -14,7 +14,6 @@ import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Relationship; import com.yahoo.elide.request.Sorting; -import lombok.extern.slf4j.Slf4j; import java.io.IOException; import java.io.Serializable; diff --git a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java index ae096090a0..77988494bd 100644 --- a/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java +++ b/elide-core/src/main/java/com/yahoo/elide/parsers/state/RelationshipTerminalState.java @@ -64,7 +64,7 @@ public Supplier> handleGet(StateContext state) { Map relationships = record.toResource(parentProjection).getRelationships(); Relationship relationship = null; if (relationships != null) { - Relationship relationship = relationships.get(relationshipName); + relationship = relationships.get(relationshipName); // Handle valid relationship diff --git a/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java b/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java index 09f24a0acc..01faf54f77 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java @@ -14,20 +14,21 @@ import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; -import com.yahoo.elide.core.filter.expression.FilterExpression; -import com.yahoo.elide.core.pagination.Pagination; -import com.yahoo.elide.core.sort.Sorting; +import com.yahoo.elide.request.Attribute; +import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.security.User; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import java.io.IOException; +import java.io.Serializable; import java.util.Arrays; -import java.util.Optional; public class DataStoreTransactionTest implements DataStoreTransaction { private static final String NAME = "name"; + private static final Attribute NAME_ATTRIBUTE = Attribute.builder().name(NAME).type(String.class).build(); private static final String ENTITY = "entity"; private RequestScope scope; @@ -68,7 +69,7 @@ public void testSupportsSorting() { @Test public void testSupportsPagination() { - boolean actual = supportsPagination(null); + boolean actual = supportsPagination(null, null); assertTrue(actual); } @@ -80,14 +81,14 @@ public void testSupportsFiltering() { @Test public void testGetAttribute() { - Object actual = getAttribute(ENTITY, NAME, scope); + Object actual = getAttribute(ENTITY, NAME_ATTRIBUTE, scope); assertEquals(3L, actual); verify(scope, times(1)).getDictionary(); } @Test public void testSetAttribute() { - setAttribute(ENTITY, NAME, null, scope); + setAttribute(ENTITY, NAME_ATTRIBUTE, scope); verify(scope, never()).getDictionary(); } @@ -105,24 +106,29 @@ public void testUpdateToManyRelation() { @Test public void testGetRelation() { - Object actual = getRelation(this, ENTITY, NAME, Optional.empty(), Optional.empty(), Optional.empty(), scope); + Object actual = getRelation(this, ENTITY, Relationship.builder() + .name(NAME) + .projection(EntityProjection.builder() + .type(String.class) + .build()) + .build(), scope); assertEquals(3L, actual); - verify(scope, times(1)).getDictionary(); } @Test public void testLoadObject() { - String string = (String) loadObject(String.class, 2L, Optional.empty(), scope); + String string = (String) loadObject(EntityProjection.builder().type(String.class).build(), 2L, scope); assertEquals(ENTITY, string); - verify(scope, times(1)).getDictionary(); } /** Implemented to support the interface only. No need to test these. **/ + @Override + public Object loadObject(EntityProjection entityProjection, Serializable id, RequestScope scope) { + return ENTITY; + } @Override - @Deprecated - public Iterable loadObjects(Class entityClass, Optional filterExpression, - Optional sorting, Optional pagination, RequestScope scope) { + public Iterable loadObjects(EntityProjection entityProjection, RequestScope scope) { return Arrays.asList(ENTITY); } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java index 229d4becc4..324267e1f4 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/EntityDictionaryTest.java @@ -12,7 +12,6 @@ import static org.junit.jupiter.api.Assertions.assertThrows; import static org.junit.jupiter.api.Assertions.assertTrue; import static org.mockito.Mockito.mock; -import static org.mockito.Mockito.verify; import com.yahoo.elide.Injector; import com.yahoo.elide.annotation.ComputedAttribute; @@ -28,7 +27,6 @@ import com.yahoo.elide.models.generics.Employee; import com.yahoo.elide.models.generics.Manager; import com.yahoo.elide.security.FilterExpressionCheck; -import com.yahoo.elide.security.RequestScope; import com.yahoo.elide.security.checks.UserCheck; import com.yahoo.elide.security.checks.prefab.Collections.AppendOnly; import com.yahoo.elide.security.checks.prefab.Collections.RemoveOnly; @@ -135,7 +133,8 @@ class Foo extends FilterExpressionCheck { Long testLong; @Override - public FilterExpression getFilterExpression(Class entityClass, RequestScope requestScope) { + public FilterExpression getFilterExpression(Class entityClass, + com.yahoo.elide.security.RequestScope requestScope) { assertEquals(testLong, 123L); return null; } @@ -162,28 +161,6 @@ public void testGetAttributeOrRelationAnnotation() { } } - @Test - public void testBindingInitializerPriorToBindingEntityClass() { - @Entity - @Include - class Foo { - @Id - private long id; - - private int bar; - } - - Initializer initializer = mock(Initializer.class); - bindInitializer(initializer, Foo.class); - - assertEquals(1, getAllFields(Foo.class).size()); - - Foo foo = new Foo(); - initializeEntity(foo); - - verify(initializer).initialize(foo); - } - @Test public void testBindingTriggerPriorToBindingEntityClass1() { @Entity @@ -563,13 +540,13 @@ class SubsubclassBinding extends SubclassBinding { assertEquals(SuperclassBinding.class, lookupEntityClass(SubclassBinding.class)); assertEquals(SuperclassBinding.class, lookupEntityClass(SubsubclassBinding.class)); - assertEquals("subclassBinding", getEntityFor(SubclassBinding.class)); + assertEquals("superclassBinding", getEntityFor(SubclassBinding.class)); assertEquals("superclassBinding", getEntityFor(SuperclassBinding.class)); - assertEquals(SubclassBinding.class, getEntityClass("subclassBinding")); + assertNull(getEntityClass("subclassBinding")); assertEquals(SuperclassBinding.class, getEntityClass("superclassBinding")); - assertEquals("subclassBinding", getJsonAliasFor(SubclassBinding.class)); + assertEquals("superclassBinding", getJsonAliasFor(SubclassBinding.class)); assertEquals("superclassBinding", getJsonAliasFor(SuperclassBinding.class)); } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java index a19a6f6f7e..6d298874ad 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/LifeCycleTest.java @@ -209,7 +209,7 @@ public void testElideGet() throws Exception { when(store.beginReadTransaction()).thenCallRealMethod(); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); MultivaluedMap headers = new MultivaluedHashMap<>(); ElideResponse response = elide.get("/book/1", headers, null); @@ -315,7 +315,7 @@ public void testElidePatchFailure() throws Exception { when(book.getId()).thenReturn(1L); when(store.beginTransaction()).thenReturn(tx); - when(tx.loadObject(eq(Book.class), any(), any(), isA(RequestScope.class))).thenReturn(book); + when(tx.loadObject(isA(EntityProjection.class), any(), isA(RequestScope.class))).thenReturn(book); doThrow(ConstraintViolationException.class).when(tx).flush(any()); String bookBody = "{\"data\":{\"type\":\"book\",\"id\":1,\"attributes\": {\"title\":\"Grapes of Wrath\"}}}"; @@ -957,7 +957,6 @@ public void createPostCommit(RequestScope scope) { Book book = new Book(); when(tx.createNewObject(Book.class)).thenReturn(book); RequestScope scope = buildRequestScope(dictionary, tx); - PersistentResource bookResource = PersistentResource.createObject(null, Book.class, scope, Optional.of("123")); PersistentResource bookResource = PersistentResource.createObject(Book.class, scope, Optional.of("123")); bookResource.updateAttribute("title", "Foo"); diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java index b96db6e073..46161d24a1 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistenceResourceTestSetup.java @@ -13,11 +13,10 @@ import com.yahoo.elide.annotation.DeletePermission; import com.yahoo.elide.annotation.Include; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.SharePermission; import com.yahoo.elide.annotation.UpdatePermission; import com.yahoo.elide.audit.AuditLogger; -import com.yahoo.elide.core.filter.expression.FilterExpression; import com.yahoo.elide.jsonapi.models.JsonApiDocument; +import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.security.ChangeSpec; import com.yahoo.elide.security.User; import com.yahoo.elide.security.checks.OperationCheck; @@ -46,11 +45,12 @@ import example.nontransferable.ContainerWithPackageShare; import example.nontransferable.ShareableWithPackageShare; import example.nontransferable.Untransferable; -import nocreate.NoCreateEntity; import lombok.AllArgsConstructor; import lombok.EqualsAndHashCode; +import nocreate.NoCreateEntity; + import java.util.Collection; import java.util.HashSet; import java.util.List; @@ -222,7 +222,6 @@ public ChangeSpecModel(final Function checkFunction) { @ReadPermission(expression = "allow all") @UpdatePermission(expression = "allow all") @DeletePermission(expression = "allow all") - @SharePermission public static final class ChangeSpecChild { @Id public long id; @@ -254,10 +253,17 @@ public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestSc } } - public static Set getRelation(PersistentResource resource, String relation) { - Optional filterExpression = - resource.getRequestScope().getExpressionForRelation(resource, relation); + public Set getRelation(PersistentResource resource, String relation) { + return resource.getRelationCheckedFiltered(getRelationship(resource.getResourceClass(), relation)); + } - return resource.getRelationCheckedFiltered(relation, filterExpression, Optional.empty(), Optional.empty()); + public com.yahoo.elide.request.Relationship getRelationship(Class type, String name) { + return com.yahoo.elide.request.Relationship.builder() + .name(name) + .alias(name) + .projection(EntityProjection.builder() + .type(type) + .build()) + .build(); } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java index bd2a48183e..44f6b40f39 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/PersistentResourceTest.java @@ -22,10 +22,9 @@ import static org.mockito.Mockito.verify; import static org.mockito.Mockito.when; + import com.yahoo.elide.annotation.Audit; import com.yahoo.elide.annotation.ReadPermission; -import com.yahoo.elide.annotation.UpdatePermission; -import com.yahoo.elide.audit.AuditLogger; import com.yahoo.elide.audit.LogMessage; import com.yahoo.elide.audit.TestAuditLogger; import com.yahoo.elide.core.exceptions.ForbiddenAccessException; @@ -71,18 +70,15 @@ import example.nontransferable.ContainerWithPackageShare; import example.nontransferable.ShareableWithPackageShare; import example.nontransferable.Untransferable; -import nocreate.NoCreateEntity; import org.apache.commons.collections4.CollectionUtils; import org.apache.commons.collections4.IterableUtils; import org.junit.jupiter.api.BeforeEach; import org.junit.jupiter.api.Test; import org.junit.jupiter.api.TestInstance; -import org.mockito.Answers; import org.mockito.ArgumentCaptor; -import lombok.AllArgsConstructor; -import lombok.EqualsAndHashCode; +import nocreate.NoCreateEntity; import java.util.ArrayList; import java.util.Arrays; @@ -1192,21 +1188,6 @@ public void testNoSaveNonModifications() { child.setReadNoAccess(secret); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); - when(tx.getRelation(any(), eq(fun), eq("relation1"), any(), any(), any(), any())).thenReturn(children1); - when(tx.getRelation(any(), eq(parent), eq("children"), any(), any(), any(), any())).thenReturn(children2); - when(tx.getRelation(any(), eq(child), eq("readNoAccess"), any(), any(), any(), any())).thenReturn(secret); - - User goodUser = new User(1); - RequestScope goodScope = buildRequestScope(tx, goodUser); - - PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); - PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); - PersistentResource secretResource = new PersistentResource<>(secret, null, "1", goodScope); - PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); -======= when(tx.getRelation(any(), eq(fun), eq(com.yahoo.elide.request.Relationship.builder() .name("relation3") .alias("relation3") @@ -1248,7 +1229,6 @@ public void testNoSaveNonModifications() { PersistentResource childResource = new PersistentResource<>(child, null, "1", childScope); PersistentResource secretResource = new PersistentResource<>(secret, null, "1", childScope); PersistentResource parentResource = new PersistentResource<>(parent, null, "1", parentScope); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) // Add an existing to-one relationship funResource.addRelation("relation3", childResource); @@ -1296,13 +1276,7 @@ public void testRemoveNonexistingToOneRelation() { Child unownedChild = newChild(2); fun.setRelation3(ownedChild); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedChild, null, "1", goodScope); @@ -1325,13 +1299,7 @@ public void testRemoveNonexistingToManyRelation() { Parent unownedParent = newParent(4, null); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); PersistentResource removeResource = new PersistentResource<>(unownedParent, null, "1", goodScope); childResource.removeRelation("parents", removeResource); @@ -1358,11 +1326,7 @@ public void testClearToManyRelationSuccess() { when(tx.getRelation(any(), eq(child), any(), any())).thenReturn(parents); -<<<<<<< HEAD - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(EntityProjection.builder() .type(Child.class) .relationship("parents", @@ -1370,7 +1334,6 @@ public void testClearToManyRelationSuccess() { .type(Parent.class) .build()) .build()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource childResource = new PersistentResource<>(child, null, "1", goodScope); @@ -1396,13 +1359,7 @@ public void testClearToOneRelationSuccess() { when(tx.getRelation(any(), eq(fun), any(), any())).thenReturn(child); -<<<<<<< HEAD - when(tx.getRelation(any(), eq(fun), eq("relation3"), any(), any(), any(), any())).thenReturn(child); - - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(EntityProjection.builder() .type(FunWithPermissions.class) .relationship("relation3", @@ -1411,7 +1368,6 @@ public void testClearToOneRelationSuccess() { .build()) .build()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); funResource.clearRelation("relation3"); @@ -1492,13 +1448,7 @@ public void testClearRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); - goodScope.setEntityProjection(EntityProjection.builder() .type(Left.class) .relationship("noUpdateOne2One", @@ -1507,7 +1457,6 @@ public void testClearRelationInvalidToOneUpdatePermission() { .build()) .build()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1526,13 +1475,7 @@ public void testNoChangeRelationInvalidToOneUpdatePermission() { left.setNoUpdateOne2One(right); right.setNoUpdateOne2One(left); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1558,13 +1501,7 @@ public void testClearRelationInvalidToManyUpdatePermission() { when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noInverseUpdate); -<<<<<<< HEAD - when(tx.getRelation(any(), eq(left), eq("noInverseUpdate"), any(), any(), any(), any())).thenReturn(noInverseUpdate); - - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(EntityProjection.builder() .type(Left.class) .relationship("noInverseUpdate", @@ -1573,7 +1510,6 @@ public void testClearRelationInvalidToManyUpdatePermission() { .build()) .build()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertThrows( @@ -1591,15 +1527,9 @@ public void testClearRelationInvalidToOneDeletePermission() { noDelete.setId(1); left.setNoDeleteOne2One(noDelete); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); + when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noDelete); - when(tx.getRelation(any(), eq(left), eq("noDeleteOne2One"), any(), any(), any(), any())).thenReturn(noDelete); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - when(tx.getRelation(any(), eq(left), any(), any())).thenReturn(noDelete); - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(EntityProjection.builder() .type(Left.class) .relationship("noDeleteOne2One", @@ -1608,7 +1538,6 @@ public void testClearRelationInvalidToOneDeletePermission() { .build()) .build()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource leftResource = new PersistentResource<>(left, null, "1", goodScope); assertTrue(leftResource.clearRelation("noDeleteOne2One")); assertNull(leftResource.getObject().getNoDeleteOne2One()); @@ -1621,13 +1550,7 @@ public void testClearRelationInvalidRelation() { Child child = newChild(1); fun.setRelation3(child); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> funResource.clearRelation("invalid")); } @@ -1636,14 +1559,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) public void testUpdateAttributeSuccess() { Parent parent = newParent(1); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); parentResource.updateAttribute("firstName", "foobar"); @@ -1657,14 +1573,7 @@ >>>>>>> ef111d6e... Create AggregationDataStore module (#845) public void testUpdateAttributeInvalidAttribute() { Parent parent = newParent(1); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User goodUser = new User(1); - RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource parentResource = new PersistentResource<>(parent, null, "1", goodScope); assertThrows(InvalidAttributeException.class, () -> parentResource.updateAttribute("invalid", "foobar")); } @@ -1674,15 +1583,8 @@ public void testUpdateAttributeInvalidUpdatePermission() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); -<<<<<<< HEAD - - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); -======= - RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); @@ -1698,15 +1600,8 @@ public void testUpdateAttributeInvalidUpdatePermissionNoChange() { FunWithPermissions fun = new FunWithPermissions(); fun.setId(1); -<<<<<<< HEAD - DataStoreTransaction tx = mock(DataStoreTransaction.class); - User badUser = new User(-1); RequestScope badScope = buildRequestScope(tx, badUser); -======= - RequestScope badScope = new RequestScope(null, null, tx, badUser, null, elideSettings); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) - PersistentResource funResource = new PersistentResource<>(fun, null, "1", badScope); assertThrows( @@ -1732,18 +1627,12 @@ public void testLoadRecords() { when(tx.loadObjects(eq(collection), any(RequestScope.class))) .thenReturn(Lists.newArrayList(child1, child2, child3, child4, child5)); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); - Set loaded = PersistentResource.loadRecords(Child.class, new ArrayList<>(), - Optional.empty(), Optional.empty(), Optional.empty(), goodScope); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(collection); Set loaded = PersistentResource.loadRecords(EntityProjection.builder() .type(Child.class) .build(), new ArrayList<>(), goodScope); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) Set expected = Sets.newHashSet(child1, child4, child5); @@ -1768,16 +1657,11 @@ public void testLoadRecordSuccess() { when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(child1); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); - PersistentResource loaded = PersistentResource.loadRecord(Child.class, "1", goodScope); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(collection); PersistentResource loaded = PersistentResource.loadRecord(EntityProjection.builder() .type(Child.class) .build(), "1", goodScope); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertEquals(child1, loaded.getObject(), "The load function should return the requested child object"); } @@ -1791,12 +1675,8 @@ public void testLoadRecordInvalidId() { when(tx.loadObject(eq(collection), eq("1"), any())).thenReturn(null); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); goodScope.setEntityProjection(collection); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) assertThrows( InvalidObjectIdentifierException.class, () -> PersistentResource.loadRecord(EntityProjection.builder() @@ -1816,15 +1696,13 @@ public void testLoadRecordForbidden() { when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noRead); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); + goodScope.setEntityProjection(collection); + assertThrows( ForbiddenAccessException.class, - () -> PersistentResource.loadRecord(NoReadEntity.class, "1", goodScope)); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); - goodScope.setEntityProjection(collection); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) + () -> PersistentResource.loadRecord(EntityProjection.builder().type(NoReadEntity.class).build(), + "1", goodScope)); } @Test() @@ -1832,13 +1710,8 @@ public void testCreateObjectSuccess() { Parent parent = newParent(1); when(tx.createNewObject(Parent.class)).thenReturn(parent); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); - PersistentResource created = PersistentResource.createObject(null, Parent.class, goodScope, Optional.of("uuid")); -======= - RequestScope goodScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); PersistentResource created = PersistentResource.createObject(Parent.class, goodScope, Optional.of("uuid")); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) parent.setChildren(new HashSet<>()); created.getRequestScope().getPermissionExecutor().executeCommitChecks(); @@ -1856,13 +1729,8 @@ public void testCreateMappedIdObjectSuccess() { when(tx.createNewObject(Job.class)).thenReturn(job); -<<<<<<< HEAD final RequestScope goodScope = buildRequestScope(tx, new User(1)); - PersistentResource created = PersistentResource.createObject(null, Job.class, goodScope, Optional.empty()); -======= - final RequestScope goodScope = new RequestScope(null, null, tx, new User(1), null, elideSettings); PersistentResource created = PersistentResource.createObject(Job.class, goodScope, Optional.empty()); ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) created.getRequestScope().getPermissionExecutor().executeCommitChecks(); assertEquals("day job", created.getObject().getTitle(), @@ -2061,13 +1929,7 @@ public void testTransferPermissionErrorOnUpdateSingularRelationship() { when(tx.loadObject(eq(collection), eq(1L), any())).thenReturn(noShare); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - - -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -2088,12 +1950,8 @@ public void testTransferPermissionErrorOnUpdateRelationshipPackageLevel() { when(tx.loadObject(any(), eq(1L), any())).thenReturn(untransferable); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); assertThrows( @@ -2115,12 +1973,7 @@ public void testTransferPermissionSuccessOnUpdateManyRelationshipPackageLevel() when(tx.loadObject(any(), eq(1L), any())).thenReturn(shareableWithPackageShare); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource containerResource = new PersistentResource<>(containerWithPackageShare, null, goodScope.getUUIDFor(containerWithPackageShare), goodScope); containerResource.updateRelation("shareableWithPackageShares", shareables.toPersistentResources(goodScope)); @@ -2147,12 +2000,7 @@ public void testTransferPermissionErrorOnUpdateManyRelationship() { when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); when(tx.loadObject(any(), eq(2L), any())).thenReturn(noShare2); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); assertThrows( @@ -2178,15 +2026,10 @@ public void testTransferPermissionSuccessOnUpdateManyRelationship() { idList.add(new ResourceIdentifier("noshare", "1").castToResource()); Relationship ids = new Relationship(null, new Data<>(idList)); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare1); when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noshares); -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShares", ids.toPersistentResources(goodScope)); @@ -2213,13 +2056,7 @@ public void testTransferPermissionSuccessOnUpdateSingularRelationship() { when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); when(tx.loadObject(any(), eq(1L), any())).thenReturn(noShare); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - - -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2243,12 +2080,7 @@ public void testTransferPermissionSuccessOnClearSingularRelationship() { when(tx.getRelation(any(), eq(userModel), any(), any())).thenReturn(noShare); - RequestScope goodScope = new TestRequestScope(tx, goodUser, dictionary); - -<<<<<<< HEAD RequestScope goodScope = buildRequestScope(tx, goodUser); -======= ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) PersistentResource userResource = new PersistentResource<>(userModel, null, goodScope.getUUIDFor(userModel), goodScope); boolean returnVal = userResource.updateRelation("noShare", ids.toPersistentResources(goodScope)); @@ -2385,12 +2217,11 @@ public void testRelationChangeSpecType() { public void testPatchRequestScope() { DataStoreTransaction tx = mock(DataStoreTransaction.class); PatchRequestScope parentScope = - new PatchRequestScope(null, tx, new User(1), elideSettings); + new PatchRequestScope("/book", tx, new User(1), elideSettings); PatchRequestScope scope = new PatchRequestScope( parentScope.getPath(), parentScope.getJsonApiDocument(), parentScope); // verify wrap works assertEquals(parentScope.isUseFilterExpressions(), scope.isUseFilterExpressions()); - assertEquals(parentScope.getSorting(), scope.getSorting()); assertEquals(parentScope.getUpdateStatusCode(), scope.getUpdateStatusCode()); assertEquals(parentScope.getObjectEntityCache(), scope.getObjectEntityCache()); @@ -2399,7 +2230,10 @@ public void testPatchRequestScope() { PersistentResource parentResource = new PersistentResource<>(parent, null, "1", scope); parentResource.updateAttribute("firstName", "foobar"); - verify(tx, times(1)).setAttribute(parent, "firstName", "foobar", scope); + ArgumentCaptor attributeArgument = ArgumentCaptor.forClass(Attribute.class); + verify(tx, times(1)).setAttribute(eq(parent), attributeArgument.capture(), eq(scope)); + assertEquals(attributeArgument.getValue().getName(), "firstName"); + assertEquals(attributeArgument.getValue().getArguments().iterator().next().getValue(), "foobar"); } @Test @@ -2431,8 +2265,6 @@ public void testSparseFields() { RequestScope scope = buildRequestScope("/", mock(DataStoreTransaction.class), new User(1), queryParams); Map> expected = ImmutableMap.of("author", ImmutableSet.of("name")); assertEquals(expected, scope.getSparseFields()); - assertEquals(10, scope.getPagination().getLimit()); - assertEquals(0, scope.getPagination().getPageTotals()); } @Test @@ -2462,123 +2294,4 @@ public void testEqualsAndHashcode() { assertNotEquals(resourceWithDifferentId, resourceWithId); assertNotEquals(resourceWithId, resourceWithDifferentId); } - - private PersistentResource bootstrapPersistentResource(T obj) { - return bootstrapPersistentResource(obj, mock(DataStoreTransaction.class)); - } - - private PersistentResource bootstrapPersistentResource(T obj, DataStoreTransaction tx) { - RequestScope requestScope = new RequestScope(null, null, tx, goodUser, null, elideSettings); - return new PersistentResource<>(obj, null, requestScope.getUUIDFor(obj), requestScope); - } - - private RequestScope getUserScope(User user, AuditLogger auditLogger) { - return new RequestScope(null, new JsonApiDocument(), null, user, null, - new ElideSettingsBuilder(null) - .withEntityDictionary(dictionary) - .withAuditLogger(auditLogger) - .build()); - } - - // Testing constructor, setId and non-null empty sets - private static Parent newParent(int id) { - Parent parent = new Parent(); - parent.setId(id); - parent.setChildren(new HashSet<>()); - parent.setSpouses(new HashSet<>()); - return parent; - } - - private Parent newParent(int id, Child child) { - Parent parent = new Parent(); - parent.setId(id); - parent.setChildren(Sets.newHashSet(child)); - parent.setSpouses(new HashSet<>()); - return parent; - } - - /* ChangeSpec-specific test elements */ - @Entity - @Include - @CreatePermission(expression = "allow all") - @ReadPermission(expression = "allow all") - @UpdatePermission(expression = "deny all") - @DeletePermission(expression = "allow all") - public static final class ChangeSpecModel { - @Id - public long id; - - @ReadPermission(expression = "deny all") - @UpdatePermission(expression = "deny all") - public Function checkFunction; - - @UpdatePermission(expression = "changeSpecNonCollection") - public String testAttr; - - @UpdatePermission(expression = "changeSpecCollection") - public List testColl; - - @OneToOne - @UpdatePermission(expression = "changeSpecNonCollection") - public ChangeSpecChild child; - - @ManyToMany - @UpdatePermission(expression = "changeSpecCollection") - public List otherKids; - - public ChangeSpecModel(final Function checkFunction) { - this.checkFunction = checkFunction; - } - } - - @Entity - @Include - @EqualsAndHashCode - @AllArgsConstructor - @CreatePermission(expression = "allow all") - @ReadPermission(expression = "allow all") - @UpdatePermission(expression = "allow all") - @DeletePermission(expression = "allow all") - public static final class ChangeSpecChild { - @Id - public long id; - } - - public static final class ChangeSpecCollection extends OperationCheck { - @Override - public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { - if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { - ChangeSpec spec = changeSpec.get(); - if (!(spec.getModified() instanceof Collection)) { - return false; - } - return ((ChangeSpecModel) object).checkFunction.apply(spec); - } - throw new IllegalStateException("Something is terribly wrong :("); - } - } - - public static final class ChangeSpecNonCollection extends OperationCheck { - @Override - public boolean ok(Object object, com.yahoo.elide.security.RequestScope requestScope, Optional changeSpec) { - if (changeSpec.isPresent() && (object instanceof ChangeSpecModel)) { - return ((ChangeSpecModel) object).checkFunction.apply(changeSpec.get()); - } - throw new IllegalStateException("Something is terribly wrong :("); - } - } - - public Set getRelation(PersistentResource resource, String relation) { - return resource.getRelationCheckedFiltered(getRelationship(resource.getResourceClass(), relation)); - } - - private com.yahoo.elide.request.Relationship getRelationship(Class type, String name) { - return com.yahoo.elide.request.Relationship.builder() - .name(name) - .alias(name) - .projection(EntityProjection.builder() - .type(type) - .build()) - .build(); - } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java index 41888644b4..15174cf479 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/utils/ClassScannerTest.java @@ -46,7 +46,7 @@ public void testGetAllAnnotatedClasses() { @Test public void testGetAnyAnnotatedClasses() { Set> classes = ClassScanner.getAnnotatedClasses(ReadPermission.class, UpdatePermission.class); - assertEquals(36, classes.size()); + assertEquals(37, classes.size()); for (Class cls : classes) { assertTrue(cls.isAnnotationPresent(ReadPermission.class) || cls.isAnnotationPresent(UpdatePermission.class)); diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java index 91a394852e..86709a4e2d 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/EntityProjectionMakerTest.java @@ -7,6 +7,7 @@ package com.yahoo.elide.jsonapi; import static org.junit.jupiter.api.Assertions.assertEquals; +import static org.junit.jupiter.api.Assertions.assertNotNull; import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.core.Path; @@ -18,7 +19,10 @@ import com.yahoo.elide.core.sort.SortingImpl; import com.yahoo.elide.request.Attribute; import com.yahoo.elide.request.EntityProjection; +import com.yahoo.elide.request.Relationship; import com.yahoo.elide.request.Sorting; + +import com.google.common.collect.Sets; import example.Address; import example.Author; import example.Book; @@ -76,7 +80,7 @@ public void testRootCollectionNoQueryParams() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -101,7 +105,7 @@ public void testRootCollectionSparseFields() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -132,7 +136,7 @@ public void testRootEntityNoQueryParams() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -165,7 +169,7 @@ public void testNestedCollectionNoQueryParams() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -197,7 +201,7 @@ public void testNestedEntityNoQueryParams() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -219,7 +223,7 @@ public void testRelationshipNoQueryParams() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -242,6 +246,7 @@ public void testRelationshipWithSingleInclude() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .build()) @@ -254,7 +259,7 @@ public void testRelationshipWithSingleInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -278,6 +283,7 @@ public void testRootCollectionWithSingleInclude() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .build()) @@ -293,7 +299,7 @@ public void testRootCollectionWithSingleInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -317,6 +323,7 @@ public void testRootEntityWithSingleInclude() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .build()) @@ -331,7 +338,7 @@ public void testRootEntityWithSingleInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -350,6 +357,7 @@ public void testRootCollectionWithNestedInclude() throws Exception { .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .attribute(Attribute.builder().name("title").type(String.class).build()) @@ -385,7 +393,7 @@ public void testRootCollectionWithNestedInclude() throws Exception { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -404,6 +412,7 @@ public void testRootEntityWithNestedInclude() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .attribute(Attribute.builder().name("title").type(String.class).build()) @@ -438,7 +447,7 @@ public void testRootEntityWithNestedInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -484,7 +493,7 @@ public void testNestedEntityWithSingleInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -531,7 +540,7 @@ public void testNestedCollectionWithSingleInclude() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -553,6 +562,7 @@ public void testRootEntityWithNestedIncludeAndSparseFields() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .relationship("books", EntityProjection.builder() .type(Book.class) .attribute(Attribute.builder().name("title").type(String.class).build()) @@ -569,7 +579,7 @@ public void testRootEntityWithNestedIncludeAndSparseFields() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -606,7 +616,7 @@ public void testRootCollectionWithGlobalFilter() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -644,7 +654,7 @@ public void testNestedCollectionWithTypedFilter() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -673,6 +683,7 @@ public void testRelationshipsAndIncludeWithFilterAndSort() { .attribute(Attribute.builder().name("name").type(String.class).build()) .attribute(Attribute.builder().name("type").type(Author.AuthorType.class).build()) .attribute(Attribute.builder().name("homeAddress").type(Address.class).build()) + .attribute(Attribute.builder().name("awards").type(String.class).build()) .filterExpression(new InInsensitivePredicate(new Path(Author.class, dictionary, "name"), "Foo")) .relationship("books", EntityProjection.builder() .type(Book.class) @@ -686,7 +697,7 @@ public void testRelationshipsAndIncludeWithFilterAndSort() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); } @Test @@ -723,6 +734,27 @@ public void testRootCollectionWithTypedFilter() { EntityProjection actual = maker.parsePath(path); - assertEquals(expected, actual); + projectionEquals(expected, actual); + } + + private void projectionEquals(EntityProjection projection1, EntityProjection projection2) { + assertEquals(projection1.getType(), projection2.getType()); + assertEquals(projection1.getSorting(), projection2.getSorting()); + assertEquals(projection1.getFilterExpression(), projection2.getFilterExpression()); + assertEquals(projection1.getPagination(), projection2.getPagination()); + + //Ignore order + assertEquals(Sets.newHashSet(projection1.getAttributes()), Sets.newHashSet(projection2.getAttributes())); + + assertEquals(projection1.getRelationships().size(), projection2.getRelationships().size()); + + projection1.getRelationships().stream().forEach((relationship -> { + assertNotNull(projection2.getRelationship(relationship.getName()).orElse(null)); + Relationship relationship2 = projection2.getRelationship(relationship.getName()).get(); + + assertEquals(relationship.getAlias(), relationship2.getAlias()); + projectionEquals(relationship.getProjection(), relationship2.getProjection()); + })); + } } diff --git a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java index 0a2b134d00..941f939416 100644 --- a/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/jsonapi/JsonApiTest.java @@ -392,6 +392,8 @@ public void compareOrder() throws JsonProcessingException { Parent parent2 = new Parent(); parent2.setId(456L); + RequestScope userScope = new TestRequestScope(tx, user, dictionary); + PersistentResource pRec1 = new PersistentResource<>(parent1, null, userScope.getUUIDFor(parent1), userScope); PersistentResource pRec2 = new PersistentResource<>(parent2, null, userScope.getUUIDFor(parent2), userScope); diff --git a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java index 2d285aba41..19ee3290ac 100644 --- a/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/security/PermissionExecutorTest.java @@ -424,7 +424,7 @@ public PersistentResource newResource(T obj, Class cls, boolean markNew) public PersistentResource newResource(Class cls, boolean markNew) { try { - T obj = cls.newInstance(); + Object obj = cls.newInstance(); return newResource(obj, cls, markNew); } catch (InstantiationException | IllegalAccessException e) { return null; diff --git a/elide-datastore/elide-datastore-jpa/pom.xml b/elide-datastore/elide-datastore-jpa/pom.xml index d8822da994..a8a0da1cac 100644 --- a/elide-datastore/elide-datastore-jpa/pom.xml +++ b/elide-datastore/elide-datastore-jpa/pom.xml @@ -75,11 +75,7 @@ com.yahoo.elide elide-integration-tests -<<<<<<< HEAD - 4.5.14-SNAPSHOT -======= 5.0.0-pr6-SNAPSHOT ->>>>>>> ef111d6e... Create AggregationDataStore module (#845) test-jar test diff --git a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java index 44713a8257..ddf2dbbb26 100644 --- a/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java +++ b/elide-graphql/src/main/java/com/yahoo/elide/graphql/PersistentResourceFetcher.java @@ -19,8 +19,6 @@ import com.google.common.collect.Sets; -import org.apache.commons.collections4.IterableUtils; - import graphql.language.Field; import graphql.schema.DataFetcher; import graphql.schema.DataFetchingEnvironment; diff --git a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java index 6cb43aee61..4b64e4ee9f 100644 --- a/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java +++ b/elide-integration-tests/src/test/java/com/yahoo/elide/tests/TransferableIT.java @@ -24,6 +24,7 @@ import com.yahoo.elide.core.HttpStatus; import com.yahoo.elide.initialization.IntegrationTest; +import com.fasterxml.jackson.databind.ObjectMapper; import com.fasterxml.jackson.databind.node.ArrayNode; import example.Left; diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java index ec6395e513..17c55ae9b3 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/test/java/com/yahoo/elide/spring/tests/AggregationStoreTest.java @@ -5,13 +5,13 @@ */ package com.yahoo.elide.spring.tests; -import static com.jayway.restassured.RestAssured.when; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attr; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.attributes; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.data; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.id; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.resource; import static com.yahoo.elide.contrib.testhelpers.jsonapi.JsonApiDSL.type; +import static io.restassured.RestAssured.when; import static org.hamcrest.Matchers.equalTo; import com.yahoo.elide.core.HttpStatus; From 1c6d99edcab11cf66412a820323b95b8e46cbf79 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Tue, 18 Feb 2020 10:15:04 -0600 Subject: [PATCH 14/16] [maven-release-plugin] prepare release 5.0.0-pr6 --- elide-annotations/pom.xml | 4 ++-- elide-contrib/elide-swagger/pom.xml | 8 ++++---- elide-contrib/elide-test-helpers/pom.xml | 4 ++-- elide-contrib/pom.xml | 6 +++--- elide-core/pom.xml | 4 ++-- .../elide-datastore-aggregation/pom.xml | 14 ++++++------- .../elide-datastore-hibernate/pom.xml | 6 +++--- .../elide-datastore-hibernate3/pom.xml | 10 +++++----- .../elide-datastore-hibernate5/pom.xml | 10 +++++----- .../elide-datastore-inmemorydb/pom.xml | 6 +++--- elide-datastore/elide-datastore-jpa/pom.xml | 8 ++++---- .../elide-datastore-multiplex/pom.xml | 10 +++++----- elide-datastore/elide-datastore-noop/pom.xml | 4 ++-- .../elide-datastore-search/pom.xml | 8 ++++---- elide-datastore/pom.xml | 6 +++--- elide-example-models/pom.xml | 4 ++-- .../elide-blog-example-resteasy/pom.xml | 6 +++--- elide-example/elide-blog-example/pom.xml | 10 +++++----- .../elide-hibernate3-mysql-example/pom.xml | 6 +++--- elide-example/pom.xml | 6 +++--- elide-graphql/pom.xml | 8 ++++---- elide-integration-tests/pom.xml | 4 ++-- .../elide-spring-boot-autoconfigure/pom.xml | 20 +++++++++---------- .../elide-spring-boot-starter/pom.xml | 20 +++++++++---------- elide-spring/pom.xml | 4 ++-- elide-standalone/pom.xml | 14 ++++++------- pom.xml | 8 ++++---- 27 files changed, 109 insertions(+), 109 deletions(-) diff --git a/elide-annotations/pom.xml b/elide-annotations/pom.xml index 177db5d79a..985aa79bd7 100644 --- a/elide-annotations/pom.xml +++ b/elide-annotations/pom.xml @@ -9,7 +9,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-contrib/elide-swagger/pom.xml b/elide-contrib/elide-swagger/pom.xml index f845698b84..b1459f1f4a 100644 --- a/elide-contrib/elide-swagger/pom.xml +++ b/elide-contrib/elide-swagger/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,14 +35,14 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -54,7 +54,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-contrib/elide-test-helpers/pom.xml b/elide-contrib/elide-test-helpers/pom.xml index 4663f250cc..e66b6bcff6 100644 --- a/elide-contrib/elide-test-helpers/pom.xml +++ b/elide-contrib/elide-test-helpers/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-contrib/pom.xml b/elide-contrib/pom.xml index e87677d9c9..52fda85475 100644 --- a/elide-contrib/pom.xml +++ b/elide-contrib/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -53,7 +53,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 diff --git a/elide-core/pom.xml b/elide-core/pom.xml index 0b166b78ff..a214e2f6b9 100644 --- a/elide-core/pom.xml +++ b/elide-core/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -40,7 +40,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml index c2da5362a9..12d2e6ff76 100644 --- a/elide-datastore/elide-datastore-aggregation/pom.xml +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -46,24 +46,24 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-graphql - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-multiplex - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -127,7 +127,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test test-jar diff --git a/elide-datastore/elide-datastore-hibernate/pom.xml b/elide-datastore/elide-datastore-hibernate/pom.xml index c50501afc0..3dfc4b56b8 100644 --- a/elide-datastore/elide-datastore-hibernate/pom.xml +++ b/elide-datastore/elide-datastore-hibernate/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -58,7 +58,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-hibernate3/pom.xml b/elide-datastore/elide-datastore-hibernate3/pom.xml index 94e16bc4d2..917a0151ac 100644 --- a/elide-datastore/elide-datastore-hibernate3/pom.xml +++ b/elide-datastore/elide-datastore-hibernate3/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -52,19 +52,19 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-hibernate5/pom.xml b/elide-datastore/elide-datastore-hibernate5/pom.xml index 445b673b68..f5b9d8cc67 100644 --- a/elide-datastore/elide-datastore-hibernate5/pom.xml +++ b/elide-datastore/elide-datastore-hibernate5/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 emptyOnAttributeCollection @@ -51,19 +51,19 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-inmemorydb/pom.xml b/elide-datastore/elide-datastore-inmemorydb/pom.xml index 6f75d107d5..600ef8838d 100644 --- a/elide-datastore/elide-datastore-inmemorydb/pom.xml +++ b/elide-datastore/elide-datastore-inmemorydb/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -51,7 +51,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-jpa/pom.xml b/elide-datastore/elide-datastore-jpa/pom.xml index a8a0da1cac..3ffe4caa6a 100644 --- a/elide-datastore/elide-datastore-jpa/pom.xml +++ b/elide-datastore/elide-datastore-jpa/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -52,7 +52,7 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -75,7 +75,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-multiplex/pom.xml b/elide-datastore/elide-datastore-multiplex/pom.xml index b4a7c68677..274855d69f 100644 --- a/elide-datastore/elide-datastore-multiplex/pom.xml +++ b/elide-datastore/elide-datastore-multiplex/pom.xml @@ -10,7 +10,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -31,7 +31,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -47,19 +47,19 @@ com.yahoo.elide elide-datastore-inmemorydb - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test com.yahoo.elide elide-datastore-hibernate5 - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test diff --git a/elide-datastore/elide-datastore-noop/pom.xml b/elide-datastore/elide-datastore-noop/pom.xml index f51f1d9618..887fa28ebe 100644 --- a/elide-datastore/elide-datastore-noop/pom.xml +++ b/elide-datastore/elide-datastore-noop/pom.xml @@ -6,7 +6,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -27,7 +27,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-datastore/elide-datastore-search/pom.xml b/elide-datastore/elide-datastore-search/pom.xml index 3d6b1467d9..5cd05f0046 100644 --- a/elide-datastore/elide-datastore-search/pom.xml +++ b/elide-datastore/elide-datastore-search/pom.xml @@ -12,7 +12,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -33,7 +33,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -57,7 +57,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test-jar test @@ -65,7 +65,7 @@ com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test diff --git a/elide-datastore/pom.xml b/elide-datastore/pom.xml index bdfe31c7d5..e56e7c19fb 100644 --- a/elide-datastore/pom.xml +++ b/elide-datastore/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -60,7 +60,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 org.hibernate diff --git a/elide-example-models/pom.xml b/elide-example-models/pom.xml index ed892206c9..4018fd4b60 100644 --- a/elide-example-models/pom.xml +++ b/elide-example-models/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -40,7 +40,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-example/elide-blog-example-resteasy/pom.xml b/elide-example/elide-blog-example-resteasy/pom.xml index 80bd25d560..6d7300c9de 100644 --- a/elide-example/elide-blog-example-resteasy/pom.xml +++ b/elide-example/elide-blog-example-resteasy/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -52,12 +52,12 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-hibernate5 - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 diff --git a/elide-example/elide-blog-example/pom.xml b/elide-example/elide-blog-example/pom.xml index 551cc5c777..e0a3cf620d 100644 --- a/elide-example/elide-blog-example/pom.xml +++ b/elide-example/elide-blog-example/pom.xml @@ -10,7 +10,7 @@ Elide Example: Hibernate5 API with Security Elide example using javax.persistence, MySQL and Elide Security com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 https://github.com/yahoo/elide @@ -45,7 +45,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -59,17 +59,17 @@ com.yahoo.elide elide-annotations - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-standalone - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-test-helpers - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 org.antlr diff --git a/elide-example/elide-hibernate3-mysql-example/pom.xml b/elide-example/elide-hibernate3-mysql-example/pom.xml index 9b7edb4b6e..3820b44e7c 100644 --- a/elide-example/elide-hibernate3-mysql-example/pom.xml +++ b/elide-example/elide-hibernate3-mysql-example/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -25,12 +25,12 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-hibernate3 - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 diff --git a/elide-example/pom.xml b/elide-example/pom.xml index a6033691a7..9b018c6ccf 100644 --- a/elide-example/pom.xml +++ b/elide-example/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -48,7 +48,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 org.apache.logging.log4j diff --git a/elide-graphql/pom.xml b/elide-graphql/pom.xml index 6880a83fce..f0419fd132 100644 --- a/elide-graphql/pom.xml +++ b/elide-graphql/pom.xml @@ -11,7 +11,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -37,19 +37,19 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-test-helpers - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.fasterxml.jackson.core diff --git a/elide-integration-tests/pom.xml b/elide-integration-tests/pom.xml index 497a138ded..270bf89dd5 100644 --- a/elide-integration-tests/pom.xml +++ b/elide-integration-tests/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -25,7 +25,7 @@ - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 diff --git a/elide-spring/elide-spring-boot-autoconfigure/pom.xml b/elide-spring/elide-spring-boot-autoconfigure/pom.xml index 9b567cbefe..b2dd9f7cb8 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/pom.xml +++ b/elide-spring/elide-spring-boot-autoconfigure/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-autoconfigure - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 jar Elide Spring Boot Autoconfigure Elide Spring Boot Autoconfigure @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -36,7 +36,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -54,42 +54,42 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true com.yahoo.elide elide-graphql - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true com.yahoo.elide elide-annotations - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true com.yahoo.elide elide-datastore-aggregation - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true com.yahoo.elide elide-swagger - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 true @@ -151,7 +151,7 @@ com.yahoo.elide elide-test-helpers - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test diff --git a/elide-spring/elide-spring-boot-starter/pom.xml b/elide-spring/elide-spring-boot-starter/pom.xml index 1f52a73450..4223533f50 100644 --- a/elide-spring/elide-spring-boot-starter/pom.xml +++ b/elide-spring/elide-spring-boot-starter/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-starter - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 jar Elide Spring Boot Starter Elide Spring Boot Starter @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -36,7 +36,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -50,37 +50,37 @@ com.yahoo.elide elide-core - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-graphql - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-annotations - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-datastore-aggregation - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-swagger - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -103,7 +103,7 @@ com.yahoo.elide elide-spring-boot-autoconfigure - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 diff --git a/elide-spring/pom.xml b/elide-spring/pom.xml index 359b3e0c4d..a9affc3849 100644 --- a/elide-spring/pom.xml +++ b/elide-spring/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 diff --git a/elide-standalone/pom.xml b/elide-standalone/pom.xml index b572f98879..d3ce456e43 100644 --- a/elide-standalone/pom.xml +++ b/elide-standalone/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-standalone - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 jar Elide Standalone Elide Standalone Application @@ -16,7 +16,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -42,7 +42,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -61,17 +61,17 @@ com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-graphql - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-swagger - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 @@ -201,7 +201,7 @@ com.yahoo.elide elide-test-helpers - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 test diff --git a/pom.xml b/pom.xml index b062fe89df..ba0f1ed336 100644 --- a/pom.xml +++ b/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-parent-pom - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 pom Elide: Parent Pom Parent pom for Elide project @@ -59,7 +59,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - HEAD + 5.0.0-pr6 @@ -103,12 +103,12 @@ com.yahoo.elide elide-annotations - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 com.yahoo.elide elide-example-models - 5.0.0-pr6-SNAPSHOT + 5.0.0-pr6 org.projectlombok From c25e945f36bd7ffd8983b48122531a1f1bfa9b46 Mon Sep 17 00:00:00 2001 From: Aaron Klish Date: Tue, 18 Feb 2020 10:15:20 -0600 Subject: [PATCH 15/16] [maven-release-plugin] prepare for next development iteration --- elide-annotations/pom.xml | 4 ++-- elide-contrib/elide-swagger/pom.xml | 8 ++++---- elide-contrib/elide-test-helpers/pom.xml | 4 ++-- elide-contrib/pom.xml | 6 +++--- elide-core/pom.xml | 4 ++-- .../elide-datastore-aggregation/pom.xml | 14 ++++++------- .../elide-datastore-hibernate/pom.xml | 6 +++--- .../elide-datastore-hibernate3/pom.xml | 10 +++++----- .../elide-datastore-hibernate5/pom.xml | 10 +++++----- .../elide-datastore-inmemorydb/pom.xml | 6 +++--- elide-datastore/elide-datastore-jpa/pom.xml | 8 ++++---- .../elide-datastore-multiplex/pom.xml | 10 +++++----- elide-datastore/elide-datastore-noop/pom.xml | 4 ++-- .../elide-datastore-search/pom.xml | 8 ++++---- elide-datastore/pom.xml | 6 +++--- elide-example-models/pom.xml | 4 ++-- .../elide-blog-example-resteasy/pom.xml | 6 +++--- elide-example/elide-blog-example/pom.xml | 10 +++++----- .../elide-hibernate3-mysql-example/pom.xml | 6 +++--- elide-example/pom.xml | 6 +++--- elide-graphql/pom.xml | 8 ++++---- elide-integration-tests/pom.xml | 4 ++-- .../elide-spring-boot-autoconfigure/pom.xml | 20 +++++++++---------- .../elide-spring-boot-starter/pom.xml | 20 +++++++++---------- elide-spring/pom.xml | 4 ++-- elide-standalone/pom.xml | 14 ++++++------- pom.xml | 8 ++++---- 27 files changed, 109 insertions(+), 109 deletions(-) diff --git a/elide-annotations/pom.xml b/elide-annotations/pom.xml index 985aa79bd7..8fd256ad32 100644 --- a/elide-annotations/pom.xml +++ b/elide-annotations/pom.xml @@ -9,7 +9,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-contrib/elide-swagger/pom.xml b/elide-contrib/elide-swagger/pom.xml index b1459f1f4a..45e05adac9 100644 --- a/elide-contrib/elide-swagger/pom.xml +++ b/elide-contrib/elide-swagger/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,14 +35,14 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -54,7 +54,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-contrib/elide-test-helpers/pom.xml b/elide-contrib/elide-test-helpers/pom.xml index e66b6bcff6..125206fee0 100644 --- a/elide-contrib/elide-test-helpers/pom.xml +++ b/elide-contrib/elide-test-helpers/pom.xml @@ -14,7 +14,7 @@ elide-contrib-parent-pom com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-contrib/pom.xml b/elide-contrib/pom.xml index 52fda85475..6533079e4d 100644 --- a/elide-contrib/pom.xml +++ b/elide-contrib/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -53,7 +53,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT diff --git a/elide-core/pom.xml b/elide-core/pom.xml index a214e2f6b9..c259453ad3 100644 --- a/elide-core/pom.xml +++ b/elide-core/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -40,7 +40,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-datastore/elide-datastore-aggregation/pom.xml b/elide-datastore/elide-datastore-aggregation/pom.xml index 12d2e6ff76..b177b40dd6 100644 --- a/elide-datastore/elide-datastore-aggregation/pom.xml +++ b/elide-datastore/elide-datastore-aggregation/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -46,24 +46,24 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-graphql - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-multiplex - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -127,7 +127,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test test-jar diff --git a/elide-datastore/elide-datastore-hibernate/pom.xml b/elide-datastore/elide-datastore-hibernate/pom.xml index 3dfc4b56b8..32293a16c5 100644 --- a/elide-datastore/elide-datastore-hibernate/pom.xml +++ b/elide-datastore/elide-datastore-hibernate/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -58,7 +58,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate3/pom.xml b/elide-datastore/elide-datastore-hibernate3/pom.xml index 917a0151ac..ef2c465eb9 100644 --- a/elide-datastore/elide-datastore-hibernate3/pom.xml +++ b/elide-datastore/elide-datastore-hibernate3/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -52,19 +52,19 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-hibernate5/pom.xml b/elide-datastore/elide-datastore-hibernate5/pom.xml index f5b9d8cc67..3388ffa9f9 100644 --- a/elide-datastore/elide-datastore-hibernate5/pom.xml +++ b/elide-datastore/elide-datastore-hibernate5/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD emptyOnAttributeCollection @@ -51,19 +51,19 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-inmemorydb/pom.xml b/elide-datastore/elide-datastore-inmemorydb/pom.xml index 600ef8838d..014a455cc9 100644 --- a/elide-datastore/elide-datastore-inmemorydb/pom.xml +++ b/elide-datastore/elide-datastore-inmemorydb/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -51,7 +51,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-jpa/pom.xml b/elide-datastore/elide-datastore-jpa/pom.xml index 3ffe4caa6a..084f8ffcc1 100644 --- a/elide-datastore/elide-datastore-jpa/pom.xml +++ b/elide-datastore/elide-datastore-jpa/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -52,7 +52,7 @@ com.yahoo.elide elide-datastore-hibernate - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -75,7 +75,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-multiplex/pom.xml b/elide-datastore/elide-datastore-multiplex/pom.xml index 274855d69f..8823615932 100644 --- a/elide-datastore/elide-datastore-multiplex/pom.xml +++ b/elide-datastore/elide-datastore-multiplex/pom.xml @@ -10,7 +10,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -31,7 +31,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -47,19 +47,19 @@ com.yahoo.elide elide-datastore-inmemorydb - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test com.yahoo.elide elide-datastore-hibernate5 - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test diff --git a/elide-datastore/elide-datastore-noop/pom.xml b/elide-datastore/elide-datastore-noop/pom.xml index 887fa28ebe..3069540a4b 100644 --- a/elide-datastore/elide-datastore-noop/pom.xml +++ b/elide-datastore/elide-datastore-noop/pom.xml @@ -6,7 +6,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -27,7 +27,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-datastore/elide-datastore-search/pom.xml b/elide-datastore/elide-datastore-search/pom.xml index 5cd05f0046..44f3dd8072 100644 --- a/elide-datastore/elide-datastore-search/pom.xml +++ b/elide-datastore/elide-datastore-search/pom.xml @@ -12,7 +12,7 @@ com.yahoo.elide elide-datastore-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -33,7 +33,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -57,7 +57,7 @@ com.yahoo.elide elide-integration-tests - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test-jar test @@ -65,7 +65,7 @@ com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test diff --git a/elide-datastore/pom.xml b/elide-datastore/pom.xml index e56e7c19fb..87f4564e92 100644 --- a/elide-datastore/pom.xml +++ b/elide-datastore/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -60,7 +60,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT org.hibernate diff --git a/elide-example-models/pom.xml b/elide-example-models/pom.xml index 4018fd4b60..6da8e96b17 100644 --- a/elide-example-models/pom.xml +++ b/elide-example-models/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -40,7 +40,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-example/elide-blog-example-resteasy/pom.xml b/elide-example/elide-blog-example-resteasy/pom.xml index 6d7300c9de..e8d4d3706f 100644 --- a/elide-example/elide-blog-example-resteasy/pom.xml +++ b/elide-example/elide-blog-example-resteasy/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -52,12 +52,12 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-hibernate5 - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT diff --git a/elide-example/elide-blog-example/pom.xml b/elide-example/elide-blog-example/pom.xml index e0a3cf620d..18242e4e7f 100644 --- a/elide-example/elide-blog-example/pom.xml +++ b/elide-example/elide-blog-example/pom.xml @@ -10,7 +10,7 @@ Elide Example: Hibernate5 API with Security Elide example using javax.persistence, MySQL and Elide Security com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT https://github.com/yahoo/elide @@ -45,7 +45,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -59,17 +59,17 @@ com.yahoo.elide elide-annotations - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-standalone - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-test-helpers - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT org.antlr diff --git a/elide-example/elide-hibernate3-mysql-example/pom.xml b/elide-example/elide-hibernate3-mysql-example/pom.xml index 3820b44e7c..4b6fbaf787 100644 --- a/elide-example/elide-hibernate3-mysql-example/pom.xml +++ b/elide-example/elide-hibernate3-mysql-example/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-example-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -25,12 +25,12 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-hibernate3 - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT diff --git a/elide-example/pom.xml b/elide-example/pom.xml index 9b018c6ccf..e5e7ac2362 100644 --- a/elide-example/pom.xml +++ b/elide-example/pom.xml @@ -14,7 +14,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -48,7 +48,7 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT org.apache.logging.log4j diff --git a/elide-graphql/pom.xml b/elide-graphql/pom.xml index f0419fd132..ef84d4dd14 100644 --- a/elide-graphql/pom.xml +++ b/elide-graphql/pom.xml @@ -11,7 +11,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -37,19 +37,19 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-test-helpers - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.fasterxml.jackson.core diff --git a/elide-integration-tests/pom.xml b/elide-integration-tests/pom.xml index 270bf89dd5..f339d8ae1e 100644 --- a/elide-integration-tests/pom.xml +++ b/elide-integration-tests/pom.xml @@ -13,7 +13,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -25,7 +25,7 @@ - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT diff --git a/elide-spring/elide-spring-boot-autoconfigure/pom.xml b/elide-spring/elide-spring-boot-autoconfigure/pom.xml index b2dd9f7cb8..916a0eb7b0 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/pom.xml +++ b/elide-spring/elide-spring-boot-autoconfigure/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-autoconfigure - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT jar Elide Spring Boot Autoconfigure Elide Spring Boot Autoconfigure @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -36,7 +36,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -54,42 +54,42 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true com.yahoo.elide elide-graphql - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true com.yahoo.elide elide-annotations - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true com.yahoo.elide elide-datastore-aggregation - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true com.yahoo.elide elide-swagger - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT true @@ -151,7 +151,7 @@ com.yahoo.elide elide-test-helpers - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test diff --git a/elide-spring/elide-spring-boot-starter/pom.xml b/elide-spring/elide-spring-boot-starter/pom.xml index 4223533f50..1371872e01 100644 --- a/elide-spring/elide-spring-boot-starter/pom.xml +++ b/elide-spring/elide-spring-boot-starter/pom.xml @@ -2,7 +2,7 @@ 4.0.0 com.yahoo.elide elide-spring-boot-starter - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT jar Elide Spring Boot Starter Elide Spring Boot Starter @@ -10,7 +10,7 @@ com.yahoo.elide elide-spring-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -36,7 +36,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -50,37 +50,37 @@ com.yahoo.elide elide-core - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-graphql - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-annotations - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-datastore-aggregation - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-swagger - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -103,7 +103,7 @@ com.yahoo.elide elide-spring-boot-autoconfigure - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT diff --git a/elide-spring/pom.xml b/elide-spring/pom.xml index a9affc3849..40d06f2c64 100644 --- a/elide-spring/pom.xml +++ b/elide-spring/pom.xml @@ -14,7 +14,7 @@ elide-parent-pom com.yahoo.elide - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -35,7 +35,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD diff --git a/elide-standalone/pom.xml b/elide-standalone/pom.xml index d3ce456e43..81f15c352a 100644 --- a/elide-standalone/pom.xml +++ b/elide-standalone/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-standalone - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT jar Elide Standalone Elide Standalone Application @@ -16,7 +16,7 @@ com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -42,7 +42,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -61,17 +61,17 @@ com.yahoo.elide elide-datastore-jpa - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-graphql - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-swagger - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT @@ -201,7 +201,7 @@ com.yahoo.elide elide-test-helpers - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT test diff --git a/pom.xml b/pom.xml index ba0f1ed336..e5b4b4bae2 100644 --- a/pom.xml +++ b/pom.xml @@ -8,7 +8,7 @@ 4.0.0 com.yahoo.elide elide-parent-pom - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT pom Elide: Parent Pom Parent pom for Elide project @@ -59,7 +59,7 @@ scm:git:ssh://git@github.com/yahoo/elide.git https://github.com/yahoo/elide.git - 5.0.0-pr6 + HEAD @@ -103,12 +103,12 @@ com.yahoo.elide elide-annotations - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT com.yahoo.elide elide-example-models - 5.0.0-pr6 + 5.0.0-pr7-SNAPSHOT org.projectlombok From 9b469996a49fc73a9bb213bac09193b8a4f6cdd4 Mon Sep 17 00:00:00 2001 From: hchen04 Date: Wed, 19 Feb 2020 15:48:36 -0800 Subject: [PATCH 16/16] sourceColumn --- .../datastores/aggregation/QueryEngine.java | 2 ++ .../aggregation/metadata/MetaDataStore.java | 12 ++++++++++++ .../aggregation/metadata/models/Column.java | 10 ++++++++++ .../queryengines/sql/metadata/SQLDimension.java | 16 ++++++++++++++++ .../sql/metadata/SQLTimeDimension.java | 16 ++++++++++++++++ .../aggregation/framework/SQLUnitTest.java | 4 +++- .../AggregationDataStoreIntegrationTest.java | 3 +++ 7 files changed, 62 insertions(+), 1 deletion(-) diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java index 43e8be5cee..a8c2327580 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -111,6 +111,8 @@ private void populateMetaData(MetaDataStore metaDataStore) { metaDataStore.getModelsToBind().stream() .map(model -> constructTable(model, metadataDictionary)) .forEach(metaDataStore::addTable); + + metaDataStore.resolveSourceColumn(); } /** diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java index 6b9c3294d8..df93d3fdb4 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/MetaDataStore.java @@ -24,6 +24,7 @@ import org.hibernate.annotations.Subselect; +import javafx.util.Pair; import lombok.Getter; import java.lang.annotation.Annotation; @@ -169,4 +170,15 @@ public static boolean isMetricField(EntityDictionary dictionary, Class cls, S public static boolean isTableJoin(Class cls, String fieldName, EntityDictionary dictionary) { return dictionary.getAttributeOrRelationAnnotation(cls, Join.class, fieldName) != null; } + + public void resolveSourceColumn() { + getMetaData(Table.class).forEach(table -> table.getColumns().forEach(column -> { + Pair sourceTableAndColumn = column.getSourceTableAndColumn(); + Table sourceTable = (Table) dataStore.get(Table.class).get(sourceTableAndColumn.getKey()); + Column sourceColumn = column instanceof Metric + ? sourceTable.getMetric(sourceTableAndColumn.getValue()) + : sourceTable.getDimension(sourceTableAndColumn.getValue()); + column.setSourceColumn(sourceColumn); + })); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java index 8b34693590..eae701a477 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/metadata/models/Column.java @@ -11,6 +11,7 @@ import com.yahoo.elide.datastores.aggregation.annotation.Meta; import com.yahoo.elide.datastores.aggregation.metadata.enums.ValueType; +import javafx.util.Pair; import lombok.Data; import lombok.EqualsAndHashCode; import lombok.ToString; @@ -45,6 +46,11 @@ public abstract class Column { private ValueType valueType; + @ToOne + @ToString.Exclude + @EqualsAndHashCode.Exclude + private Column sourceColumn; + @ToString.Exclude private Set columnTags; @@ -83,4 +89,8 @@ public static ValueType getValueType(Class tableClass, String fieldName, Enti } } } + + public Pair getSourceTableAndColumn() { + return new Pair<>(table.getId(), getName()); + } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java index 7381063f82..9330a4a32b 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLDimension.java @@ -9,11 +9,13 @@ import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.models.Dimension; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import javafx.util.Pair; import lombok.Getter; /** @@ -26,6 +28,8 @@ public class SQLDimension extends Dimension implements SQLColumn { @Getter private final JoinPath joinPath; + private EntityDictionary metadataDictionary; + public SQLDimension(Table table, String fieldName, EntityDictionary dictionary) { super(table, fieldName, dictionary); Class tableClass = dictionary.getEntityClass(table.getId()); @@ -40,5 +44,17 @@ public SQLDimension(Table table, String fieldName, EntityDictionary dictionary) this.reference = generateColumnReference(path, dictionary); this.joinPath = path; } + + this.metadataDictionary = dictionary; + } + + @Override + public Pair getSourceTableAndColumn() { + if (joinPath == null) { + return super.getSourceTableAndColumn(); + } else { + Path.PathElement last = joinPath.lastElement().get(); + return new Pair<>(metadataDictionary.getJsonAliasFor(last.getType()), last.getFieldName()); + } } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java index fff0a73be0..705f8e7d6d 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/metadata/SQLTimeDimension.java @@ -9,11 +9,13 @@ import static com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine.getClassAlias; import com.yahoo.elide.core.EntityDictionary; +import com.yahoo.elide.core.Path; import com.yahoo.elide.datastores.aggregation.core.JoinPath; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.JoinTo; +import javafx.util.Pair; import lombok.Getter; /** @@ -26,6 +28,8 @@ public class SQLTimeDimension extends TimeDimension implements SQLColumn { @Getter private final JoinPath joinPath; + private EntityDictionary metadataDictionary; + public SQLTimeDimension(Table table, String fieldName, EntityDictionary dictionary) { super(table, fieldName, dictionary); Class tableClass = dictionary.getEntityClass(table.getId()); @@ -40,5 +44,17 @@ public SQLTimeDimension(Table table, String fieldName, EntityDictionary dictiona this.reference = generateColumnReference(path, dictionary); this.joinPath = path; } + + this.metadataDictionary = dictionary; + } + + @Override + public Pair getSourceTableAndColumn() { + if (joinPath == null) { + return super.getSourceTableAndColumn(); + } else { + Path.PathElement last = joinPath.lastElement().get(); + return new Pair<>(metadataDictionary.getJsonAliasFor(last.getType()), last.getFieldName()); + } } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java index 6fc0b28798..d8141b5b36 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/SQLUnitTest.java @@ -39,7 +39,7 @@ public abstract class SQLUnitTest { protected static Table playerStatsTable; protected static EntityDictionary dictionary; protected static RSQLFilterDialect filterParser; - protected static MetaDataStore metaDataStore = new MetaDataStore(); + protected static MetaDataStore metaDataStore; protected static final Country HONG_KONG = new Country(); protected static final Country USA = new Country(); @@ -49,6 +49,8 @@ public abstract class SQLUnitTest { protected static QueryEngine engine; public static void init() { + metaDataStore = new MetaDataStore(); + emf = Persistence.createEntityManagerFactory("aggregationStore"); dictionary = new EntityDictionary(new HashMap<>()); dictionary.bindEntity(PlayerStatsWithView.class); diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java index 0b4f981854..bd1eba8c73 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/integration/AggregationDataStoreIntegrationTest.java @@ -690,6 +690,7 @@ public void metaDataTest() { .statusCode(HttpStatus.SC_OK) .body("data.attributes.name", equalTo("playerName")) .body("data.attributes.valueType", equalTo("TEXT")) + .body("data.relationships.sourceColumn.data.id", equalTo("player.name")) .body("data.relationships.table.data.id", equalTo("playerStats")); given() @@ -699,6 +700,7 @@ public void metaDataTest() { .statusCode(HttpStatus.SC_OK) .body("data.attributes.name", equalTo("lowScore")) .body("data.attributes.valueType", equalTo("INTEGER")) + .body("data.relationships.sourceColumn.data.id", equalTo("playerStats.lowScore")) .body("data.relationships.table.data.id", equalTo("playerStats")) .body("data.relationships.metricFunction.data.id", equalTo("playerStats.lowScore[min]")) .body("included.id", hasItem("playerStats.lowScore[min]")) @@ -713,6 +715,7 @@ public void metaDataTest() { .statusCode(HttpStatus.SC_OK) .body("data.attributes.name", equalTo("recordedDate")) .body("data.attributes.valueType", equalTo("TIME")) + .body("data.relationships.sourceColumn.data.id", equalTo("playerStats.recordedDate")) .body("data.relationships.table.data.id", equalTo("playerStats")) .body( "data.relationships.supportedGrains.data.id",