From 04cf829ae991df9396fbb9f80d187d6f3dcb4aad Mon Sep 17 00:00:00 2001 From: Daksh Varshney Date: Fri, 24 Jul 2020 10:57:14 -0500 Subject: [PATCH] Aggregation DataStore QueryLogger Implementation (#1448) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit * parent a14ccfbb7b558d27799b4e7b1916850519639708 author Jack (정환) 1559678988 -0700 committer Aaron Klish 1589583449 -0500 Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) [maven-release-plugin] prepare release 5.0.0-pr1 [maven-release-plugin] prepare for next development iteration Renamed graphQL file to match test (#1002) [maven-release-plugin] prepare release 5.0.0-pr2 [maven-release-plugin] prepare for next development iteration Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment Manager transacton manually (#1021) * Manager transacton manually * Add readonly Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause function name fixed to enableISO8601Dates (#1052) Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView Fixed issues with rebase Add auto configuration for aggregation store (#1087) * Added autoconfiguration for QueryEngineFactory * Unified class scanning. Started cleaning up datastores so they only register the entities they manage * Full build passes * Minor cleanup * Minor refactoring * Added EntityManagerFactory bean configuratino * Refactored class scanning for Elide standalone * Updated spring boot starter pom * Removed @Entity from all metadata models. Started cleaning up entity dictionary entity registration * Broken implementation. Just checking in so I can revert if needed. * All tests pass * Added unit tests * Minor cleanup * One more fix * Fixed broken tests * Added package include support back * Class scanning for annotations ignores inherited * Added a test based on inspection comments * Inspection comment fix * Changed initalization of MetadataStore * More inspection rework * Turned back on OWASP scanning * More rework remove @Inherited (#1092) Support Non JPA Entity in AggregationDataStore (#1051) * Create AggregationDataStore module (#845) * Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (#867) Fixed rebase on master SQL Query Engine (#878) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (#949) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (#987) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (#986) * AggregationDataStore: Schema (#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (#993) Making TimeDimension an interface (#992) * [maven-release-plugin] prepare release 5.0.0-pr1 * [maven-release-plugin] prepare for next development iteration * Renamed graphQL file to match test (#1002) * [maven-release-plugin] prepare release 5.0.0-pr2 * [maven-release-plugin] prepare for next development iteration * Add JoinTo annotation (#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework * Column annotation (#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (#1019) * Add SQLMetrix, rearrange packages (#1020) * Add SQLMetrix, rearrange packages * address comment * Manager transacton manually * Add readonly * Manager transacton manually (#1021) * Manager transacton manually * Add readonly * Hydrate GraphQL Schema with parameterized attributes (#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set * AggregationStore: Add multiple time grain definitions to schema (#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments * some rework * use getTimeDimension() * change exception * Refactor time dimension logic (#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect (#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @Subselect * Address comments * ISSUE-1027 Support join for having clause (#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause * View Design * Add tests and cleanup * rename annotation * function name fixed to enableISO8601Dates (#1052) * fix bugs * Support for multiple queries at root is added (#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved * merge annotations * don't group by view relationship * Add time grain to GraphQL schema (#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix * Support multiple query of same entity with different alias (#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute * MetadataStore Models (#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check * SQLQueryTemplate Model (#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments * Integrate Metadata Model and SQLQueryTemplate Model (#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView * integrate view with aggregation and metadata * remove includeField * remove @view * Use NonEntityDictionary * remove id * revert access changes * fix JPA entity check * remove @Entity from analyticViews * use table name as relationship type id * revert NonEntitydictinoary * tiny rework * Integration tests * Add jsonapi ittest * aggregation data store doesn't manage jpa entities * address comments fix integration dependencies (#1093) [maven-release-plugin] prepare release 5.0.0-pr3 [maven-release-plugin] prepare for next development iteration Fixed elide standalone pom from rebase Fixed minor bug in rebase Fixed rebase Improving class scanning performance for MetadataStore (#1117) Enable elide5 travis builds (#1129) * Move repeated @Sql annotations to class level (#1119) * Turning on travis builds with code coverage for Elide 5.x * Fixing security issue in spring-boot-web Co-authored-by: Brutus5000 Fix sorting and ambiguous join issue (#1127) * Added sorting on aggregated metric based on latest elide-5.x * Fix ambiguity problem * update comments * fix codacy * refactor generateColumnReference * update comment * address comments * test cleanup * update unittest * fix elide core alias * QueryValidatorTest * EntityProjectionTranslatorTest * go joinFragment approach * delete jointrienode Support no metric query (#1137) [maven-release-plugin] prepare release 5.0.0-pr4 [maven-release-plugin] prepare for next development iteration Check dependency injection (#1138) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Adding support for dependency injection of Checks. Added test injection classes * Unit tests pass * Tests pass * Removed Initializer Concept Co-authored-by: Brutus5000 Fix travis log length (#1140) * Move repeated @Sql annotations to class level (#1119) * Fixing OWASP security warning for Tomcat dependency in Spring Web (#1132) * Removed unnecessary request/response logging (to shorten travis logs) * Address inspection comments * Address inspection comments * Address inspection comments * Removed logging of graphQL model building to shorten length * Fixed compilation error Co-authored-by: Brutus5000 [maven-release-plugin] prepare release 5.0.0-pr5 [maven-release-plugin] prepare for next development iteration Refactoring Elide Security Checks (#1144) Removed UpdateOnCreate. Refactored AuditLogger, Pagination, & Sorting (#1146) * Removed UpdateOnCreate. Refactored AuditLogger * Refactored Sorting * Refactored Pagination * Refactored Pagination * Pagination refactor builds and tests pass * Codacy fixes * Inspection rework * Fixes build * More inspection rework * Fix build Refactor share permission (#1154) * Refactored Sorting * Refactored Pagination * Refactored Pagination * Pagination refactor builds and tests pass * Codacy fixes * Refactored SharePermission to NonTransferable * Fixed build * Fixed startup bug * Fixed codacy and inspection comments * Update elide-core/src/main/java/com/yahoo/elide/core/EntityDictionary.java Co-Authored-By: Jon Kilroy * Inspection rework Co-authored-by: Jon Kilroy metadata refactor (#1179) * metadata refactor * merge table and analyticView * fix reflection package * Make Table constrcut its own columns * table json type alias * add comment @Join and JoinPath add comment hide non-jpd entities in grpahql hide joins refactor hidden remove ant remove relationshp, update model (#1186) Rebased on master [maven-release-plugin] prepare release 5.0.0-pr6 [maven-release-plugin] prepare for next development iteration sourceColumn (#1196) * sourceColumn * address comment * change to sourcePath Cleaning up ElideSettings Added more test fixes Fixed a number of broken tests Build completes Fixed JSON-API Patch Response Fixed GraphQL errors. Added better errors for Forbidden Access Exception. Minor fix Added setId to EntityDictionary Initial version All test pass except for spring Build passes Adding checkstyle comments for classes Inspection comments Fixed checkstyle issues Change the way types are named in the GraphQL schema (#1215) * name utils * ready for review * fix: AggregationDataStoreIntegrationTest#testGraphQLSchema Label Resolver for Dimension Formula (#1208) * sql expression to dimension formula * Metric formula * Add unit tests * refacot formula references * fix comment * fix comment * resolve physical column * refactor dimensionFormula * cleanup * label resolver * cleanup * refactor * cleanup * move code * labelStore * labelStore 2 * remove generator * refactor metric formula * address comments * move symbol table into sql query engine * remove sourceColumn * update reference expression * visitor design * add comments * add unit join path test * fix timeDimensionProjection * fix null value number * address comments * fix codacy Co-authored-by: hchen04 Fixing broken javadoc ColumnProjection Refactor (#1239) * unify projections * remove getFunction() * add table into query template Co-authored-by: hchen04 Refactored metric SQL expansion to occur dynamically at query time ra… (#1270) * Refactored metric SQL expansion to occur dynamically at query time rather than statically during service initialization * Update elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/query/SQLMetricProjection.java Co-Authored-By: Jon Kilroy Co-authored-by: Aaron Klish Co-authored-by: Jon Kilroy Remove path logic from aggregation store (#1271) * Extended query validation to ensure where clause and sorting clauses don't traverse relationships * Added error check for relationship traversal for Having clauses * Hacked up logic to remove reference Table resolve references that take a path * Minor refactoring * Removed logic to extend join path * Refactored column projections to use generics * Removed reference functions from MetricProjection base class * Refactored so that all SQL generation is done inside the ColumnProjection * Refactored so all projection happens through projections * Refactored column projection creation * Removed unnecessary code * Added templateQuery to arguments required to generate SQL in column projections * Fixed codacy issues Co-authored-by: Aaron Klish Added InMemoryStore to list of datastores that run IT tests (#1225) Co-authored-by: Aaron Klish Adding TimeDimensions to Table (#1273) Elide 5.x async (#1203) * Adding Async Entity Models Co-authored-by: moizarafat * Adding async module and fixing parent pom version Co-authored-by: moizarafat * Adding async service classes, security and cleanup services Co-authored-by: moizarafat * Adding Copyright License Header to Async module classes Co-authored-by: moizarafat * Using new request scope for datastore transactions Co-authored-by: moizarafat * Remove thread sleep used in testing Co-authored-by: moizarafat * Fixing based on code quality review from codacy Co-authored-by: moizarafat * Fixing additional issues based on code quality review from codacy Co-authored-by: moizarafat * Adding getter for AsyncQueryResult Co-authored-by: moizarafat * Reformatting. Co-authored-by: Abhino * Adding mappedBy. Co-authored-by: Abhino * Fixing several review comments - pom, lombok, thread exec Co-authored-by: moizarafat * Fixing codacity errors Co-authored-by: moizarafat * Moving all the DB ORM logic to a utility class for Async Co-authored-by: moizarafat * Remove dependency on number of hosts, delete permission, separate method for interrupttime calculation Co-authored-by: Abhino * Variable Naming convention. Co-authored-by: Abhino * Variable Naming convention. Co-authored-by: Abhino * Removing unwanted params. Co-authored-by: Abhino * Making cleaner a separate service. Co-authored-by: Abhino * Remove unwanted params. Co-authored-by: Abhino * Remove unwanted tabs. Co-authored-by: Abhino * Updating order of modules. Co-authored-by: Abhino * Adding cleanup sql. Co-authored-by: Abhino * Adding delete logic. Co-authored-by: Abhino * Adding query result retention. Co-authored-by: Abhino * Resolving build error. Co-authored-by: Abhino * Changing AsyncDbUtil to use Functional Interface Co-authored-by: moizarafat * Removing sleep Co-authored-by: moizarafat * Adding functional interface logic for executeInTransaction Co-authored-by: moizarafat * Adding debug statements for AsyncDbUtil Co-authored-by: moizarafat * Codacity errors, exception handling Co-authored-by: moizarafat * Adding Async Entity Models Co-authored-by: moizarafat * Adding async module and fixing parent pom version Co-authored-by: moizarafat * Adding async service classes, security and cleanup services Co-authored-by: moizarafat * Adding Copyright License Header to Async module classes Co-authored-by: moizarafat * Using new request scope for datastore transactions Co-authored-by: moizarafat * Remove thread sleep used in testing Co-authored-by: moizarafat * Fixing based on code quality review from codacy Co-authored-by: moizarafat * Fixing additional issues based on code quality review from codacy Co-authored-by: moizarafat * Adding getter for AsyncQueryResult Co-authored-by: moizarafat * Reformatting. Co-authored-by: Abhino * Adding mappedBy. Co-authored-by: Abhino * Fixing several review comments - pom, lombok, thread exec Co-authored-by: moizarafat * Fixing codacity errors Co-authored-by: moizarafat * Moving all the DB ORM logic to a utility class for Async Co-authored-by: moizarafat * Remove dependency on number of hosts, delete permission, separate method for interrupttime calculation Co-authored-by: Abhino * Variable Naming convention. Co-authored-by: Abhino * Variable Naming convention. Co-authored-by: Abhino * Removing unwanted params. Co-authored-by: Abhino * Making cleaner a separate service. Co-authored-by: Abhino * Remove unwanted params. Co-authored-by: Abhino * Remove unwanted tabs. Co-authored-by: Abhino * Updating order of modules. Co-authored-by: Abhino * Adding cleanup sql. Co-authored-by: Abhino * Adding delete logic. Co-authored-by: Abhino * Adding query result retention. Co-authored-by: Abhino * Resolving build error. Co-authored-by: Abhino * Changing AsyncDbUtil to use Functional Interface Co-authored-by: moizarafat * Removing sleep Co-authored-by: moizarafat * Adding functional interface logic for executeInTransaction Co-authored-by: moizarafat * Adding debug statements for AsyncDbUtil Co-authored-by: moizarafat * Codacity errors, exception handling Co-authored-by: moizarafat * Rebased with elide-5.x and changed User principal logic Co-authored-by: moizarafat * Rebase Co-authored-by: Abhino * Changing CleanerThread to use ExecuteInTransaction Co-authored-by: moizarafat * Removing unused imports Co-authored-by: moizarafat * Delete Method changes Co-authored-by: Abhino * Overriding hashCode and equals Co-authored-by: Abhino * Overriding hashCode and equals Co-authored-by: Abhino * Removing unused method Co-authored-by: Abhino * Changing response logic for AsyncQuery Co-authored-by: moizarafat * Adding return to delete method * Resolving Review Comments Co-authored-by: Abhino * Resolve some review comments Co-authored-by: moizarafat * Changing the DAO contract and updating references to DAO Co-authored-by: moizarafat * Removing singletons and extra constructors, simplyfing logic Co-authored-by: moizarafat * Adding default constructor and setters for DefaultDAO Co-authored-by: moizarafat * Fixing codacy error Co-authored-by: moizarafat * Changing Base to use correct obj Co-authored-by: moizarafat * Fixing review comments Co-authored-by: moizarafat * Resolving review comments Co-authored-by: moizarafat * Review comments Co-authored-by: Abhino * Elide 5.x elide-async integration in elide-standalone (#1205) * Integrating elide-async. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Integrating elide-async. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Resolving review comments. Co-authored-by: Abhino * Resolving review comments. Co-authored-by: Abhino * Removing unwanted params. Co-authored-by: Abhino * Binding Cleaner service. Co-authored-by: Abhino Binding Cleaner service. Co-authored-by: Abhino * Adding query result retention. Co-authored-by: Abhino * Integrating elide-async. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Default enableAsync to False. Co-authored-by: Abhino * Integrating elide-async. Co-authored-by: Abhino * Resolving review comments. Co-authored-by: Abhino * Resolving review comments. Co-authored-by: Abhino * Removing unwanted params. Co-authored-by: Abhino * Binding Cleaner service. Co-authored-by: Abhino Binding Cleaner service. Co-authored-by: Abhino * Adding query result retention. Co-authored-by: Abhino * Making AsyncQueryDAO configurable Co-authored-by: Abhino * Making AsyncQueryDAO configurable Co-authored-by: Abhino * Review comments Co-authored-by: Abhino * Review comments Co-authored-by: abhino * Changing call to DefaultAsyncQueryDAO * Review comments Co-authored-by: Abhino Co-authored-by: Abhino * Elide 5.x elide-async integration in elide-spring (#1204) * Integrating elide-async. Co-authored-by: Abhino Integrating elide-async. Co-authored-by: Abhino * Removing unwanted params. Co-authored-by: Abhino * Binding Cleaner service. Co-authored-by: Abhino * Adding query result retention. Co-authored-by: Abhino * Adding DAO configuration Co-authored-by: Abhino * Checkstyle error Co-authored-by: Abhino * Review comments Co-authored-by: Abhino * Review comments Co-authored-by: Abhino * Review comments Co-authored-by: Abhino * Codacy Error Co-authored-by: Abhino * Changing include type to asyncQuery to avoid conflict Co-authored-by: moizarafat * Sync Co-authored-by: Abhino * Singleton for Cleaner Service Co-authored-by: Abhino * Singleton for Cleaner Service Co-authored-by: Abhino * Singleton for Cleaner Service Co-authored-by: Abhino * Singleton for Executor Service Co-authored-by: Abhino * Sync with Standalone Value * Changes to change UUID columns type as varchar(36) * Review Comments * Review Comments * Fix review comments Co-authored-by: Abhino * Updating per review comments * Remove unused import * Removing status change to Queued * removing unused import * prepersist for status Co-authored-by: moizarafat Co-authored-by: avijay Co-authored-by: moiz arafat Co-authored-by: Moiz Arafat <4307621+moizarafat@users.noreply.github.com> Co-authored-by: moizarafat Simplify life cycle hooks (#1224) * Initial version * Finished code. Starting to flesh out tests * Elide core compiles * Removed CRUDEvent.CrudAction * Added new test structure for LifeCycleTest * Fixed bug in Persistent Resource where relationship reads did not tigger a lifecycle event * Added several LifeCycle Tests * Added create test for persistent resource * Added Elide Persistent Resource Update Test * Added update with change spec test * Added Relationship Edit test * Added relationship test * Fixed checkstyle and compilation errors * Added remove from collection test * Added exception tests * Added delete test * Added read test * Fixed checkstyles * Elide core builds and tests pass * Fixed graphQL tests * Full build now passes * Removed old life cycle annotations * Minor cleanup * Fixed codacy issues * Update README.md Co-Authored-By: Jon Kilroy Co-authored-by: Aaron Klish Co-authored-by: Jon Kilroy Fixed elide-5.x build Fixed rebase issues Turning off retireJS for dependency check Better errors for missing IDs in Patch Extension Request. (#1278) (#1281) * Better errors for missing IDs in Patch Extension Request. (#1278) * Return a better error when handling invalid patch extension requests that are missing IDs * Added tests Co-authored-by: Aaron Klish * Expect encoded response as array Co-authored-by: Aaron Klish Co-authored-by: Aaron Klish Co-authored-by: wcekan [maven-release-plugin] prepare release 5.0.0-pr8 [maven-release-plugin] prepare for next development iteration Patch Extension Lifecycle tests (#1280) * Patch Extension Lifecycle tests * Missing check * Update error body Co-authored-by: wcekan Add version support (#1295) * Initial non working version * More changes * Build passes * Expanded version param outside entity dictionary * Plumbed API version up through RequestScope * Plumbed API version up to controllers and endpoints * All code minus async written and working * Code complete * Build passes * All tests are passing * Changed aggregation store Table id to include version. Metadata will now surface table versions * Added a swagger IT test with API versions * Replace empty version string with constant. Fixed a few bugs in JSON-API parser * Added graphql type introspection test * Added graphql type introspection test * Added spring controller test. Fixed issue with version header parsing * Added spring controller test for graphql * Added swagger controller tests * Added more happy path versioned tests * Fixed bug in GraphQLIT * Added graphQL test for invalid API version * ADded invalid API version graphql controller test * Fixed checkstyles * Added standalone and patch extension tests * Fixed build errors and codacy errors Co-authored-by: Aaron Klish Migrated spring controllers to use async (#1296) Co-authored-by: Aaron Klish fixes for Async Models lifecycle hooks failing (#1294) * elide standalone fixes for lifecycle hooks failing * Manual Binding of lifecycle hooks * Manual binding of lifecycle in elide spring * Refactor * Review Comments * elide standalone fixes for lifecycle hooks failing * Manual Binding of lifecycle hooks * Manual binding of lifecycle in elide spring * Refactor * Review Comments * Rebase with 5.x Co-authored-by: moiz arafat Implement equals/hashCode and immutability where needed (#1297) * ColumnProjection must implement equals/hashCode * Use @Value for SQLTimeDimensionProjection * Use @Value for Argument * Use @Value for TimeDimensionGrain * Use @Value for TimeDimension * Make Column/Metric immutable * Table can be immutable * Use @Value to simplify Query Changes elide to only inject models annotated with @Inject (#1299) * Changes elide to only inject models annotated with @Inject * Codacy fix Co-authored-by: Aaron Klish Enhancemnents to Aggregation Store (#1300) Co-authored-by: moiz arafat QueryEngine result cache (#1279) * Add QueryEngine cache API * Remove outdated javadoc text Elide Async Feature Unit and Integration Tests (#1311) * Added integration,unit test setup for Async Module with few tests Co-authored-by: moizarafat * Adding Licence headers, Sample entity and working Async test for POST,GET Co-authored-by: moizarafat * Adding additional integration tests Co-authored-by: moizarafat * Adding remaining integration tests and updating javadocs Co-authored-by: moizarafat * Adding all unit tests Co-authored-by: moizarafat * Updating tests to work with Singleton changes Co-authored-by: moizarafat * Modifying tests to work with singleton logic Co-authored-by: moizarafat * Resolving some codacy errors Co-authored-by: moizarafat * Adding DSL for GraphQL and addressing review comments Co-authored-by: moizarafat * Fixing unit tests and checkstyle errors Co-authored-by: moizarafat * Moving tests to new format Co-authored-by: moizarafat * Fixing session not closed error and checkstyle errors * Fixing imports Co-authored-by: moizarafat * Removing unused method and updating harness Co-authored-by: moizarafat * Updating unit tests Co-authored-by: moizarafat * Updating ResourceConfig for AsyncTest Co-authored-by: moizarafat * Moving logic for resource config to new test binder Co-authored-by: moizarafat * Adding bindFactory logic for async services Co-authored-by: moizarafat * Fixing IT * Fixing IT Co-authored-by: Abhino * Adding Remaining integration tests Co-authored-by: moizarafat * Consolidating filter logic for integration tests Co-authored-by: moizarafat * Fixing legacy Hibernate entity manager store (so it doesn't recycle the entity manager * Adding additional integration tests for Standalone and Spring boot Co-authored-by: moizarafat * Fixing codacy errors Co-authored-by: moizarafat * Review Comments Co-authored-by: abhino Co-authored-by: avijay Co-authored-by: moizarafat Co-authored-by: moiz arafat Co-authored-by: Aaron Klish in memory compilation integrated with dynamic config helpers (#1255) * Rebase against 5.x Co-authored-by: Ayeswarya Co-authored-by: AvaniMakwana * pom fix Co-authored-by: Ayeswarya Co-authored-by: AvaniMakwana * pom fix Co-authored-by: Ayeswarya Co-authored-by: AvaniMakwana * Merging 1220 Co-authored-by: rishi-aga Co-authored-by: AvaniMakwana * Spring Boot Changes Co-authored-by: Ayeswarya * Spring Boot Changes Co-authored-by: Ayeswarya * Spring Boot Changes Co-authored-by: Ayeswarya * Review Comments Co-authored-by: Ayeswarya * Update DynamicConfigTest.java * Review Comments Co-authored-by: Ayeswarya Co-authored-by: moiz arafat Co-authored-by: AvaniMakwana Co-authored-by: rishi-aga * Fixed issues with rebase * Finished rebase * Async ID change from UUID to String and Dynamic Config FIx (#1325) * Async UUID to String, Dynamic Config NPE * Update DefaultAsyncQueryDAOTest.java * TANDS-19093-transactionRegistry-interface (#1332) * TANDS-19093-transactionRegistry-interface * TANDS-19093-transactionRegistry-interface * TANDS-19093-transactionRegistry-interface * TANDS-19093-fixing-bugs * TANDS-19093-fixing-bugs * TANDS-19093-fixing-bugs * TANDS-19093-fixing-bugs Co-authored-by: Ramsha Rao * add requestId (#1331) Co-authored-by: aaggarwal * [maven-release-plugin] prepare release 5.0.0-pr9 * [maven-release-plugin] prepare for next development iteration * Introduce QueryResult class for QueryEngine caching (#1333) * Make QueryEngine cache bypass flag part of Query * Forming the cache key is not free, so don't a use stub cache * Add QueryResult class * Add pageTotals to QueryResult * Pass page totals through QueryResult instead of Pagination * Codacy doesn't know @Value makes fields private * remove missing javadoc warnings (#1337) Co-authored-by: Chandrasekar Rajasekar * Elide 5.x dynamic config standalone (#1259) * Fix method call, Swagger Doc Update Co-authored-by: AvaniMakwana * Swagger doc update Co-authored-by: AvaniMakwana * update pom.xml Co-authored-by: AvaniMakwana * update pom.xml Co-authored-by: AvaniMakwana * review comments Co-authored-by: AvaniMakwana * Update ElideStandaloneSettings.java * Update pom.xml * Update pom.xml Co-authored-by: moiz arafat * addTransaction-removeTransaction (#1338) * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169 * TANDS-19169-changing-CVSS-score * addTransaction-removeTransaction * addTransaction-removeTransaction * addTransaction-removeTransaction * addTransaction-removeTransaction * addTransaction-removeTransaction * addressing-comments * addressing-comments * addressing-comments * addressing-comments * addressing-comments * addressing-comments * addressing-comments * addressing-comments * addressing-comments * making DataStore an abstract class * making DataStore an abstract class * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * adding-transaction-id to transaction implementations * adding-transaction-id to transaction implementations * adding-transaction-id to transaction implementations * adding-transaction-id to transaction implementations * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * addressing comments * addressing comments * addressing comments * fixing bugs Co-authored-by: Ramsha Rao * Fixed rebase * Removed old elide example modules * [maven-release-plugin] prepare release 5.0.0-pr10 * [maven-release-plugin] prepare for next development iteration * Validation for Model Configs (#1306) * Dynamic Config Validator Co-authored-by: rishi-aga * Review Comments * Review Comments * Use DynamicConfigValidator instead of ElideConfigParser Co-authored-by: moiz arafat * add explicit join (#1364) Co-authored-by: Chandrasekar Rajasekar * Fixes - Honor Doc Version in SwaggerController and Use ISO8601Dates Serde Format in Date Formatting (#1336) * misc_fixes * review comments * Review Comments * DyFixes * DyFixes * Elide dynamic config model verification (#1354) * sign & Verify model * review comments * typo fix * indent fix * review comment * additional testcases * revert system exit * fix codacy * add return * add system exit Co-authored-by: amakwana * Add caching support to SQLQueryEngine (#1319) * Support caching in SQLQueryEngine * Allow SQLUnitTest.init to set cache * Add SQLQueryEngine cache unit tests * Declare query dimensions / metrics non-null * Handle table version as String * Polish QueryKeyExtractor * Add QueryKeyExtractor UTs * Manage QueryEngine.Transaction in AggregationDataStoreTransaction * Fix UTs * Fix assertEquals in existing UT * Move caching to AggregationDataStoreTransaction * Change getTableVersion to take table argument * Address review comments * Miscellanous fix (#1371) * Debug * Update DefaultAsyncQueryDAO.java * Update DefaultAsyncQueryDAO.java * Throw exception instead * Update AsyncQueryInterruptThread.java * Update pom.xml * Unit test case update * Update DefaultAsyncQueryDAO.java * implement cancel function (#1367) * implement cancel function * addressing comments * addressing comments * Future implementation * Future implementation * Future implementation * Future implementation * Future implementation * adding Future Task * adding Future Task * adding Future Task * fixing future implementation * fixing future implementation * fixing future implementation * fixing future implementation * fixing future implementation * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing issues * fixing bugs * fixing issues * fixing issues * fixing issues * fixing issues * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * fixing bugs * ExecutionException * fixing issues * aggregation changes * aggregation changes * aggregation changes * fixing bugs * fixing bugs * fixing bugs * rebasing * adddressing comments * adddressing comments * addressing comments * addressing comments Co-authored-by: Ramsha Rao * Enable query caching by default (#1378) * Create new cache package * Add caffeine cache based impl * Configure Caffeine cache by default * Publish cache metrics under Spring/micrometer * Lint * Fix javadoc * Rename max cache size for clarity * Update ElideAutoConfiguration.java Co-authored-by: Aaron Klish * Multiplex Transaction Lazy Initialize and Dynamic Config fixes (#1383) * CARBON-212 TimeZone Serde (#1386) * serde code * default val * timezone serde tests * refactor * review comments * Fixes for query cache spring integration (#1381) * Test that query cache is active and publishing metrics * Update README.md with new property * Align columns * Allow injected Cache to be null * Fix typo * [maven-release-plugin] prepare release 5.0.0-pr11 * [maven-release-plugin] prepare for next development iteration * Update: Remove slf4j-simple dependency (#1402) * Fix: Check for null super in `getInheritedTypes` (#1401) * Carbon 167 Async Enhancements Embeddable (#1353) * changes from old branch for async enhancements * remove dependencies and column annotation * pr11 * remove commented line * gs * async after test scenarios per review comments * reduce redundancy * changes to async query execution * review tweaks * remove unused method * Test cases * integration test * exception name * commit hook and async it tests * Async Happy Path tests and new delay datastore * extend IntegrationTest * checkstyle fix * transient update query worker * exception catch change * unit test and comments * codacy * codacy * remove imports * add to remove * remove interrupt thread * interrrupt thread * delete interrupt thread * add back interrupt thread * Review comments * add interrupt * remove interrupt * include interrupt thread to overcome merge conflict * merge 5.x * test changes * remove asyncinterruptthread * graphql tests and remove mocking for queryObj * dsl generation of queries for happy path graphql tests * review changes * move sleep delay flag to aftereach method * [maven-release-plugin] prepare release 5.0.0-pr12 * [maven-release-plugin] prepare for next development iteration * Async change to use onReadPreSecurity Hook (#1409) * Fix for graphql async * Aysnc ReadPreSecurity Hook * Review comments * Review comments * Review comments * fix for NPE * checkstyle fix * Carbon 213 - Add Category to table and columns (#1407) * add category to aggregate * initial working code * refactor testcases: * test table category * remove category for metric function * Added tests for GraphQL Fragment Regression (#1414) Co-authored-by: Aaron Klish * [maven-release-plugin] prepare release 5.0.0-pr13 * [maven-release-plugin] prepare for next development iteration * Elide 5.x - Async Tests, Break loop when fail (#1422) * Loop Break when query fails * adding assert fail * Removing queued * Import fix * Tomcat version fix * Assert import * Error message * Elide 5.x Async Create Permission Fix (#1425) * NCreate Permission Check on AsyncQuery * Create Permission Check on AsyncQuery * Updating test case * Cleanup * Elide 5- Unify request ids (#1423) * Unified request ID for elide 5 * Fixing issues * Fixed some tests * Fixed build * Removed Data Store Transaction changes for request ID * Inspection rework Co-authored-by: Aaron Klish * Basic Implementation Ready * Tests Updated * AggregationDS Query Logger Implemented * First set of reviews fixed * Second Set of Reviews Fixed * Thirs Set of Reviews Fixed * Fourth Set of Reviews Fixed Co-authored-by: Jack (정환) Co-authored-by: moiz arafat Co-authored-by: AvaniMakwana Co-authored-by: rishi-aga Co-authored-by: Aaron Klish Co-authored-by: ramshakr Co-authored-by: Ramsha Rao Co-authored-by: Ashish Aggarwal Co-authored-by: aaggarwal Co-authored-by: John Karp Co-authored-by: Chandrasekar-Rajasekar Co-authored-by: Chandrasekar Rajasekar Co-authored-by: Rkr1992 <65733786+Rkr1992@users.noreply.github.com> Co-authored-by: rishi-aga <61517466+rishi-aga@users.noreply.github.com> Co-authored-by: amakwana Co-authored-by: Aaron Klish Co-authored-by: ayeswarya Co-authored-by: Jon Kilroy --- .../elide/core/DataStoreTransaction.java | 3 +- .../com/yahoo/elide/core/RequestScope.java | 5 +- .../inmemory/HashMapStoreTransaction.java | 2 +- .../inmemory/InMemoryStoreTransaction.java | 4 +- .../datastore/wrapped/TransactionWrapper.java | 4 +- .../elide/core/DataStoreTransactionTest.java | 2 +- .../aggregation/AggregationDataStore.java | 4 +- .../AggregationDataStoreTransaction.java | 61 ++++++--- .../datastores/aggregation/QueryEngine.java | 7 + .../aggregation/core/NoopQueryLogger.java | 41 ++++++ .../aggregation/core/QueryLogger.java | 55 ++++++++ .../aggregation/core/QueryResponse.java | 19 +++ .../queryengines/sql/SQLQueryEngine.java | 5 + .../AggregationDataStoreTransactionTest.java | 127 +++++++++++++++++- .../AggregationDataStoreTestHarness.java | 2 + .../hibernate3/HibernateTransaction.java | 2 +- .../hibernate5/HibernateTransaction.java | 2 +- .../transaction/AbstractJpaTransaction.java | 2 +- .../multiplex/MultiplexTransaction.java | 4 +- .../datastores/multiplex/TestDataStore.java | 2 +- .../bridgeable/BridgeableRedisStore.java | 2 +- .../datastores/noop/NoopTransaction.java | 2 +- .../spring/config/ElideAutoConfiguration.java | 24 +++- .../config/ElideStandaloneSettings.java | 3 +- 24 files changed, 338 insertions(+), 46 deletions(-) create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/NoopQueryLogger.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryLogger.java create mode 100644 elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryResponse.java diff --git a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java index 2e3be0cc11..f931f57761 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java @@ -266,6 +266,7 @@ default boolean supportsPagination(Class entityClass, FilterExpression expres /** * Cancel running transaction. * Implementation must be thread-safe. + * @param scope contains request level metadata. */ - void cancel(); + void cancel(RequestScope scope); } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java index 5b4cf9ae69..a57584e276 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/RequestScope.java @@ -65,10 +65,11 @@ public class RequestScope implements com.yahoo.elide.security.RequestScope { @Getter private final int updateStatusCode; @Getter private final MultipleFilterDialect filterDialect; @Getter private final String apiVersion; + @Getter @Setter private Map headers; //TODO - this ought to be read only and set in the constructor. @Getter @Setter private EntityProjection entityProjection; - private final UUID requestId; + @Getter private final UUID requestId; private final Map expressionsByType; private PublishSubject lifecycleEvents; @@ -143,6 +144,7 @@ public RequestScope(String baseUrlEndPoint, this.dirtyResources = new LinkedHashSet<>(); this.deletedResources = new LinkedHashSet<>(); this.requestId = requestId; + this.headers = new HashMap<>(); Function permissionExecutorGenerator = elideSettings.getPermissionExecutor(); this.permissionExecutor = (permissionExecutorGenerator == null) @@ -231,6 +233,7 @@ protected RequestScope(String path, String apiVersion, this.updateStatusCode = outerRequestScope.updateStatusCode; this.queuedLifecycleEvents = outerRequestScope.queuedLifecycleEvents; this.requestId = outerRequestScope.requestId; + this.headers = outerRequestScope.headers; } public Set getNewResources() { diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java index cc3ccbcf73..d782e09fe7 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/HashMapStoreTransaction.java @@ -189,7 +189,7 @@ private boolean containsObject(Object obj) { } @Override - public void cancel() { + public void cancel(RequestScope scope) { //nothing to cancel in HashMap store transaction } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java index 4752764968..061e3de6f9 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/inmemory/InMemoryStoreTransaction.java @@ -469,7 +469,7 @@ private Pair, Optional> splitPagination( } @Override - public void cancel() { - tx.cancel(); + public void cancel(RequestScope scope) { + tx.cancel(scope); } } diff --git a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java index 69f4472207..28ec39074e 100644 --- a/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java +++ b/elide-core/src/main/java/com/yahoo/elide/core/datastore/wrapped/TransactionWrapper.java @@ -126,7 +126,7 @@ public void close() throws IOException { } @Override - public void cancel() { - tx.cancel(); + public void cancel(RequestScope scope) { + tx.cancel(scope); } } diff --git a/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java b/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java index a431da77cb..d45df17508 100644 --- a/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java +++ b/elide-core/src/test/java/com/yahoo/elide/core/DataStoreTransactionTest.java @@ -162,7 +162,7 @@ public void createObject(Object entity, RequestScope scope) { } @Override - public void cancel() { + public void cancel(RequestScope scope) { //nothing } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java index e5d492e8e1..098b52525b 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStore.java @@ -11,6 +11,7 @@ import com.yahoo.elide.core.EntityDictionary; import com.yahoo.elide.datastores.aggregation.annotation.Join; import com.yahoo.elide.datastores.aggregation.cache.Cache; +import com.yahoo.elide.datastores.aggregation.core.QueryLogger; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.metadata.models.TimeDimension; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; @@ -36,6 +37,7 @@ public class AggregationDataStore implements DataStore { @NonNull private final QueryEngine queryEngine; private final Cache cache; private final Set> dynamicCompiledClasses; + private final QueryLogger queryLogger; /** * These are the classes the Aggregation Store manages. @@ -74,6 +76,6 @@ public void populateEntityDictionary(EntityDictionary dictionary) { @Override public DataStoreTransaction beginTransaction() { - return new AggregationDataStoreTransaction(queryEngine, cache); + return new AggregationDataStoreTransaction(queryEngine, cache, queryLogger); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java index 21db3f81d4..e8a8f5d73c 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransaction.java @@ -6,9 +6,13 @@ package com.yahoo.elide.datastores.aggregation; import com.yahoo.elide.core.DataStoreTransaction; +import com.yahoo.elide.core.HttpStatus; import com.yahoo.elide.core.RequestScope; +import com.yahoo.elide.core.exceptions.HttpStatusException; import com.yahoo.elide.datastores.aggregation.cache.Cache; import com.yahoo.elide.datastores.aggregation.cache.QueryKeyExtractor; +import com.yahoo.elide.datastores.aggregation.core.QueryLogger; +import com.yahoo.elide.datastores.aggregation.core.QueryResponse; import com.yahoo.elide.datastores.aggregation.metadata.models.Table; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.query.QueryResult; @@ -19,6 +23,7 @@ import lombok.ToString; import java.io.IOException; + /** * Transaction handler for {@link AggregationDataStore}. */ @@ -27,11 +32,14 @@ public class AggregationDataStoreTransaction implements DataStoreTransaction { private final QueryEngine queryEngine; private final Cache cache; private final QueryEngine.Transaction queryEngineTransaction; + private final QueryLogger queryLogger; - public AggregationDataStoreTransaction(QueryEngine queryEngine, Cache cache) { + public AggregationDataStoreTransaction(QueryEngine queryEngine, Cache cache, + QueryLogger queryLogger) { this.queryEngine = queryEngine; this.cache = cache; this.queryEngineTransaction = queryEngine.beginTransaction(); + this.queryLogger = queryLogger; } @Override @@ -61,27 +69,43 @@ public void createObject(Object entity, RequestScope scope) { @Override public Iterable loadObjects(EntityProjection entityProjection, RequestScope scope) { - Query query = buildQuery(entityProjection, scope); QueryResult result = null; - + QueryResponse response = null; String cacheKey = null; - if (cache != null && !query.isBypassingCache()) { - String tableVersion = queryEngine.getTableVersion(query.getTable(), queryEngineTransaction); - if (tableVersion != null) { - cacheKey = tableVersion + ';' + QueryKeyExtractor.extractKey(query); - result = cache.get(cacheKey); + try { + queryLogger.acceptQuery(scope.getRequestId(), scope.getUser(), scope.getHeaders(), + scope.getApiVersion(), scope.getQueryParams(), scope.getPath()); + Query query = buildQuery(entityProjection, scope); + if (cache != null && !query.isBypassingCache()) { + String tableVersion = queryEngine.getTableVersion(query.getTable(), queryEngineTransaction); + if (tableVersion != null) { + cacheKey = tableVersion + ';' + QueryKeyExtractor.extractKey(query); + result = cache.get(cacheKey); + } } - } - if (result == null) { - result = queryEngine.executeQuery(query, queryEngineTransaction); - if (cacheKey != null) { - cache.put(cacheKey, result); + boolean isCached = result == null ? false : true; + String queryText = queryEngine.explain(query); + queryLogger.processQuery(scope.getRequestId(), query, queryText, isCached); + if (result == null) { + result = queryEngine.executeQuery(query, queryEngineTransaction); + if (cacheKey != null) { + cache.put(cacheKey, result); + } } + if (entityProjection.getPagination() != null && entityProjection.getPagination().returnPageTotals()) { + entityProjection.getPagination().setPageTotals(result.getPageTotals()); + } + response = new QueryResponse(HttpStatus.SC_OK, result.getData(), null); + return result.getData(); + } catch (HttpStatusException e) { + response = new QueryResponse(e.getStatus(), null, e.getMessage()); + throw e; + } catch (Exception e) { + response = new QueryResponse(HttpStatus.SC_INTERNAL_SERVER_ERROR, null, e.getMessage()); + throw e; + } finally { + queryLogger.completeQuery(scope.getRequestId(), response); } - if (entityProjection.getPagination() != null && entityProjection.getPagination().returnPageTotals()) { - entityProjection.getPagination().setPageTotals(result.getPageTotals()); - } - return result.getData(); } @Override @@ -101,7 +125,8 @@ Query buildQuery(EntityProjection entityProjection, RequestScope scope) { } @Override - public void cancel() { + public void cancel(RequestScope scope) { + queryLogger.cancelQuery(scope.getRequestId()); queryEngineTransaction.cancel(); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java index 74c41318f6..79d644cb30 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/QueryEngine.java @@ -193,4 +193,11 @@ public interface Transaction extends AutoCloseable { public Table getTable(String classAlias) { return tables.get(classAlias); } + + /** + * Explains the specified query passed in + * @param query The query customized for a particular persistent storage or storage client + * @return SQL string corresponding to the given query + */ + public abstract String explain(Query query); } diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/NoopQueryLogger.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/NoopQueryLogger.java new file mode 100644 index 0000000000..2a5ef3083a --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/NoopQueryLogger.java @@ -0,0 +1,41 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.core; + +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.security.User; + +import java.util.Map; +import java.util.Optional; +import java.util.UUID; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Default NoopQuery Logger Implementation for Elide + */ +public class NoopQueryLogger implements QueryLogger { + + @Override + public void acceptQuery(UUID queryId, User user, Map headers, String apiVer, + Optional> queryParams, String path) { + //does nothing + } + + @Override + public void processQuery(UUID queryId, Query query, String apiQuery, boolean isCached) { + //does nothing + } + + @Override + public void cancelQuery(UUID queryId) { + //does nothing + } + + @Override + public void completeQuery(UUID queryId, QueryResponse response) { + //does nothing + } +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryLogger.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryLogger.java new file mode 100644 index 0000000000..7a2be19d75 --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryLogger.java @@ -0,0 +1,55 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.core; + +import com.yahoo.elide.datastores.aggregation.query.Query; +import com.yahoo.elide.security.User; + +import java.util.Map; +import java.util.Optional; +import java.util.UUID; +import javax.ws.rs.core.MultivaluedMap; + +/** + * Query Logger Interface for Elide + */ +public interface QueryLogger { + + /** + * Accepts the incoming JSON API query and notes the start time for the query + * @param queryId The RequestScope requestId. + * @param user The Principal user + * @param headers Http Request Headers + * @param apiVer API Version + * @param queryParams QueryParams for the incoming JSON API query + * @param path The apiQuery endpoint path for the incoming query + */ + void acceptQuery(UUID queryId, User user, Map headers, String apiVer, + Optional> queryParams, String path); + + /** + * Processes and logs all the queries from QueryDetail + * @param queryId The RequestScope requestId. + * @param query The underlying Query + * @param apiQuery The output querytext + * @param isCached Whether the result came from a cache or not + */ + void processQuery(UUID queryId, Query query, String apiQuery, boolean isCached); + + /** + * Cancels all queries currently running for a particular requestId + * Implementation must be thread-safe. + * @param queryId The RequestScope requestId. + */ + void cancelQuery(UUID queryId); + + /** + * Registers the endtime for a query and logs it out + * @param queryId The RequestScope requestId. + * @param response The ElideResponse object + */ + void completeQuery(UUID queryId, QueryResponse response); +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryResponse.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryResponse.java new file mode 100644 index 0000000000..d08cb6100d --- /dev/null +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/core/QueryResponse.java @@ -0,0 +1,19 @@ +/* + * Copyright 2020, Yahoo Inc. + * Licensed under the Apache License, Version 2.0 + * See LICENSE file in project root for terms. + */ +package com.yahoo.elide.datastores.aggregation.core; + +import lombok.AllArgsConstructor; +import lombok.Getter; + +/** + * QueryLogger Response Class + */ +@AllArgsConstructor +public class QueryResponse { + @Getter private final int responseCode; + @Getter private final Iterable data; + @Getter private final String errorMessage; +} diff --git a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java index 869de2d9c5..c392cb2717 100644 --- a/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java +++ b/elide-datastore/elide-datastore-aggregation/src/main/java/com/yahoo/elide/datastores/aggregation/queryengines/sql/SQLQueryEngine.java @@ -228,6 +228,11 @@ public String getTableVersion(Table table, Transaction transaction) { return tableVersion; } + @Override + public String explain(Query query) { + return toSQL(query).toString(); + } + /** * Translates the client query into SQL. * diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransactionTest.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransactionTest.java index 2e83131659..e25e3dcf32 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransactionTest.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/AggregationDataStoreTransactionTest.java @@ -11,17 +11,23 @@ import static org.junit.jupiter.api.Assertions.assertEquals; import static org.mockito.ArgumentMatchers.any; import static org.mockito.ArgumentMatchers.anyString; +import static org.mockito.ArgumentMatchers.argThat; +import static org.mockito.Mockito.doThrow; import static org.mockito.Mockito.never; +import static org.mockito.Mockito.times; import static org.mockito.Mockito.when; import com.yahoo.elide.core.RequestScope; import com.yahoo.elide.core.pagination.PaginationImpl; import com.yahoo.elide.datastores.aggregation.cache.Cache; import com.yahoo.elide.datastores.aggregation.cache.QueryKeyExtractor; +import com.yahoo.elide.datastores.aggregation.core.QueryLogger; +import com.yahoo.elide.datastores.aggregation.core.QueryResponse; import com.yahoo.elide.datastores.aggregation.example.PlayerStats; import com.yahoo.elide.datastores.aggregation.framework.SQLUnitTest; import com.yahoo.elide.datastores.aggregation.query.Query; import com.yahoo.elide.datastores.aggregation.query.QueryResult; +import com.yahoo.elide.datastores.aggregation.queryengines.sql.query.SQLQuery; import com.yahoo.elide.request.EntityProjection; import com.yahoo.elide.request.Pagination; @@ -42,6 +48,7 @@ class AggregationDataStoreTransactionTest extends SQLUnitTest { @Mock private QueryEngine.Transaction qeTransaction; @Mock private RequestScope scope; @Mock private Cache cache; + @Mock private QueryLogger queryLogger; private Query query = Query.builder().table(playerStatsTable).build(); private final String queryKey = QueryKeyExtractor.extractKey(query); @@ -50,8 +57,8 @@ class AggregationDataStoreTransactionTest extends SQLUnitTest { // inject our own query instead of using buildQuery impl private class MyAggregationDataStoreTransaction extends AggregationDataStoreTransaction { - public MyAggregationDataStoreTransaction(QueryEngine queryEngine, Cache cache) { - super(queryEngine, cache); + public MyAggregationDataStoreTransaction(QueryEngine queryEngine, Cache cache, QueryLogger queryLogger) { + super(queryEngine, cache, queryLogger); } @Override @@ -72,10 +79,17 @@ public void setUp() { @Test public void loadObjectsPopulatesCache() { + Mockito.reset(queryLogger); + QueryResult queryResult = QueryResult.builder().data(DATA).build(); + SQLQuery myQuery = SQLQuery.builder().clientQuery(query) + .fromClause(query.getTable().getName()) + .projectionClause(" ").build(); when(queryEngine.getTableVersion(playerStatsTable, qeTransaction)).thenReturn("foo"); when(queryEngine.executeQuery(query, qeTransaction)).thenReturn(queryResult); - AggregationDataStoreTransaction transaction = new MyAggregationDataStoreTransaction(queryEngine, cache); + when(queryEngine.explain(query)).thenReturn(myQuery.toString()); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); EntityProjection entityProjection = EntityProjection.builder().type(PlayerStats.class).build(); assertEquals(DATA, transaction.loadObjects(entityProjection, scope)); @@ -84,15 +98,29 @@ public void loadObjectsPopulatesCache() { Mockito.verify(cache).get(cacheKey); Mockito.verify(cache).put(cacheKey, queryResult); Mockito.verifyNoMoreInteractions(cache); + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(false)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); } @Test public void loadObjectsUsesCache() { + Mockito.reset(queryLogger); + String cacheKey = "foo;" + queryKey; QueryResult queryResult = QueryResult.builder().data(DATA).build(); + SQLQuery myQuery = SQLQuery.builder().clientQuery(query) + .fromClause(query.getTable().getName()) + .projectionClause(" ").build(); when(cache.get(cacheKey)).thenReturn(queryResult); when(queryEngine.getTableVersion(playerStatsTable, qeTransaction)).thenReturn("foo"); - AggregationDataStoreTransaction transaction = new MyAggregationDataStoreTransaction(queryEngine, cache); + when(queryEngine.explain(query)).thenReturn(myQuery.toString()); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); EntityProjection entityProjection = EntityProjection.builder().type(PlayerStats.class).build(); assertEquals(DATA, transaction.loadObjects(entityProjection, scope)); @@ -100,14 +128,28 @@ public void loadObjectsUsesCache() { Mockito.verify(queryEngine, never()).executeQuery(any(), any()); Mockito.verify(cache).get(cacheKey); Mockito.verifyNoMoreInteractions(cache); + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(true)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); } @Test public void loadObjectsPassesPagination() { + Mockito.reset(queryLogger); + QueryResult queryResult = QueryResult.builder().data(DATA).pageTotals(314L).build(); + SQLQuery myQuery = SQLQuery.builder().clientQuery(query) + .fromClause(query.getTable().getName()) + .projectionClause(" ").build(); when(cache.get(anyString())).thenReturn(queryResult); when(queryEngine.getTableVersion(playerStatsTable, qeTransaction)).thenReturn("foo"); - AggregationDataStoreTransaction transaction = new MyAggregationDataStoreTransaction(queryEngine, cache); + when(queryEngine.explain(query)).thenReturn(myQuery.toString()); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); Pagination pagination = new PaginationImpl( String.class, null, null, DEFAULT_PAGE_LIMIT, MAX_PAGE_LIMIT, true, false); EntityProjection entityProjection = EntityProjection.builder() @@ -120,32 +162,103 @@ public void loadObjectsPassesPagination() { Mockito.verify(queryEngine, never()).executeQuery(any(), any()); Mockito.verify(cache).get(cacheKey); Mockito.verifyNoMoreInteractions(cache); + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(true)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); } @Test public void loadObjectsNoTableVersion() { + Mockito.reset(queryLogger); + + SQLQuery myQuery = SQLQuery.builder().clientQuery(query) + .fromClause(query.getTable().getName()) + .projectionClause(" ").build(); when(queryEngine.executeQuery(query, qeTransaction)) .thenReturn(QueryResult.builder().data(Collections.emptyList()).build()); - AggregationDataStoreTransaction transaction = new MyAggregationDataStoreTransaction(queryEngine, cache); + when(queryEngine.explain(query)).thenReturn(myQuery.toString()); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); EntityProjection entityProjection = EntityProjection.builder().type(PlayerStats.class).build(); transaction.loadObjects(entityProjection, scope); Mockito.verifyNoInteractions(cache); + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(false)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); } @Test public void loadObjectsBypassCache() { + Mockito.reset(queryLogger); + query = Query.builder().table(playerStatsTable).bypassingCache(true).build(); + SQLQuery myQuery = SQLQuery.builder().clientQuery(query) + .fromClause(query.getTable().getName()) + .projectionClause(" ").build(); QueryResult queryResult = QueryResult.builder().data(DATA).build(); when(queryEngine.executeQuery(query, qeTransaction)).thenReturn(queryResult); - AggregationDataStoreTransaction transaction = new MyAggregationDataStoreTransaction(queryEngine, cache); + when(queryEngine.explain(query)).thenReturn(myQuery.toString()); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); EntityProjection entityProjection = EntityProjection.builder().type(PlayerStats.class).build(); assertEquals(DATA, transaction.loadObjects(entityProjection, scope)); Mockito.verify(queryEngine, never()).getTableVersion(any(), any()); Mockito.verifyNoInteractions(cache); + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(false)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); + } + + @Test + public void loadObjectsExceptionThrownTest() throws Exception { + Mockito.reset(queryLogger); + String nullPointerExceptionMessage = "Cannot dereference an object with value Null"; + try { + query = Query.builder().table(playerStatsTable).bypassingCache(true).build(); + doThrow(new NullPointerException(nullPointerExceptionMessage)) + .when(queryEngine).executeQuery(query, qeTransaction); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); + EntityProjection entityProjection = EntityProjection.builder().type(PlayerStats.class).build(); + transaction.loadObjects(entityProjection, scope); + } catch (Exception e) { + assertEquals(nullPointerExceptionMessage, e.getMessage()); + Mockito.verify(queryLogger).completeQuery(Mockito.eq(scope.getRequestId()), + argThat((QueryResponse qResponse) -> qResponse.getErrorMessage() == e.getMessage())); + } + + Mockito.verify(queryLogger, times(1)).acceptQuery( + Mockito.eq(scope.getRequestId()), + any(), any(), any(), any(), any()); + Mockito.verify(queryLogger, times(1)).processQuery( + Mockito.eq(scope.getRequestId()), any(), any(), Mockito.eq(false)); + Mockito.verify(queryLogger, times(1)).completeQuery( + Mockito.eq(scope.getRequestId()), any()); + } + + @Test + public void aggregationQueryLoggerCancelQueryTest() { + Mockito.reset(queryLogger); + AggregationDataStoreTransaction transaction = + new MyAggregationDataStoreTransaction(queryEngine, cache, queryLogger); + transaction.cancel(scope); + Mockito.verify(queryLogger, times(1)).cancelQuery(Mockito.eq(scope.getRequestId())); } } diff --git a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java index 44ca681891..51af2d1105 100644 --- a/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java +++ b/elide-datastore/elide-datastore-aggregation/src/test/java/com/yahoo/elide/datastores/aggregation/framework/AggregationDataStoreTestHarness.java @@ -8,6 +8,7 @@ import com.yahoo.elide.core.DataStore; import com.yahoo.elide.core.datastore.test.DataStoreTestHarness; import com.yahoo.elide.datastores.aggregation.AggregationDataStore; +import com.yahoo.elide.datastores.aggregation.core.NoopQueryLogger; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.jpa.JpaDataStore; @@ -35,6 +36,7 @@ public DataStore getDataStore() { AggregationDataStore aggregationDataStore = AggregationDataStore.builder() .queryEngine(new SQLQueryEngine(metaDataStore, entityManagerFactory, txCancel)) + .queryLogger(new NoopQueryLogger()) .build(); DataStore jpaStore = new JpaDataStore( diff --git a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java index a507fffb9e..5d7c7c9636 100644 --- a/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate3/src/main/java/com/yahoo/elide/datastores/hibernate3/HibernateTransaction.java @@ -306,7 +306,7 @@ public Integer getQueryLimit() { } @Override - public void cancel() { + public void cancel(RequestScope scope) { session.cancelQuery(); } } diff --git a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java index 51c9024af9..5b59f2a18c 100644 --- a/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java +++ b/elide-datastore/elide-datastore-hibernate5/src/main/java/com/yahoo/elide/datastores/hibernate5/HibernateTransaction.java @@ -302,7 +302,7 @@ public void close() throws IOException { } @Override - public void cancel() { + public void cancel(RequestScope scope) { session.cancelQuery(); } } diff --git a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java index ca3c3f5a0b..b56804c8b4 100644 --- a/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java +++ b/elide-datastore/elide-datastore-jpa/src/main/java/com/yahoo/elide/datastores/jpa/transaction/AbstractJpaTransaction.java @@ -311,7 +311,7 @@ private Long getTotalRecords(AbstractHQLQueryBuilder.Relationship relationsh } @Override - public void cancel() { + public void cancel(RequestScope scope) { jpaTransactionCancel.accept(em); } } diff --git a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java index a3f2048e72..3209f6d6bf 100644 --- a/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java +++ b/elide-datastore/elide-datastore-multiplex/src/main/java/com/yahoo/elide/datastores/multiplex/MultiplexTransaction.java @@ -245,7 +245,7 @@ private Serializable extractId(FilterExpression filterExpression, } @Override - public void cancel() { - transactions.values().forEach(dataStoreTransaction -> dataStoreTransaction.cancel()); + public void cancel(RequestScope scope) { + transactions.values().forEach(dataStoreTransaction -> dataStoreTransaction.cancel(scope)); } } diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java index d861eaeb18..063fab3841 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/TestDataStore.java @@ -79,7 +79,7 @@ public Iterable loadObjects( } @Override - public void cancel() { + public void cancel(RequestScope scope) { // Nothing } } diff --git a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java index 57bb8141aa..3d0d20e330 100644 --- a/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java +++ b/elide-datastore/elide-datastore-multiplex/src/test/java/com/yahoo/elide/datastores/multiplex/bridgeable/BridgeableRedisStore.java @@ -244,7 +244,7 @@ public T createNewObject(Class entityClass) { } @Override - public void cancel() { + public void cancel(RequestScope scope) { // Nothing } } diff --git a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java index 59ba8c2431..9769e25eba 100644 --- a/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java +++ b/elide-datastore/elide-datastore-noop/src/main/java/com/yahoo/elide/datastores/noop/NoopTransaction.java @@ -123,7 +123,7 @@ public void close() throws IOException { * No-op transaction, do nothing. */ @Override - public void cancel() { + public void cancel(RequestScope scope) { // No-op transaction, do nothing. } } diff --git a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java index c8ff2a8ec3..e498edf4b9 100644 --- a/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java +++ b/elide-spring/elide-spring-boot-autoconfigure/src/main/java/com/yahoo/elide/spring/config/ElideAutoConfiguration.java @@ -19,6 +19,8 @@ import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.cache.Cache; import com.yahoo.elide.datastores.aggregation.cache.CaffeineCache; +import com.yahoo.elide.datastores.aggregation.core.NoopQueryLogger; +import com.yahoo.elide.datastores.aggregation.core.QueryLogger; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; @@ -35,6 +37,7 @@ import org.springframework.boot.context.properties.EnableConfigurationProperties; import org.springframework.context.annotation.Bean; import org.springframework.context.annotation.Configuration; +import org.springframework.context.annotation.DependsOn; import io.micrometer.core.instrument.MeterRegistry; import io.micrometer.core.instrument.binder.cache.CaffeineCacheMetrics; @@ -180,9 +183,13 @@ public QueryEngine buildQueryEngine(EntityManagerFactory entityManagerFactory, */ @Bean @ConditionalOnMissingBean - public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, QueryEngine queryEngine, - ObjectProvider dynamicCompiler, ElideConfigProperties settings, - @Autowired(required = false) Cache cache) + @DependsOn({"buildQueryLogger"}) + public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, + QueryEngine queryEngine, + ObjectProvider dynamicCompiler, + ElideConfigProperties settings, + @Autowired(required = false) Cache cache, + @Autowired(required = false) QueryLogger querylogger) throws ClassNotFoundException { AggregationDataStore.AggregationDataStoreBuilder aggregationDataStoreBuilder = AggregationDataStore.builder() .queryEngine(queryEngine); @@ -193,6 +200,7 @@ public DataStore buildDataStore(EntityManagerFactory entityManagerFactory, Query aggregationDataStoreBuilder.dynamicCompiledClasses(annotatedClass); } aggregationDataStoreBuilder.cache(cache); + aggregationDataStoreBuilder.queryLogger(querylogger); AggregationDataStore aggregationDataStore = aggregationDataStoreBuilder.build(); JpaDataStore jpaDataStore = new JpaDataStore(entityManagerFactory::createEntityManager, @@ -220,6 +228,16 @@ public Cache buildQueryCache(ElideConfigProperties settings) { return cache; } + /** + * Creates a querylogger to be used by {@link #buildDataStore} for aggregation + * @return The default Noop QueryLogger. + */ + @Bean + @ConditionalOnMissingBean + public QueryLogger buildQueryLogger() { + return new NoopQueryLogger(); + } + /** * Creates a singular swagger document for JSON-API. * @param dictionary Contains the static metadata about Elide models. diff --git a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java index b108253a78..2ab89deadf 100644 --- a/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java +++ b/elide-standalone/src/main/java/com/yahoo/elide/standalone/config/ElideStandaloneSettings.java @@ -24,6 +24,7 @@ import com.yahoo.elide.datastores.aggregation.QueryEngine; import com.yahoo.elide.datastores.aggregation.cache.Cache; import com.yahoo.elide.datastores.aggregation.cache.CaffeineCache; +import com.yahoo.elide.datastores.aggregation.core.NoopQueryLogger; import com.yahoo.elide.datastores.aggregation.metadata.MetaDataStore; import com.yahoo.elide.datastores.aggregation.queryengines.sql.SQLQueryEngine; import com.yahoo.elide.datastores.aggregation.queryengines.sql.annotation.FromSubquery; @@ -419,7 +420,7 @@ default DataStore getDataStore(MetaDataStore metaDataStore, AggregationDataStore default AggregationDataStore getAggregationDataStore(QueryEngine queryEngine, Optional optionalCompiler) { AggregationDataStore.AggregationDataStoreBuilder aggregationDataStoreBuilder = AggregationDataStore.builder() - .queryEngine(queryEngine); + .queryEngine(queryEngine).queryLogger(new NoopQueryLogger()); if (enableDynamicModelConfig()) { Set> annotatedClasses = getDynamicClassesIfAvailable(optionalCompiler, FromTable.class);