Create AggregationDataStore module (yahoo#845)

* Create AggregationDataStore module * Address Aaron's comments * Fix build failure AggregationDataStore: Schema (yahoo#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Define QueryEngine Contract (yahoo#867) Fixed rebase on master SQL Query Engine (yahoo#878) * AggregationDataStore: Schema (yahoo#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron Added calcite as a dependency. Merged in changes for QueryEngine interface Fixed checkstyle issues Added basic H2 DB test harness Started breaking out projections Moved getValue and setValue from PersistentResource to EntityDictionary Added basic logic to hydrate entities Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration Minor cleanup Refactored HQLFilterOperation to take an alias generator Added test support for RSQL filter generation. Some cleanup Added basic support for WHERE clause filtering on the fact table Added working test for subquery SQL Added basic join logic for filters Added a test with a subquery and a filter join Refactored Schema classes and Query to support metric aggregation SQL expansion Added group by support Added logic for ID generation Added sorting logic and test Added pagination support and testing All column references use proper name now for SQL Removed calcite as a query engine Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton Added HAVING clause support Changed Query to take schema instead of entityClass First pass at cleanup Fixed checkstyles Cleanup Cleanup Added a complex SQL expression test and fixed bugs Fixed merge issues. Added another test. Added better logging Fixed bug in pagination SQL generation * Build is working * Inspection rework Add EntityProjection plumbing (yahoo#949) * AggregationDataStore: Schema (yahoo#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * Removed duplicated Schema class from rebase * Entity projection with aliases (yahoo#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * Wire in entity projection4 json api (yahoo#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (yahoo#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup Hydrate Relationship (yahoo#987) * AggregationDataStore: Schema (yahoo#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Added basic H2 DB test harness * Started breaking out projections * Moved getValue and setValue from PersistentResource to EntityDictionary * Added basic logic to hydrate entities * Added FromTable and FromSubquery annotations. Add explicit exclusion of entity relationship hydration * Refactored HQLFilterOperation to take an alias generator * Added test support for RSQL filter generation. Some cleanup * Added basic support for WHERE clause filtering on the fact table * Added working test for subquery SQL * Added basic join logic for filters * Added a test with a subquery and a filter join * Refactored Schema classes and Query to support metric aggregation SQL expansion * Added group by support * Added logic for ID generation * Added sorting logic and test * Added pagination support and testing * All column references use proper name now for SQL * Removed calcite as a query engine * Refactored HQLFilterOperation so it can be used for Having and Where clause generaiton * Added HAVING clause support * Changed Query to take schema instead of entityClass * First pass at cleanup * Fixed checkstyles * Cleanup * Hydrate Relationship * Cleanup * Added a complex SQL expression test and fixed bugs * Fixed merge issues. Added another test. Added better logging * Self-review * Self-review * Self-review * Self-review * Self-review * Address comments from @aklish * Refactor EntityHydrator (yahoo#893) * rebase * keep Jiaqi's changes * fix id * fix maven verify * Remove HQLFilterOperation * fix dictionary * fix SqlEngineTest * remove unused part * make codacy happy * should use getParametrizedType * address comments Implement GraphQLEntityProjectionMaker (yahoo#986) * AggregationDataStore: Schema (yahoo#846) * AggregationDataStore: Static Attribute Aggregation * Address comments * Implement TimeDimension and all its supporting components * refactor * Address comments from @aklish * Address comments from @aklish && Tests & Javadoc * Address comments from @aklish * Address comments from @aklish and @hellohanchen * Address comments from Aaron * ToMany is not supported * Address comments from Aaron * Initial sketch PersistentResourceTest now passes LifeCycleTest tests now pass More API changes for data store transaction. Also fixed createObject in persistent resource to take the correct projection Started to refactor tests IncludedProcessorTest refactored Refactored LifeCycleTest Started refactor on PersistentResourceTest More refactoring. Fixed a bug in Resource.toPersistentResource Only one test failing in PersistentResourceTest PersistentResourceTests now pass UpdateOnCreateTests now pass Added skeleton for translating JSON-API URL path into an EntityProjection Basic EntityProjectionMaker almost complete Added ability to merge entity projections by relationship Added first test for EntityProjectionMaker Non-working veresion (but clean) Tests now pass All EntityProjectioNMaker tests pass Elide-Core now builds Added handling of sparse attributes and relationships Expanding attributes for included entities Fixed a number of bugs found in IT tests Fixed some of the EntityProjectionMaker tests Fixed unit tests Made temporary modifications to exclude GraphQL (Build now passes) Added sparse field unit tests for EntityProjectionMaker * Fixed build issues after rebase * GraphQL projection maker using document * Argument handling and fragment check * Add comments * Add fragment resolver * fix typo * break code into more methods * remove pagination and sorting * Removed duplicated Schema class from rebase * re-arrange keywords * Address comment * Add arguments for attribute fields * Handle arguments * support partial query, update edges/node logic * Entity projection with aliases * Entity projection with aliases (yahoo#963) * Hacked up PersistentResource with new design * Core now compiles (and tests can run * EntityProjectionMaker tests pass * Build now passes (major cleanup still needed * fix create relationship object using entity * Add tests passed * code clean up * refactor fatcher, fix test cases * rename keywords * rebase branch (#12) * rebased * Graphql projection refactor (#13) * fix fragment resolver * Fix variable resolver * Wire in entity projection4 json api (yahoo#964) * Fixed DataStore API. Fixed a lot of the core unit tests * Checkstyles and more fixes * Hibernate 5 Tests Pass * Full build passes * Wire in entity projection4 json api (yahoo#965) * Initial concept. No testing changed. * Core compiles and EntityProjectionMaker tests (original ones) now pass * Minor edits to TestRequestScope * Full build passes now * removed entity dictionary from entity projection * Pre-inspection cleanup * minor inspection fixup * rebase * Rebased on AggregationDataStore * clean up extra new lines * address comments * Builder pattern * update comments * remove projection in entity * fix jackson * Hydrate Relationship (yahoo#987) (#15) * Address some codecy comments * Add comment for partial query * Reenable tests * Address comments, refactor alias * Add test for alias * swapped test case * fix get type Added AggregationDataStore Code (yahoo#991) * Adding testing for aggregation data store * Debugging integration tests * Continuing testing work * AggregationDataStore * AggregationDataStore testing * Added more tests * Aggregation Data Store * Cleaned up testing code * Cleaned up code, fixed helper for AggregationDataStore * end * Fixed checkstyle, other minor fixes * fixed comment * Minor fixes * Fixed id type issue, added exception for queries with no metrics Fixed build (yahoo#993) Making TimeDimension an interface (yahoo#992) [maven-release-plugin] prepare release 5.0.0-pr1 [maven-release-plugin] prepare for next development iteration Renamed graphQL file to match test (yahoo#1002) [maven-release-plugin] prepare release 5.0.0-pr2 [maven-release-plugin] prepare for next development iteration Add JoinTo annotation (yahoo#1006) * Added JoinTo Annotation * Added working test * Added TODO comment for next PR * Added TODO comment for next PR * Added Sorting and Filtering support for JoinTo Columns * Fixed IT tests for Aggregation Data Store * Moved entityManager creation to happen separately for each query (yahoo#1008) * Moved entityManager creation to happen separately for each query * Closing EntityManager after each query * Inspection rework Column annotation (yahoo#1017) * Solved column issue and added QueryEngineFactory * Caching query engine in AggregationDataStore * Fixed column description * Update SQLQueryEngine.java (yahoo#1019) * Add SQLMetrix, rearrange packages (yahoo#1020) * Add SQLMetrix, rearrange packages * address comment Manager transacton manually (yahoo#1021) * Manager transacton manually * Add readonly Hydrate GraphQL Schema with parameterized attributes (yahoo#1018) * GraphQL schema expose expected argument name and its type for each attribute * Change empty arguments to unmodifable set AggregationStore: Add multiple time grain definitions to schema (yahoo#1022) * Fixed checkstyle warnings and errors. Separated the Query dimension interface from the Schema dimension interface * Added skeleton code to convert entity projection arguments into time grains * Cleanup * Class renames per inspection comments * Inspection comments Refactor time dimension logic (yahoo#1028) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception ISSUE-1026 Add support for @subselect (yahoo#1038) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1026 Add support for @subselect * Address comments ISSUE-1027 Support join for having clause (yahoo#1039) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * ISSUE-1027 Support join for having clause function name fixed to enableISO8601Dates (yahoo#1052) Support for multiple queries at root is added (yahoo#1044) * Support for multiple queries at root is added * Added test with alias * comments resolved Add time grain to GraphQL schema (yahoo#1042) * Added basic plumbing to push attributes from the entity projection down to the QueryEngine * Added logic to expand SQL time expression in SQLQueryEngine * Added SQLQueryEngine tests * Added IT tests * The AggregationStore now adds graphql parameters for parameterized columns * Minor refactor * Inspection rework * Minor fix Support multiple query of same entity with different alias (yahoo#1055) * Support multiple query of same entity with different alias * add static method to generate keyname for GraphQLProjectionInfo projections * Remove aliasPartialQuerySameAttribute MetadataStore Models (yahoo#1068) * Manager transacton manually * Add readonly * some rework * use getTimeDimension() * change exception * Metadatastore models * Address comments * address comments * move root * fix style check SQLQueryTemplate Model (yahoo#1073) * SQLQueryTemplate * SQLTables * refactor * update sql dimension projection * update sql dimension projection * clean up dimension projection * refactor sql components * aggregatable field rework * add comments * rearrange packages * Add dimension projection back * fix checkstyle * Add dictionary * Simplify MetricFunction and SQLQueryTemplate * Address comments Integrate Metadata Model and SQLQueryTemplate Model (yahoo#1083) * Integrate Metadata Model and SQLQueryTemplate Model * remove AggregationDictionary and AggregationManager * Add timezone * Can only query analyticView
hellohanchen · Dec 19, 2019 · 98fd8ef · 98fd8ef
1 parent 279c86b
commit 98fd8ef
Show file tree

Hide file tree

Showing 234 changed files with 12,440 additions and 1,909 deletions.
diff --git a/checkstyle-suppressions.xml b/checkstyle-suppressions.xml
@@ -10,7 +10,7 @@
     "http://www.puppycrawl.com/dtds/suppressions_1_0.dtd">
 <suppressions>
   <!-- Suppress generated sources -->
-  <suppress files="[/\\]src[/\\]test[/\\]" checks="(JavadocType|LineLength|MethodLength)"/>
+  <suppress files="[/\\]src[/\\]test[/\\]" checks="(JavadocType|LineLength|MethodLength|MethodCount)"/>
   <suppress files="[/\\]src[/\\]main[/\\]webapp[/\\]WEB-INF[/\\]api-docs[/\\]" checks=".*"/>
   <suppress files="[/\\]target[/\\]" checks=".*"/>
   <suppress files="\.csv" checks=".*"/>

diff --git a/elide-annotations/pom.xml b/elide-annotations/pom.xml
@@ -9,7 +9,7 @@
     <parent>
         <groupId>com.yahoo.elide</groupId>
         <artifactId>elide-parent-pom</artifactId>
-        <version>4.5.10-SNAPSHOT</version>
+        <version>5.0.0-pr3-SNAPSHOT</version>
     </parent>
 
     <licenses>
@@ -53,5 +53,4 @@
             </plugin>
         </plugins>
     </build>
-
 </project>
diff --git a/elide-contrib/elide-swagger/pom.xml b/elide-contrib/elide-swagger/pom.xml
@@ -14,7 +14,7 @@
     <parent>
         <artifactId>elide-contrib-parent-pom</artifactId>
         <groupId>com.yahoo.elide</groupId>
-        <version>4.5.10-SNAPSHOT</version>
+        <version>5.0.0-pr3-SNAPSHOT</version>
     </parent>
 
     <licenses>
@@ -42,7 +42,7 @@
         <dependency>
             <groupId>com.yahoo.elide</groupId>
     	    <artifactId>elide-core</artifactId>
-            <version>4.5.10-SNAPSHOT</version>
+            <version>5.0.0-pr3-SNAPSHOT</version>
         </dependency>
 
         <dependency>
@@ -54,7 +54,7 @@
         <dependency>
             <groupId>com.yahoo.elide</groupId>
     	    <artifactId>elide-integration-tests</artifactId>
-            <version>4.5.10-SNAPSHOT</version>
+            <version>5.0.0-pr3-SNAPSHOT</version>
             <type>test-jar</type>
             <scope>test</scope>
         </dependency>

diff --git a/elide-contrib/elide-test-helpers/pom.xml b/elide-contrib/elide-test-helpers/pom.xml
@@ -14,7 +14,7 @@
     <parent>
         <artifactId>elide-contrib-parent-pom</artifactId>
         <groupId>com.yahoo.elide</groupId>
-        <version>4.5.10-SNAPSHOT</version>
+        <version>5.0.0-pr3-SNAPSHOT</version>
     </parent>
 
     <licenses>

diff --git a/...de-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java b/...de-test-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/GraphQLDSL.java
@@ -416,6 +416,10 @@ public static Selection field(String name, Arguments arguments, SelectionSet...
         return new Field(null, name, arguments, relayWrap(Arrays.asList(selectionSet)));
     }
 
+    public static Selection field(String alias, String name, Arguments arguments, SelectionSet... selectionSet) {
+        return new Field(alias, name, arguments, relayWrap(Arrays.asList(selectionSet)));
+    }
+
     /**
      * Creates an attribute(scalar field) selection.
      *

diff --git a/...est-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java b/...est-helpers/src/main/java/com/yahoo/elide/contrib/testhelpers/graphql/elements/Field.java
@@ -83,7 +83,7 @@ public String toGraphQLSpec() {
 
     @Override
     public String toResponse() {
-        if (selectionSet instanceof String) {
+        if (selectionSet instanceof String || selectionSet instanceof Number) {
             // scalar response field
             return String.format(
                     "\"%s\":%s",
@@ -92,7 +92,8 @@ public String toResponse() {
                             ? "{\"edges\":[]}"
                             : getSelectionSet().toString()
             );
-        } else {
+        }
+        else {
             // object response field
             return String.format("\"%s\":%s", getName(), ((SelectionSet) getSelectionSet()).toResponse());
         }

diff --git a/elide-contrib/pom.xml b/elide-contrib/pom.xml
@@ -14,7 +14,7 @@
     <parent>
         <artifactId>elide-parent-pom</artifactId>
         <groupId>com.yahoo.elide</groupId>
-        <version>4.5.10-SNAPSHOT</version>
+        <version>5.0.0-pr3-SNAPSHOT</version>
     </parent>
 
     <licenses>
@@ -53,7 +53,7 @@
             <dependency>
                 <groupId>com.yahoo.elide</groupId>
                 <artifactId>elide-core</artifactId>
-                <version>4.5.10-SNAPSHOT</version>
+                <version>5.0.0-pr3-SNAPSHOT</version>
             </dependency>
         </dependencies>
     </dependencyManagement>

diff --git a/elide-core/pom.xml b/elide-core/pom.xml
@@ -14,7 +14,7 @@
     <parent>
         <groupId>com.yahoo.elide</groupId>
         <artifactId>elide-parent-pom</artifactId>
-        <version>4.5.10-SNAPSHOT</version>
+        <version>5.0.0-pr3-SNAPSHOT</version>
     </parent>
 
     <licenses>

diff --git a/elide-core/src/main/java/com/yahoo/elide/Elide.java b/elide-core/src/main/java/com/yahoo/elide/Elide.java
@@ -22,6 +22,7 @@
 import com.yahoo.elide.core.exceptions.TransactionException;
 import com.yahoo.elide.extensions.JsonApiPatch;
 import com.yahoo.elide.extensions.PatchRequestScope;
+import com.yahoo.elide.jsonapi.EntityProjectionMaker;
 import com.yahoo.elide.jsonapi.JsonApiMapper;
 import com.yahoo.elide.jsonapi.models.JsonApiDocument;
 import com.yahoo.elide.parsers.BaseVisitor;
@@ -91,6 +92,8 @@ public ElideResponse get(String path, MultivaluedMap<String, String> queryParams
         return handleRequest(true, opaqueUser, dataStore::beginReadTransaction, (tx, user) -> {
             JsonApiDocument jsonApiDoc = new JsonApiDocument();
             RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, queryParams, elideSettings);
+            requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(),
+                    requestScope).parsePath(path));
             BaseVisitor visitor = new GetVisitor(requestScope);
             return visit(path, requestScope, visitor);
         });
@@ -109,6 +112,8 @@ public ElideResponse post(String path, String jsonApiDocument, Object opaqueUser
         return handleRequest(false, opaqueUser, dataStore::beginTransaction, (tx, user) -> {
             JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument);
             RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings);
+            requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(),
+                    requestScope).parsePath(path));
             BaseVisitor visitor = new PostVisitor(requestScope);
             return visit(path, requestScope, visitor);
         });
@@ -143,6 +148,8 @@ public ElideResponse patch(String contentType, String accept,
             handler = (tx, user) -> {
                 JsonApiDocument jsonApiDoc = mapper.readJsonApiDocument(jsonApiDocument);
                 RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings);
+                requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(),
+                        requestScope).parsePath(path));
                 BaseVisitor visitor = new PatchVisitor(requestScope);
                 return visit(path, requestScope, visitor);
             };
@@ -165,6 +172,8 @@ public ElideResponse delete(String path, String jsonApiDocument, Object opaqueUs
                     ? new JsonApiDocument()
                     : mapper.readJsonApiDocument(jsonApiDocument);
             RequestScope requestScope = new RequestScope(path, jsonApiDoc, tx, user, null, elideSettings);
+            requestScope.setEntityProjection(new EntityProjectionMaker(elideSettings.getDictionary(),
+                    requestScope).parsePath(path));
             BaseVisitor visitor = new DeleteVisitor(requestScope);
             return visit(path, requestScope, visitor);
         });

diff --git a/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java b/elide-core/src/main/java/com/yahoo/elide/core/ArgumentType.java
@@ -0,0 +1,23 @@
+/*
+ * Copyright 2015, Yahoo Inc.
+ * Licensed under the Apache License, Version 2.0
+ * See LICENSE file in project root for terms.
+ */
+package com.yahoo.elide.core;
+
+import lombok.Getter;
+
+/**
+ * Argument Type wraps an argument to the type of value it accepts.
+ */
+public class ArgumentType {
+    @Getter
+    private String name;
+    @Getter
+    private Class<?> type;
+
+    public ArgumentType(String name, Class<?> type) {
+        this.name = name;
+        this.type = type;
+    }
+}
diff --git a/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java b/elide-core/src/main/java/com/yahoo/elide/core/DataStoreTransaction.java
@@ -8,14 +8,15 @@
 import com.yahoo.elide.core.filter.InPredicate;
 import com.yahoo.elide.core.filter.expression.AndFilterExpression;
 import com.yahoo.elide.core.filter.expression.FilterExpression;
-import com.yahoo.elide.core.pagination.Pagination;
 import com.yahoo.elide.core.sort.Sorting;
+import com.yahoo.elide.request.Attribute;
+import com.yahoo.elide.request.EntityProjection;
+import com.yahoo.elide.request.Relationship;
 import com.yahoo.elide.security.User;
 
 import java.io.Closeable;
 import java.io.Serializable;
 import java.util.Iterator;
-import java.util.Optional;
 import java.util.Set;
 
 /**
@@ -112,34 +113,37 @@ default <T> T createNewObject(Class<T> entityClass) {
     }
 
     /**
-     * Loads an object by ID.
+     * Loads an object by ID.  The reason we support both load by ID and load by filter is that
+     * some legacy stores are optimized to load by ID.
      *
-     * @param entityClass the type of class to load
+     * @param entityProjection the collection to load.
      * @param id - the ID of the object to load.
-     * @param filterExpression - security filters that can be evaluated in the data store.
      * @param scope - the current request scope
      * It is optional for the data store to attempt evaluation.
      * @return the loaded object if it exists AND any provided security filters pass.
      */
-    default Object loadObject(Class<?> entityClass,
-                      Serializable id,
-                      Optional<FilterExpression> filterExpression,
-                      RequestScope scope) {
+    default Object loadObject(EntityProjection entityProjection,
+                              Serializable id,
+                              RequestScope scope) {
+        Class<?> entityClass = entityProjection.getType();
+        FilterExpression filterExpression = entityProjection.getFilterExpression();
+
         EntityDictionary dictionary = scope.getDictionary();
         Class idType = dictionary.getIdType(entityClass);
         String idField = dictionary.getIdFieldName(entityClass);
         FilterExpression idFilter = new InPredicate(
                 new Path.PathElement(entityClass, idType, idField),
                 id
         );
-        FilterExpression joinedFilterExpression = filterExpression
-                .map(fe -> (FilterExpression) new AndFilterExpression(idFilter, fe))
-                .orElse(idFilter);
-        Iterable<Object> results = loadObjects(entityClass,
-                Optional.of(joinedFilterExpression),
-                Optional.empty(),
-                Optional.empty(),
+        FilterExpression joinedFilterExpression = (filterExpression != null)
+                ? new AndFilterExpression(idFilter, filterExpression)
+                : idFilter;
+
+        Iterable<Object> results = loadObjects(entityProjection.copyOf()
+                .filterExpression(joinedFilterExpression)
+                .build(),
                 scope);
+
         Iterator<Object> it = results == null ? null : results.iterator();
         if (it != null && it.hasNext()) {
             return it.next();
@@ -150,45 +154,31 @@ default Object loadObject(Class<?> entityClass,
     /**
      * Loads a collection of objects.
      *
-     * @param entityClass - the class to load
-     * @param filterExpression - filters that can be evaluated in the data store.
-     * It is optional for the data store to attempt evaluation.
-     * @param sorting - sorting which can be pushed down to the data store.
-     * @param pagination - pagination which can be pushed down to the data store.
+     * @param entityProjection - the class to load
      * @param scope - contains request level metadata.
      * @return a collection of the loaded objects
      */
     Iterable<Object> loadObjects(
-            Class<?> entityClass,
-            Optional<FilterExpression> filterExpression,
-            Optional<Sorting> sorting,
-            Optional<Pagination> pagination,
+            EntityProjection entityProjection,
             RequestScope scope);
 
     /**
      * Retrieve a relation from an object.
      *
      * @param relationTx - The datastore that governs objects of the relationhip's type.
      * @param entity - The object which owns the relationship.
-     * @param relationName - name of the relationship.
-     * @param filterExpression - filtering which can be pushed down to the data store.
-     * It is optional for the data store to attempt evaluation.
-     * @param sorting - sorting which can be pushed down to the data store.
-     * @param pagination - pagination which can be pushed down to the data store.
+     * @param relationship - the relationship to fetch.
      * @param scope - contains request level metadata.
      * @return the object in the relation
      */
     default Object getRelation(
             DataStoreTransaction relationTx,
             Object entity,
-            String relationName,
-            Optional<FilterExpression> filterExpression,
-            Optional<Sorting> sorting,
-            Optional<Pagination> pagination,
+            Relationship relationship,
             RequestScope scope) {
-        return PersistentResource.getValue(entity, relationName, scope);
-    }
 
+        return PersistentResource.getValue(entity, relationship.getName(), scope);
+    }
 
     /**
      * Elide core will update the in memory representation of the objects to the requested state.
@@ -230,14 +220,14 @@ default void updateToOneRelation(DataStoreTransaction relationTx,
      * Get an attribute from an object.
      *
      * @param entity - The object which owns the attribute.
-     * @param attributeName - name of the attribute.
+     * @param attribute - The attribute to fetch
      * @param scope - contains request level metadata.
      * @return the value of the attribute
      */
     default Object getAttribute(Object entity,
-                                String attributeName,
+                                Attribute attribute,
                                 RequestScope scope) {
-        return PersistentResource.getValue(entity, attributeName, scope);
+        return PersistentResource.getValue(entity, attribute.getName(), scope);
 
     }
 
@@ -248,13 +238,11 @@ default Object getAttribute(Object entity,
      * This function allow a data store to optionally persist the attribute if needed.
      *
      * @param entity - The object which owns the attribute.
-     * @param attributeName - name of the attribute.
-     * @param attributeValue - the desired attribute value.
+     * @param attribute - the attribute to set.
      * @param scope - contains request level metadata.
      */
     default void setAttribute(Object entity,
-                              String attributeName,
-                              Object attributeValue,
+                              Attribute attribute,
                               RequestScope scope) {
     }
 
@@ -270,7 +258,7 @@ default FeatureSupport supportsFiltering(Class<?> entityClass, FilterExpression
 
     /**
      * Whether or not the transaction can sort the provided class.
-     * @param entityClass
+     * @param entityClass The entity class that is being sorted.
      * @return true if sorting is possible
      */
     default boolean supportsSorting(Class<?> entityClass, Sorting sorting) {
@@ -279,7 +267,7 @@ default boolean supportsSorting(Class<?> entityClass, Sorting sorting) {
 
     /**
      * Whether or not the transaction can paginate the provided class.
-     * @param entityClass
+     * @param entityClass The entity class that is being paged.
      * @return true if pagination is possible
      */
     default boolean supportsPagination(Class<?> entityClass) {