Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Paginated Watchers for GraphQL Queries #3007

Closed
wants to merge 72 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
72 commits
Select commit Hold shift + click to select a range
8b76cc7
Simple paginated watcher implementation
Iron-Ham May 11, 2023
369f760
include tests
Iron-Ham May 11, 2023
0ab2630
Get rid of an unnecessary completion block
Iron-Ham May 11, 2023
5ec8f5d
Remove debugging code
Iron-Ham May 11, 2023
b530005
Manage pages
Iron-Ham May 11, 2023
4cb2869
Rename fetchNext to fetchMore for consistency with React
Iron-Ham May 11, 2023
933e086
Failing test
Iron-Ham May 11, 2023
512a4c2
Succeeding test
Iron-Ham May 11, 2023
9ae40df
Account for source for merging server vs cache update
Iron-Ham May 12, 2023
79f7aac
Cleanup test file
Iron-Ham May 12, 2023
f15d774
Simplify transform
Iron-Ham May 12, 2023
9e39373
Wrap with a model
Iron-Ham May 12, 2023
d793b56
Test for deletion of cache data
Iron-Ham May 12, 2023
d270a62
Cleanup Pagination + Tests
Iron-Ham May 15, 2023
d8921ac
Add support for fetching individual pages
Iron-Ham May 15, 2023
2d0ca4f
Start transition towards merge strategies
Iron-Ham May 16, 2023
74f21ff
Transition to strategies
Iron-Ham May 16, 2023
9bdaab4
Make sure we only notify observers once
Iron-Ham May 16, 2023
a149d51
Spacing cleanup
Iron-Ham May 16, 2023
c15366d
More tests
Iron-Ham May 16, 2023
9ba3544
Cleanup docs
Iron-Ham May 17, 2023
26e9de8
Explicit self
Iron-Ham May 17, 2023
fbe5baa
Typo fix
Iron-Ham May 17, 2023
506e444
Add initializer to PaginationDataResponse
Iron-Ham May 17, 2023
5357668
Update comments
Iron-Ham May 17, 2023
593fe8c
Update comments, again
Iron-Ham May 17, 2023
5a5640e
Public values
Iron-Ham May 18, 2023
0576f30
Add completion block to fetchMore
Iron-Ham May 18, 2023
e7f7675
Source should be included as part of the result handler
Iron-Ham May 18, 2023
d5515d2
Add a type erased GraphQLPaginatedQueryWatcher
Iron-Ham May 18, 2023
a47ce7d
AnyPaginationStrategy
Iron-Ham May 22, 2023
a525e59
Add protocol for paginated query watchers
Iron-Ham May 22, 2023
ba3bb4d
Check for cocoapods
Iron-Ham May 22, 2023
f8bf35c
Add comments
Iron-Ham May 22, 2023
5f219d4
More docs
Iron-Ham May 23, 2023
aa8c94d
Deleted AnyGraphQLPaginatedQueryWatcher
Iron-Ham May 24, 2023
bfeee53
RelayPaginationStrategy
Iron-Ham May 24, 2023
7f6575f
Pagination Merge strategies
Iron-Ham May 24, 2023
ee01b24
Start fixing tests
Iron-Ham May 24, 2023
586e999
Refetch
Iron-Ham May 24, 2023
067eebf
With keypaths
Iron-Ham May 24, 2023
b7cf1f6
Refetch
Iron-Ham May 24, 2023
d9921f2
Fetch + Local Cache Update
Iron-Ham May 24, 2023
d5a73c8
Custom Strategy tests
Iron-Ham May 24, 2023
b0baeb9
Simple multipage
Iron-Ham May 24, 2023
0d08a4e
Targeted
Iron-Ham May 24, 2023
850b109
All Green
Iron-Ham May 24, 2023
4372bb3
Some docs
Iron-Ham May 25, 2023
cc61d00
More docs
Iron-Ham May 25, 2023
fdacf41
Merge branch 'main' into hs/paginated-watcher
Iron-Ham May 25, 2023
d7659d4
Update with main
Iron-Ham May 25, 2023
82cb86c
More docs
Iron-Ham May 25, 2023
975bce8
Spacing
Iron-Ham May 25, 2023
587664c
Prefer private extension
Iron-Ham May 25, 2023
f3c7c91
Green again
Iron-Ham May 25, 2023
92b17b6
Union of fragments
Iron-Ham May 25, 2023
c76914d
public _resultHandler
Iron-Ham May 25, 2023
1492973
Make KeyPaths optional
Iron-Ham May 26, 2023
f773878
Allow the relay pagination strategy to use the protocol
Iron-Ham Jun 5, 2023
284126b
Update resultHandler
Iron-Ham Jun 8, 2023
7fa6f54
Rough draft of docs for pagination
Iron-Ham Jun 12, 2023
ff84911
Merge branch 'main' of github.com:Iron-Ham/apollo-ios into hs/paginat…
Iron-Ham Jul 5, 2023
badeddb
Update schema
Iron-Ham Jul 6, 2023
014f1d5
Tests
Iron-Ham Jul 6, 2023
c8206ce
Cleanup
Iron-Ham Jul 6, 2023
c881000
Xcodeproj
Iron-Ham Jul 6, 2023
bf51e47
Docs
Iron-Ham Jul 6, 2023
3554f38
Update website docs
Iron-Ham Jul 6, 2023
f97ba72
Fix test
Iron-Ham Jul 6, 2023
4440a5c
empty commit
Iron-Ham Jul 12, 2023
015b02d
empty commit2
Iron-Ham Jul 12, 2023
8c13b61
Import Dispatch
Iron-Ham Jul 12, 2023
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
212 changes: 184 additions & 28 deletions Apollo.xcodeproj/project.pbxproj

Large diffs are not rendered by default.

23 changes: 23 additions & 0 deletions Sources/Apollo/CustomDataTransformer.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// Gives the caller full control over how to transform data.
/// Can be used to output a custom type, or can be used for fine grain control over how a `Query.Data` is merged.
public struct CustomDataTransformer<Query: GraphQLQuery, Output: Hashable>: DataTransformer {

private let _transform: (Query.Data) -> Output?

/// Designated intializer
/// - Parameter transform: A user provided function which can transform a given network response into any `Hashable` output.
public init(transform: @escaping (Query.Data) -> Output?) {
self._transform = transform
}

/// The function by which we transform a `Query.Data` into an output
/// - Parameter data: Network response
/// - Returns: `Output`
public func transform(data: Query.Data) -> Output? {
_transform(data)
}
}
16 changes: 16 additions & 0 deletions Sources/Apollo/CustomNextPageStrategy.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// The strategy by which we create the next query in a series of paginated queries. Gives full custom control over mapping a `Page` into a `Query`.
public struct CustomNextPageStrategy<Page: Hashable, Query: GraphQLQuery>: NextPageStrategy {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure I see the value in including this at all. I think this goes for the CustomDataTransformer and CustomPaginationMergeStrategy as well. If people need to implement a custom implementation of any of these, you should just define your own types that conforms to the protocols.

I guess I see why it is ergonomically valuable to be able to use these to essentially turn the initialization of PaginationStrategy into closure based parameters (as is shown in the unit tests for this), so I can be persuaded here. Let's discuss this more with the team.


If it makes sense, what I would like to include is an implementation of a NextPageStrategy that does this for you given just a query type specifically for the cursor-based and offset + count pagination methods?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We discussed a bit on our call.

I think we could probably collapse both CustomNextPageStrategy and the need for an initialQuery into a new QueryProvider protocol that takes in a Page? as an argument (as opposed to the Page that we take in now).

We could default to generating the initial query if the Page is nil.

private let _transform: (Page) -> Query

public init(transform: @escaping (Page) -> Query) {
self._transform = transform
}

public func createNextPageQuery(page: Page) -> Query {
_transform(page)
}
}
22 changes: 22 additions & 0 deletions Sources/Apollo/CustomPaginationMergeStrategy.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,22 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// A `PaginationMergeStrategy` which gives the caller fine-grain control over how to merge data together.
public class CustomPaginationMergeStrategy<Query: GraphQLQuery, Output: Hashable>: PaginationMergeStrategy {

let _transform: (PaginationDataResponse<Query, Output>) -> Output

/// Designated initializer
/// - Parameter transform: a user-defined function which can transform a `PaginationDataResponse` into an `Output`.
public init(transform: @escaping (PaginationDataResponse<Query, Output>) -> Output) {
self._transform = transform
}

/// The function by which we merge several responses, in the form of a `PaginationDataResponse` into one `Output`.
/// - Parameter paginationResponse: A data type which contains the most recent response, the source of that response, and all other responses.
/// - Returns: `Output`
public func mergePageResults(paginationResponse: PaginationDataResponse<Query, Output>) -> Output {
_transform(paginationResponse)
}
}
14 changes: 14 additions & 0 deletions Sources/Apollo/DataTransformer.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// The protocol by which we transform a network/cache response into some `Output`.
public protocol DataTransformer {
associatedtype Query: GraphQLQuery
associatedtype Output: Hashable

/// Given a network response, transform it into the intended result type of the `PaginationStrategy`.
/// - Parameter data: A network response
/// - Returns: `Output`
func transform(data: Query.Data) -> Output?
}
102 changes: 102 additions & 0 deletions Sources/Apollo/GraphQLPaginatedQueryWatcher.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,102 @@
#if !COCOAPODS
import ApolloAPI
#endif
import Dispatch

/// Handles pagination in the queue by managing multiple query watchers.
public final class GraphQLPaginatedQueryWatcher<Strategy: PaginationStrategy> {
public typealias Page = Strategy.NextPageConstructor.Page
private typealias ResultHandler = (Result<GraphQLResult<Strategy.Query.Data>, Error>) -> Void

private let client: any ApolloClientProtocol
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

GraphQLQueryWatcher has always held a weak reference to the client. I'm wondering if this should do the same. I don't imagine that this would create a retain cycle, as the retain cycle would be broken as soon as you call cancel().

Any thoughts on this? @Iron-Ham @calvincestari @BobaFetters

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think this sounds reasonable

private var watchers: [GraphQLQueryWatcher<Strategy.Query>] = []
private var callbackQueue: DispatchQueue
let strategy: Strategy

/// Designated initalizer
/// - Parameters:
/// - client: the Apollo client
/// - callbackQueue: The `DispatchQueue` that results are returned on.
/// - strategy: The `PaginationStrategy` that this watcher employs.
/// - initialQuery: The `Query` that is being watched.
public init(
client: ApolloClientProtocol,
callbackQueue: DispatchQueue = .main,
strategy: Strategy,
initialQuery: Strategy.Query
) {
self.callbackQueue = callbackQueue
self.client = client
self.strategy = strategy
let initialWatcher = GraphQLQueryWatcher(
client: client,
query: initialQuery,
callbackQueue: callbackQueue,
resultHandler: strategy.onWatchResult(result:)
)
watchers = [initialWatcher]
}

/// Fetch the first page
/// NOTE: Does not refresh subsequent pages nor remove them from the return value.
public func fetch(cachePolicy: CachePolicy = .returnCacheDataAndFetch) {
watchers.first?.fetch(cachePolicy: cachePolicy)
}

/// Fetches the first page and purges all data from subsequent pages.
public func refetch(cachePolicy: CachePolicy = .fetchIgnoringCacheData) {
// Reset mapping of data and order of data
strategy.reset()
// Remove and cancel all watchers aside from the first page
guard let initialWatcher = watchers.first else { return }
let subsequentWatchers = watchers.dropFirst()
subsequentWatchers.forEach { $0.cancel() }
watchers = [initialWatcher]
initialWatcher.refetch(cachePolicy: cachePolicy)
}

/// Fetches the next page
@discardableResult public func fetchMore(
cachePolicy: CachePolicy = .fetchIgnoringCacheData,
completion: (() -> Void)? = nil
) -> Bool {
guard strategy.canFetchNextPage(),
let currentPage = strategy.currentPage
else { return false }
let nextPageQuery = strategy.nextPageStrategy.createNextPageQuery(page: currentPage)

let nextPageWatcher = client.watch(
query: nextPageQuery,
cachePolicy: cachePolicy,
callbackQueue: callbackQueue
) { [weak self] result in
self?.strategy.onWatchResult(result: result)
completion?()
}
watchers.append(nextPageWatcher)

return true
}

/// Refetches data for a given page.
/// NOTE: Does not refresh previous or subsequent pages nor remove them from the return value.
public func refresh(page: Page?, cachePolicy: CachePolicy = .returnCacheDataAndFetch) {
guard let page else {
// Fetch first page
return fetch(cachePolicy: cachePolicy)
}
guard let index = strategy.pages.firstIndex(where: { $0 == page }),
watchers.count > index
else { return }
watchers[index].fetch(cachePolicy: cachePolicy)
}

/// Cancel any in progress fetching operations and unsubscribe from the store.
public func cancel() {
watchers.forEach { $0.cancel() }
}

deinit {
cancel()
}
}
12 changes: 12 additions & 0 deletions Sources/Apollo/NextPageStrategy.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// The strategy by which we create the next query in a series of paginated queries.
public protocol NextPageStrategy {
associatedtype Query: GraphQLQuery
associatedtype Page: Hashable

/// Given some `Page`, returns a formed `Query` that uses the information contained within the `Page` to paginate.
func createNextPageQuery(page: Page) -> Query
}
77 changes: 77 additions & 0 deletions Sources/Apollo/OffsetPageExtractor.swift
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
#if !COCOAPODS
import ApolloAPI
#endif

/// Can extract and identify a `Page` for cursor-based endpoints
public struct OffsetPageExtractor<Query: GraphQLQuery>: PageExtractionStrategy {

/// A formed input for the `OffsetPageExtractor`
public struct Input: Hashable {

/// The `Query.Data`
public let data: Query.Data

/// The current offset
public let offset: Int

/// The number of expected results per page
public let pageSize: Int

init(data: Query.Data, offset: Int, pageSize: Int) {
self.data = data
self.offset = offset
self.pageSize = pageSize
}
}

/// A minimal definiton for a `Page`
public struct Page: Hashable {
/// Where in the list the server should start when returning items for a particular query
public let offset: Int

/// Whether or not there is potentially another page of results
public let hasNextPage: Bool

/// Designated Initializer
/// - Parameters:
/// - offset: Where in the list the server should start when returning items for a particular query
/// - hasNextPage: Whether or not there is potentially another page of results
public init(offset: Int, hasNextPage: Bool) {
self.offset = offset
self.hasNextPage = hasNextPage
}
}

private let _transform: (Input) -> Page

/// Designated initializer
/// - Parameter transform: A user provided function which can extract a `Page` from a `Query.Data`
public init(transform: @escaping (Input) -> Page) {
self._transform = transform
}

/// Convenience initializer
/// - Parameter arrayKeyPath: A `KeyPath` over a `Query.Data` which identifies the array key within the `Query.Data`.
public init(arrayKeyPath: KeyPath<Query.Data, [(some SelectionSet)?]?>) {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This doesn't account for 2 things:

  1. Lists of scalar/custom scalars
  2. Nested lists (2D, 3D, etc.)

I'm wondering if it would work if we change this to:

public init(arrayKeyPath: KeyPath<Query.Data, some Collection>) {

self._transform = { input in
let count = input.data[keyPath: arrayKeyPath]?.count ?? 0
return Page(offset: input.offset + count, hasNextPage: count == input.pageSize)
}
}

/// Convenience initializer
/// - Parameter arrayKeyPath: A `KeyPath` over a `Query.Data` which identifies the array key within the `Query.Data`.
public init(arrayKeyPath: KeyPath<Query.Data, [(some SelectionSet)]?>) {
self._transform = { input in
let count = input.data[keyPath: arrayKeyPath]?.count ?? 0
return Page(offset: input.offset + count, hasNextPage: count == input.pageSize)
}
}

/// Transforms the `Query.Data` into a `Page` by utilizing the user-provided functions or key paths.
/// - Parameter input: A query response data.
/// - Returns: A `Page`.
public func transform(input: Input) -> Page {
_transform(input)
}
}
Loading