Updating cached data from multiple parameterized queries after a mutation

[steve-a-jones]

Appears that the 2 related issues apollographql/react-apollo#708 and #1697 were closed without an efficient solution - so re-opening here as I am having the same issue.

The crux of the problem is this:

1. I have a mutation that creates/deletes nodes.
2. I have multiple co-located, parameterized queries that need to re-run when a create/delete mutation happens.
3. Updating the store manually inside update is non-trivial given that all co-located and parameterized queries that were affected by the create/delete mutation need to be re-run.
4. The variables applied to each parameterized query are applied dynamically and are not easily discoverable or in scope at the update call site.

Interested to hear thoughts from the apollo team on an update pattern that fits here.

Thanks!

[steve-a-jones]

It appears that refetchQueries already provides this functionality...

From the docs...
https://www.apollographql.com/docs/react/basics/mutations.html#graphql-mutation-options-refetchQueries

If options.refetchQueries is an array of strings then Apollo Client will look for any queries with the same names as the provided strings and will refetch those queries with their current variables.

This actually solves my current problem as I simply needed a way to re-run queries by name without needing to know what arguments were applied to the query, as they may differ throughout the application.

If I have a query
allJobsByOwner($ownerId:ID!)

every time the query is run apollo will create a new key under ROOT_QUERY..

ROOT_QUERY:
    allJobsByOwner(ownerId:1)
    allJobsByOwner(ownerId:2)
    allJobsByOwner(ownerId:3)
etc..

I was under the impression that the query could not be re-fetched without re-applying the arguments back to query in refetchQuery. But if you supply strings to refetchQueries apollo will look them up by name and re-run them with the last variables applied.

For those curious please see:

apollo-client/src/core/QueryManager.ts

Lines 1305 to 1322 in 3b5045d

	// Refetches a query given that query's name. Refetches
	// all ObservableQuery instances associated with the query name.
	private refetchQueryByName(queryName: string) {
	const refetchedQueries = this.queryIdsByName[queryName];
	// Warn if the query named does not exist (misnamed, or merely not yet fetched)
	if (refetchedQueries === undefined) {
	console.warn(
	`Warning: unknown query with name ${queryName} asked to refetch`,
	);
	return;
	} else {
	return Promise.all(
	refetchedQueries.map(queryId =>
	this.observableQueries[queryId].observableQuery.refetch(),
	),
	);
	}
	}

and

apollo-client/src/core/ObservableQuery.ts

Line 207 in 3b5045d

this.variables = {

[juanpaco]

Confirmed that this worked. Might be worth highlighting that the query name comes from the operation name. I missed that in steve-a-jones's example, and it tripped me up for a few minutes. So:

query thisIsWhatYouPassToRefetchQueries($ownerId: ID!) {
  doNotPassThisToRefetchQueries(ownerId: $ownerId) {
    ...
  }
}

[snaumov]

The solution with refetchQueries works, however, what if we are not to poll the data from a server, but instead write it to cache straight away. It can be done with readQuery and writeQuery but it's tricky since you'll need to provide the exact same variables to the query as in a cache.

Is there any way round it?

[minardimedia]

Also what happen if you and to update/refetch in ALL the variables not just the last variables applied

[alexdilley]

A couple of peeps at Xero came up with a middleware-esque pattern to tackle this sort of concern: apollo-link-watched-mutation.

[SachaG]

Could we reopen this? I feel like the use case @snaumov mentions ("what if we are not to poll the data from a server, but instead write it to cache straight away") is still unsolved. I would be curious to hear from the Apollo team on this and know if they suggest using the Watched-Mutation pacakage or if there's another way?

[snaumov]

@SachaG I was able to solve the mentioned problem of not knowing which arguments to use for a cached query here:
https://www.apollographql.com/docs/react/advanced/caching.html#connection-directive
so, essentially, if the query has @connection directive in it, in the cache it'll be stored without extra arguments.

[SachaG]

Good to know, thanks! In the end I decided to go with watched-mutation because my update functions are not really aware of my queries (they're handled by different components).

[kepi0809]

The solution with refetchQueries works, however, what if we are not to poll the data from a server, but instead write it to cache straight away. It can be done with readQuery and writeQuery but it's tricky since you'll need to provide the exact same variables to the query as in a cache.

Is there any way round it?

this was what came to my mind too, but I can't believe that this is the only way… like storing all the previous filters and updating them accordingly doesn't sound like an optimal solution to me, too many cases to handle. In my case there was a status change of an item from pending to failed, which in UI should be updated by filter with 3 statuses (in pending should be removed, failed: should be added, any: status should be updated)
In this example there are only 3 different cases to handle which should be okay, but it can escalate really quickly. And even handling it with update doesn't sound perfect, because of pagination, in failed there will be one more element which could cause issues and in pending will be one fewer which also could lead to some issues.

Should I just go with refetchQueries and that's it or do you have any ideas for this case?

[ayubov]

@SachaG I was able to solve the mentioned problem of not knowing which arguments to use for a cached query here:
https://www.apollographql.com/docs/react/advanced/caching.html#connection-directive
so, essentially, if the query has @connection directive in it, in the cache it'll be stored without extra arguments.

@snaumov Thanks a lot for that point! Worked for me.

[raysuelzer]

Perhaps this will be helpful. In my case, I have a typeahead which searches the backend and takes about 5-6 additional variables in addition to the keyword. I am not using watch queries (although this method also works with them). I needed a way to evict a whole bunch of queries from the cache based upon only a subset of the variables.

For example I have the following queries which are cached (grossly simplified).

searchTags("keyword:" null, "categoryId":"10")
searchTags("keyword": "fo", "categoryId":"10")
searchTags("keyword": "foo", "categoryId":"10")
searchTags("keyword:" null, "categoryId":"11")
searchTags("keyword:" "bar", "categoryId":"12")

At some point the user adds a new tag to categoryId of 10. We need to invalidate any query that has the categoryId of 10 as an arg. We do not want to clear the cache for any other queries, there may be a ton.

I created a helper function (for Apollo Client 3) to accomplish this, as well as one for Apollo 2 that uses internal APIs. This function is passed to the update option on an Apollo Mutation. I am using Angular Apollo, so this might look a bit different than what you are used to seeing.

Also, please note, I removed some additional code here for simplicity and this might break when passed invalid parameters, etcs.

Usage:

this.createStandardTagGQL.mutate(
                    { input, isInCampaignId },
                    {
                        update: (cache: ApolloCache<any>, data) => {
                            const categoryId = data.createStandardTag.tag.categoryId;
                            // remove items from cache which include this string
                            invalidateCacheFor(
                                cache, 
                                 (key) => key.includes(`"categoryId":"${categoryId}`)
                            );
                        }
                    }
                )

invalidateApolloCacheFor could be further improved by having the predicate test the fieldName and args instead of the key string. But, this is what I have for now. You should be able to adapt it to your needs.

/**
 * Helper function evicts items in the ROOT_QUERY of the
 * cache where the key matches the predicate passed
 *
 * @param cache Apollo Cache Client (i.e. In memory cache)
 * @param keyTest: Prediciate to test key name
 */
export const invalidateApolloCacheFor = (
    cache: ApolloCache<any>,
    keyTest: (string) => boolean) => {

    // Extracts all keys on the ROOT_QUERY
    const rootQueryKeys = Object.keys(cache.extract().ROOT_QUERY);

    // Filters to keys matching the test
    // Then converts the keys to an object     
    // with the fieldName and the arguments as JSON
    // {fieldName, args}
    const itemsToEvict = rootQueryKeys
        .filter(keyTest)
        .map(key => extractFieldNameAndArgs(key));

    itemsToEvict.forEach(({ fieldName, args }) => {
        cache.evict(
            {
                id: 'ROOT_QUERY',
                fieldName,
                args
            }
        );
    });
};

/**
 * Extracts the field name and the keyArgs
 * from an Apollo cache key.
 * eg: searchTags:{"keyword":"F"} =>
 * returns { fieldName: 'searchTags', args: { keyword: "F" }  }
 *
 * @param key key in the apollo cache.
 */
export const extractFieldNameAndArgs = (key: string) => {
    if (!key.includes(':')) {
        return { fieldName: key, args: null };
    }
    const seperatorIndex = key.indexOf(':');
    const fieldName = key.slice(0, seperatorIndex);
    const args = convertKeyArgs(key);

    return { fieldName, args };
};

// Convert the keyArgs stored as a string in the query key
// to a json object to be used for the args when evicting from the cache
const convertKeyArgs = (key: string): Record<string, any> => {
    const seperatorIndex = key.indexOf(':');
    const keyArgs = key.slice(seperatorIndex + 1);

    // @connection directives wrap the keyArgs in ()
    // TODO: Remove when legacy @connection directives are removed
    const isLegacyArgs = keyArgs?.startsWith('(') && keyArgs.endsWith(')');

    const toParse = isLegacyArgs ? keyArgs.slice(1, keyArgs.length - 1) : keyArgs;

    // We should have a string here that can be parsed to JSON, or null
    // getSafe is an internal helper function that wraps a try catch
    const args = getSafe(() => JSON.parse(toParse), null);
    return args;
};

If you are on Apollo2, you don't have cache.evict but you do something similar like this. Not this is not like RefetchQueries where is uses the client query name, this looks at the actual cache key from apollo which will be the query name:

 * This helper function removes any cached data where
 * the cache query key cointains the given string.
 * For example,
 * removeFromCache(cache, "searchPeople")
 * will remove any queries which match 'searchPeople' from the cache.
 *
 * A more detailed approach is here:
 * https://medium.com/@martinseanhunt/how-to-invalidate-cached-data-in-apollo-and-handle-updating-paginated-queries-379e4b9e4698
 *
 * @param cacheProxy Apollo Cache Client (i.e. In memory cache)
 * @param keyTest: String or Prediciate of keys to check
 */
export function invalidateCacheForLegacy(cacheProxy: any, keyTest: ((string) => boolean) | string) {  
    const dataProxy = cacheProxy.data;
    const filterFunction: ((string) => boolean) = _isString(keyTest) ?
        (k: string) => k.includes(keyTest as string)
        : keyTest as ((string) => boolean);

    const matchedKeys = Object.keys(dataProxy.data)
        .filter(filterFunction);

    matchedKeys.forEach(key => {
        dataProxy.delete(key);
    });
}

[mindnektar]

I've created a package in an attempt to solve these issues and make complex cache updates as painless as possible. https://www.npmjs.com/package/apollo-augmented-hooks

See https://github.com/appmotion/apollo-augmented-hooks/#--modifiers and the linked caching guide in particular.

I've been using this in production for a while now and just made it open source so other people can benefit from it as well.