Datastore - Selective Sync

[nithin-seenivasan]

Would be great to have selective sync for Flutter, similar to what exists in Amplify JS. https://docs.amplify.aws/lib/datastore/sync/q/platform/js#selectively-syncing-a-subset-of-your-data

I would ideally like to have a proper control over the syncing process. Ideas could include:

1. Configure a time interval for a sync to happen
2. The client can trigger a sync operation
   3. The sync can be on listening to any changes to the datasource like DynamoDB (resolved using Appsync)

Related to issue #297
Also discussed on #5

[nithin-seenivasan]

Updated the post above to reflect current status.

[apolo92]

Is there any forecast of when this new functionality will be available?
If we don't have this functionality, is there any way to control the synchronization? By default I understand that it will synchronize the entire cloud DB locally. Does this mean that user A will have user B data on his local device?

[cto-leaps]

Any idea when selective sync will be available?

[seyaobey]

This is indeed a much needed feature. Amplify is an amazing product. But I have switched to another platform for my real-time database since for privacy reasons, I cannot download the whole database into every device.

[apolo92]

This is indeed a much needed feature. Amplify is an amazing product. But I have switched to another platform for my real-time database since for privacy reasons, I cannot download the whole database into every device.

What platform have you used to replace datastore? firebase?

[nithin-seenivasan]

Is there any forecast of when this new functionality will be available?
If we don't have this functionality, is there any way to control the synchronization? By default I understand that it will synchronize the entire cloud DB locally. Does this mean that user A will have user B data on his local device?

Not really, Datastore uses Appsync, and user A will only get the data synced that they are 'subscribed' to. So even though the underlying DynamoDB table is common, the userdata will be separate with no possibility of data leaking between accounts.

[kjones]

Not really, Datastore uses Appsync, and user A will only get the data synced that they are 'subscribed' to. So even though the underlying DynamoDB table is common, the userdata will be separate with no possibility of data leaking between accounts.

This isn't true by default. You have to explicitly add authorization rules to your GraphQL schema to control what gets synced. The default is everything in the table.

[nithin-seenivasan]

Not really, Datastore uses Appsync, and user A will only get the data synced that they are 'subscribed' to. So even though the underlying DynamoDB table is common, the userdata will be separate with no possibility of data leaking between accounts.

This isn't true by default. You have to explicitly add authorization rules to your GraphQL schema to control what gets synced. The default is everything in the table.

You are right, @kjones , I forgot that one had to explicitly control what gets synced in the schema. That should solve @apolo92 's issue.

[kjones]

I'm more interested in selective sync from a performance and cost perspective. Say the table has millions of records and I have auth rules set up. I don't see how AppSync can optimize this query given that all items in the table are only unique by ID. It must do a DynamoDB scan operation on this table and then filter the results by owner. At most it can add a filter expression to the query which still scans the entire table but returns a result set with only the owner's records. With selective sync I should be able to create a GSI on the table and then use a query expression that turns the table scan into an optimized query for the user's data.

[apolo92]

Not really, Datastore uses Appsync, and user A will only get the data synced that they are 'subscribed' to. So even though the underlying DynamoDB table is common, the userdata will be separate with no possibility of data leaking between accounts.

This isn't true by default. You have to explicitly add authorization rules to your GraphQL schema to control what gets synced. The default is everything in the table.

You are right, @kjones , I forgot that one had to explicitly control what gets synced in the schema. That should solve @apolo92 's issue.

Thanks @nithin-seenivasan and @kjones I use @owner permissions in appsync and only sync data user. But during testing I have seen that the Amplify.Datastore.clear () does not work on ios. This command do not nothing :(. Do you have any issue on this topic?

[kjones]

DataStore.clear() does not work on ios

If this means it doesn't actually clear the local data then I'm of no help.

I have noticed inconsistencies between iOS and Android after the clear() call.

• DataStore does not have a start()/stop() API like the underlying Android and iOS SDKs so you have to do a throw-away query() to get DataStore() going again.
• Android seems to restart DataStore immediately without the throw-away query to get it going again. This may not still be true and I may be thinking of earlier days pre-2021.
• On iOS, observe() stops working after a clear(). There is a fix for this but it hasn't been released yet.

[rajd90]

i really need this feature as well. selective sync is critical for performance & cost reasons, as we really need to avoid doing scans and use query (as indicated in the many dynamodb best practice examples). is it already possible and just not documented?

[rajd90]

is there any update / timeline on this issue?

[cshfang]

is there any update / timeline on this issue?

Thanks all for your valued feedback on this feature. I am unable to provide anything close to concrete dates at this time but it is something we are investigating.

[cshfang]

As of v0.2.1, Amplify Flutter should now support selective sync. More information about this can be found in the Amplify Docs here.