Securing the sharding data

A developer used the AuthP library to create a multi-tenant application that uses sharding and, by mistake, he deleted the FileStore Cache file (see issue #115 for one way to delete the FileStore Cache file). At that point he lost the sharding information and he couldn't get it back. While the deletion of a FileStore Cache file is very rare, the consequences of loosing the sharding information on an active application would be very bad. Therefore I have added code in AuthP version 8.1.0 that fixes this. This page shows what you need to do to use this new feature.

NOTE: You may ask why I use the FileStore Cache if it has this problem?. Its all about performance. Every access to the tenant's data needs the specific sharding data for that tenant, and typically more that 90% of all the requests needs the sharding data. In terms of speed the FileStore Cache get the sharding in ~25 ns, while using a SQL Server distributed cache would take at least 0.1 ms – that means FileStore cache is >4,000 faster.

Here are the sections in this page:

• Overview of the "Secure your sharding data" feature
• What the CheckTwoShardingSources method does
• Things to do if there are errors

Overview of the "Secure your sharding data" feature

The solution in AuthP version 8.1.0 adds a new database called ShardingEntryBackup (referred to as 'ShardingBackup database') to the AuthP's AuthPermissionsDbContext and extra code is added to the GetSetShardingEntriesFileStoreCache so any change to a sharding information in the the FileStore Cache it will also apply the same change to the ShardingBackup database.

So, if your FileStore Cache file is deleted you need run a new method called CheckTwoShardingSources in the GetSetShardingEntriesFileStoreCache which will copy the existing FileStore Cache's sharding data to the ShardingEntryBackup. You need to run this manually, typically by an Admin user. The next section details what the CheckTwoShardingSources does.

If you want to see or try the CheckTwoShardingSources method then the AuthP's Example6 or Example7 project contain the new code (the code are the same in the two Examples). Most of the code you need to add is in the Controller.ShardingController class.

NOTE: if you are updating a existing multi-tenant applications to AuthP version 8.1.0, then you must run the CheckTwoShardingSources method, to be fully covered if the apps' FileStore Cache file is deleted. That will will copy the your shardings from your existing FileStore Cache into the 'ShardingBackup database.

What the CheckTwoShardingSources method does

When the CheckTwoShardingSources method runs it looks at the two sharding sources, FileStore Cache and the ShardingBackup database, and runs the appropriate stage. The list below describes the four stages do:

1. EMPTY-OK: no sharding data found

The EMPTY-OK stage is run when both sharding sources are empty. This happens when there are no sharding databases (apart of the default database if the HybridMode option is true). For instance, the first deploy of the application there are no tenants, so there wouldn't any sharding entries. This stage doesn't do anything because there is no sharding entries. This returns an success message, starting with "EMPTY-OK".

2. BACKUP-SHARDINGS: The ShardingBackup database needs to be filled

The BACKUP-SHARDINGS stage is run when the FileStore Cache has entries, but the ShardingBackup database is empty. This happens when you have updated to AuthP version 8.1.0 or above and it will copy over the shardings from the FileStore Cache into the ShardingBackup database. Once this stage is had executed you have secured your sharding data and it returns a success message, starting with "BACKUP-SHARDINGS".

NOTE: once your is updated to AuthP version 8.1.0, then any change to a sharding are written to both the FileStore Cache AND the ShardingBackup database.

3. RESTORE-SHARDINGS: The FileStore Cache is restored

The RESTORE-SHARDINGS stage is run when the FileStore Cache is empty, but the ShardingBackup database had entries. This happens when the FileStore Cache file is deleted (this was the problem the developer had), and it copies the data from the ShardingBackup database into the FileStore Cache. It returns a success message, starting with "RESTORE-SHARDINGS" which shows that the FileStore Cache has the correct shardings entries.

4. CHECK-SHARDINGS: checks the sharding data

If the first three stages aren't triggered, then four stage, CHECK-SHARDINGS, is run. This stage checks that the properties in each sharding entry are the same in both sharding sources. If everything was OK, then it returns a success error starting with "CHECK-SHARDINGS". But if there are differences between the FileStore Cache and the ShardingBackup database it will a list of errors - see the next section which has suggestions on how to fix the different errors.

NOTE: It is very rare that the two sharding sources could be different, but it could happen. For instance, if the FileStore Cache update works but the ShardingBackup database update has an error. Therefore I added this check which provides details information on any differences.

Things to do if there are errors

If the "CHECK-SHARDINGS" stage finds errors, then will be either:

1. A sharding entry is missing, e.g. the FileStore Cache has a sharding with the Name of "XXX", but ShardingBackup database hasn't a sharding entry with the Name of "XXX".
2. A sharding entry has different data, e.g. a FileStore Cache has a sharding entry where Name of "XXX" and its ConnectionName value is different from the sharding entry in the ShardingBackup database with the same Name, "XXX.

If you have either a missing or different data errors from the same sharding source, then its simpler. Here are the two versions for the two sharding sources.

All errors in the ShardingBackup database

1. Delete all the entries in the ShardingBackup database.
2. Run CheckTwoShardingSources method again, which run the BACKUP-SHARDINGS stage, which copies the data from the FileStore Cache into the ShardingBackup database.

All errors in the FileStore Cache

1. Delete the FileStore Cache File. See the NOTE below if there are non-sharding entries
2. Run CheckTwoShardingSources method again, which will run the RESTORE-SHARDINGS stage, which copies the data from the the ShardingBackup database into the FileStore Cache File

NOTE: If you have entries that aren't sharding entries in the FileStore Cache, such as the "Down for maintenance" feature its harder. You need to:

1. Manually copy out the non-sharding data in the FileStore Cache File.
2. Delete the FileStore Cache File.
3. Run CheckTwoShardingSources method again.
4. Manually add the non-sharding data back into the FileStore Cache File.

Yes, I know that is difficult, but as I said such a error is very rare.

Additional resources

• Multi tenant explained
• Sharding explained
• How AuthP handles sharding