backupccl: revision_history full backups can fail when attempting to merge spans beyond table GC TTL #66797
Labels
A-disaster-recovery
C-bug
Code not up to spec/doc, specs & docs deemed correct. Solution expected to change code/behavior.
T-disaster-recovery
Describe the problem
During backup planning, backup attempts to find index spans that can be merged to reduce the number of individual spans that needed their timestamp protected. One of the checks performed is a KV scan between non-contiguous public indexes to check that there is no data in the dropped index. We check this for every version of the schema for backups with revision_history.
When considering a version of the schema that is older than the GC TTL of the table, this scan is illegal. This raises an error in backup. As an example, consider the following scenario where
x
represents the existence of a public index at that time:Backup will first look at the schema at t3, see only 1 index (t@1) and continue. Then it will look at t1 and sees 2 new indexes that it can potentially merge, t@2 and t@4. It scans at t@3 at timestamp t1 which will fail the backup since the scan is before the gc threshold.
One possible solution to this problem would be to treat reads that return the "exceed gc threshold" error as *non-*empty (and we don't merge).
The text was updated successfully, but these errors were encountered: