fix: Shard MeasurementsByContinuationToken index to avoid hotspotting

[SanjayVas]

From the Cloud Spanner documentation on secondary indexes:

Be aware that using the commit timestamp column as the first part of the secondary index can create hotspots and reduce write performance.

Sharding is one of the recommendations given for how to avoid hotspotting.

[wfa-reviewable]

This change is 

[renjiezh]

Reviewed 3 of 3 files at r1, all commit messages.
Reviewable status: all files reviewed, 1 unresolved discussion (waiting on @SanjayVas)

---

src/main/resources/kingdom/spanner/shard-measurements-by-continuation-token.sql line 36 at r1 (raw file):

);

RUN BATCH;

nit: empty line

[kungfucraig]

Reviewed 2 of 3 files at r1, all commit messages.
Reviewable status: all files reviewed, 2 unresolved discussions (waiting on @SanjayVas)

---

src/main/resources/kingdom/spanner/shard-measurements-by-continuation-token.sql line 27 at r1 (raw file):

) STORED;

DROP INDEX MeasurementsByContinuationToken;

This is not going to roll out well.

[SanjayVas]

Reviewable status: 2 of 3 files reviewed, 1 unresolved discussion (waiting on @kungfucraig and @renjiezh)

---

src/main/resources/kingdom/spanner/shard-measurements-by-continuation-token.sql line 27 at r1 (raw file):

Previously, kungfucraig (Craig Wright) wrote…

This is not going to roll out well.

True, there will be a period of time where there is no index. I thought this was acceptable in the current state of the application.

1. We need the original index to be dropped to avoid the write hotspotting
2. There's no way to rename an index, only drop and recreate. To get zero downtime without an index and get the original index name, it would mean two backfills.

[kungfucraig]

Reviewable status: 2 of 3 files reviewed, all discussions resolved (waiting on @renjiezh)

---

src/main/resources/kingdom/spanner/shard-measurements-by-continuation-token.sql line 27 at r1 (raw file):

Previously, SanjayVas (Sanjay Vasandani) wrote…

True, there will be a period of time where there is no index. I thought this was acceptable in the current state of the application.

1. We need the original index to be dropped to avoid the write hotspotting
2. There's no way to rename an index, only drop and recreate. To get zero downtime without an index and get the original index name, it would mean two backfills.

In a perfect world this change would be rolled out across three pushes, but that's not the world we live in.

So let's just flag this in the release notes.

[stevenwarejones]

Reviewed 2 of 3 files at r1, 1 of 1 files at r2, all commit messages.
Reviewable status: complete! all files reviewed, all discussions resolved (waiting on @renjiezh)