deprecate v1 format version

[RaduBerinde]

So far we have kept compatibility all the way back to RocksDB and the first versions. However, this comes with a burden - there is a lot of complicated code that is no longer necessary with newer formats (and which is undertested as a consequence). In particular, newer formats no longer have the split key issue, where different instances of the same user key can be split into adjacent SSTs.

In order to simplify our code and improve quality for the newer versions, we should stop supporting these old versions. Note that Cockroach 23.1 guarantees no split keys, even on clusters upgraded from older versions.

We should come up with a plan on how to do this so that other users of Pebble can migrate to the newer versions. I propose the following:

• create a v1.0.0 release tag with the crl-release-23.2 commit that goes out with 23.2. We can also release any updates in the 23.2 series as 1.0.x, which will be the most up-to-date version which still supports v1. In the short term, any users who need compatibility can just use this version.
• deprecate the old versions on master, clean up code
• work on a command line tool that can be used to upgrade an existing datastore.
• going forward, we can tag releases from the next series with 2.x.y, until we bump up the minimum supported format again (or modify the API in an incompatible manner).

Jira issue: PEBBLE-83

[RaduBerinde]

CC @jbowens @sumeerbhola Please add your feedback.

[jbowens]

Linking to some related issues #1813, #1913, #1887, #1064, #2587.

I started to document format major versions and the upgrade path in #2162, but I haven't touched it in a year 😬

[RaduBerinde]

Thanks, this is very helpful. Regarding #1064, I think it would be reasonable to "stamp" the version that will go out with 23.2 as 1.0.0 and subsequent 23.2 changes as 1.0.1 etc. Any subsequent version we stamp from the 24.1 series will be 2.0.0 as it will not support the v1 format anymore. Updated the issue description.

[jbowens]

There’s also this release-engineering ticket where we were hoping to tie pebble tags to corresponding cockroach tags: https://cockroachlabs.atlassian.net/browse/RE-430

That might be a more convenient (for us) versioning scheme.

[RaduBerinde]

The problem is that other users of the repo would expect semantic versioning, not sure how much the 2X.Y.Z scheme is of use to them. Note that the cockroach tags can be something like cockroach-2X.Y.Z and live alongside the semantic versions like 1.0.0.

[RaduBerinde]

If we deprecate all format major versions smaller than FormatFlushableIngest, it looks like we will also deprecate TableFormatLevelDB and TableFormatRocksDBv2

There is this code that is a little worrisome because it defaults the sstable writer to TableFormatRocksDBv2: https://github.com/cockroachdb/pebble/blob/master/sstable/options.go#L299
AFAICT the Cockroach code that calls NewWriter sets the format and recent releases would never set TableFormatRocksDBv2, and internal code also sets the table format.

Can anyone confirm that indeed 23.1+ will never have tables with format below TableFormatPebblev1?