KAFKA-15537: Fix metadata downgrade documentation (apache#14484)

In KIP-778 we introduced the "unsafe" (lossy) downgrade in case metadata has changes in one of the versions between target and current, as defined in MetadataVersion. The documentation says it is possible: "Note that the cluster metadata version cannot be downgraded to a pre-production 3.0.x, 3.1.x, or 3.2.x version once it has been upgraded. However, it is possible to downgrade to production versions such as 3.3-IV0, 3.3-IV1, etc." The command line tool shows that this doesn't work: bin/kafka-features.sh --bootstrap-server :9092 downgrade --metadata 3.4 --unsafe Could not downgrade metadata.version to 8. Invalid metadata.version 8. Unsafe metadata downgrade is not supported in this version. 1 out of 1 operation(s) failed. In addition to unsafe, also safe metadata downgrades are not supported in practice. For example, when you upgrade to 3.5, you land on 3.5-IV2 as metadata version, which has metadata changes and won't let you to downgrade. This is true for every other release at the moment. This change fixes the documentation to reflect that, and improves the error messages. Signed-off-by: Federico Valeri <[email protected]> Reviewers: Luke Chen <[email protected]>, Jakub Scholz <[email protected]>
dima5rr · Oct 12, 2023 · aec07f7 · aec07f7
1 parent 7d1847c
commit aec07f7
Show file tree

Hide file tree

Showing 4 changed files with 17 additions and 11 deletions.
diff --git a/docs/upgrade.html b/docs/upgrade.html
@@ -144,8 +144,10 @@ <h5><a id="upgrade_350_kraft" href="#upgrade_350_kraft">Upgrading KRaft-based cl
                 ./bin/kafka-features.sh upgrade --metadata 3.5
             </code>
         </li>
-        <li>Note that the cluster metadata version cannot be downgraded to a pre-production 3.0.x, 3.1.x, or 3.2.x version once it has been upgraded.
-            However, it is possible to downgrade to production versions such as 3.3-IV0, 3.3-IV1, etc.</li>
+        <li>Note that cluster metadata downgrade is not supported in this version since it has metadata changes.
+            Every <a href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java">MetadataVersion</a>
+            after 3.2.x has a boolean parameter that indicates if there are metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means this version has metadata changes).
+            Given your current and target versions, a downgrade is only possible if there are no metadata changes in the versions between.</li>
     </ol>
 
 <h4><a id="upgrade_3_4_0" href="#upgrade_3_4_0">Upgrading to 3.4.0 from any version 0.8.x through 3.3.x</a></h4>
@@ -205,8 +207,10 @@ <h4><a id="upgrade_kraft_3_4_0" href="#upgrade_kraft_3_4_0">Upgrading a KRaft-ba
                 ./bin/kafka-features.sh upgrade --metadata 3.4
             </code>
         </li>
-        <li>Note that the cluster metadata version cannot be downgraded to a pre-production 3.0.x, 3.1.x, or 3.2.x version once it has been upgraded.
-            However, it is possible to downgrade to production versions such as 3.3-IV0, 3.3-IV1, etc.</li>
+        <li>Note that cluster metadata downgrade is not supported in this version since it has metadata changes.
+            Every <a href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java">MetadataVersion</a>
+            after 3.2.x has a boolean parameter that indicates if there are metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means this version has metadata changes).
+            Given your current and target versions, a downgrade is only possible if there are no metadata changes in the versions between.</li>
     </ol>
 
 <h5><a id="upgrade_340_notable" href="#upgrade_340_notable">Notable changes in 3.4.0</a></h5>
@@ -273,7 +277,10 @@ <h4><a id="upgrade_kraft_3_3_1" href="#upgrade_kraft_3_3_1">Upgrading a KRaft-ba
         ./bin/kafka-features.sh upgrade --metadata 3.3
         </code>
     </li>
-    <li>Note that the cluster metadata version cannot be downgraded to a pre-production 3.0.x, 3.1.x, or 3.2.x version once it has been upgraded. However, it is possible to downgrade to production versions such as 3.3-IV0, 3.3-IV1, etc.</li>
+    <li>Note that cluster metadata downgrade is not supported in this version since it has metadata changes.
+        Every <a href="https://github.com/apache/kafka/blob/trunk/server-common/src/main/java/org/apache/kafka/server/common/MetadataVersion.java">MetadataVersion</a>
+        after 3.2.x has a boolean parameter that indicates if there are metadata changes (i.e. <code>IBP_3_3_IV3(7, "3.3", "IV3", true)</code> means this version has metadata changes).
+        Given your current and target versions, a downgrade is only possible if there are no metadata changes in the versions between.</li>
 </ol>
 
 <h5><a id="upgrade_331_notable" href="#upgrade_331_notable">Notable changes in 3.3.1</a></h5>

diff --git a/metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java b/metadata/src/main/java/org/apache/kafka/controller/FeatureControlManager.java
@@ -346,9 +346,10 @@ private ApiError updateMetadataVersion(
                 return invalidMetadataVersion(newVersionLevel, "Unsafe metadata downgrade is not supported " +
                         "in this version.");
             } else {
+                // The phrase "Retry using UNSAFE_DOWNGRADE if you want to force the downgrade to proceed." has been removed
+                // because unsafe metadata downgrades are not yet supported. We can add it back when implemented (KAFKA-13896).
                 return invalidMetadataVersion(newVersionLevel, "Refusing to perform the requested " +
-                        "downgrade because it might delete metadata information. Retry using " +
-                        "UNSAFE_DOWNGRADE if you want to force the downgrade to proceed.");
+                        "downgrade because it might delete metadata information.");
             }
         } else {
             log.warn("Upgrading metadata.version from {} to {}.", currentVersion, newVersion);

diff --git a/metadata/src/test/java/org/apache/kafka/controller/FeatureControlManagerTest.java b/metadata/src/test/java/org/apache/kafka/controller/FeatureControlManagerTest.java
@@ -317,8 +317,7 @@ public void testCannotUseSafeDowngradeIfMetadataChanged() {
         assertEquals(ControllerResult.of(Collections.emptyList(),
             singletonMap(MetadataVersion.FEATURE_NAME, new ApiError(Errors.INVALID_UPDATE_VERSION,
                 "Invalid metadata.version 4. Refusing to perform the requested downgrade because " +
-                "it might delete metadata information. Retry using UNSAFE_DOWNGRADE if you want to " +
-                "force the downgrade to proceed."))),
+                "it might delete metadata information."))),
             manager.updateFeatures(
                 singletonMap(MetadataVersion.FEATURE_NAME, MetadataVersion.IBP_3_3_IV0.featureLevel()),
                 singletonMap(MetadataVersion.FEATURE_NAME, FeatureUpdate.UpgradeType.SAFE_DOWNGRADE),

diff --git a/tools/src/test/java/org/apache/kafka/tools/FeatureCommandTest.java b/tools/src/test/java/org/apache/kafka/tools/FeatureCommandTest.java
@@ -145,8 +145,7 @@ public void testDowngradeMetadataVersionWithKRaft() {
 
         );
         assertEquals("Could not downgrade metadata.version to 4. Invalid metadata.version 4. " +
-                "Refusing to perform the requested downgrade because it might delete metadata information. " +
-                "Retry using UNSAFE_DOWNGRADE if you want to force the downgrade to proceed.", commandOutput);
+                "Refusing to perform the requested downgrade because it might delete metadata information.", commandOutput);
 
         commandOutput = ToolsTestUtils.captureStandardOut(() ->
                 assertEquals(1, FeatureCommand.mainNoExit("--bootstrap-server", cluster.bootstrapServers(),