[CELEBORN-1037] Incorrect output for metrics of Prometheus

### What changes were proposed in this pull request? The new added `deadlocks` metrics in `ThreadStatesGaugeSet` is a Set<String>, which is invalid. So here add a filter at the `addGauge` extrance. ### Why are the changes needed? Ditto ### Does this PR introduce _any_ user-facing change? Remove the unused metrics. BTW the template use `metrics_jvm_thread_deadlock_count_Value` ### How was this patch tested? Manual test Closes #1981 from onebox-li/fix-1037. Authored-by: onebox-li <[email protected]> Signed-off-by: mingji <[email protected]>
apache · Oct 13, 2023 · 8b1bd07 · 8b1bd07
1 parent c97628c
commit 8b1bd07
Show file tree

Hide file tree

Showing 2 changed files with 234 additions and 194 deletions.
diff --git a/common/src/main/scala/org/apache/celeborn/common/metrics/source/AbstractSource.scala b/common/src/main/scala/org/apache/celeborn/common/metrics/source/AbstractSource.scala
@@ -73,7 +73,13 @@ abstract class AbstractSource(conf: CelebornConf, role: String)
       name: String,
       labels: Map[String, String],
       gauge: Gauge[T]): Unit = {
-    namedGauges.add(NamedGauge(name, gauge, labels ++ staticLabels))
+    // filter out non-number type gauges
+    if (gauge.getValue.isInstanceOf[Number]) {
+      namedGauges.add(NamedGauge(name, gauge, labels ++ staticLabels))
+    } else {
+      logWarning(
+        s"Add gauge $name failed, the value type ${gauge.getValue.getClass} is not a number")
+    }
   }
 
   def addGauge[T](