Spline motivation: Identification of performance bottlenecks

[uday1409]

How can we track performance using Spline? Could you please elaborate on this how to identify in Spline UI

[wajda]

I assume you are talking about the jobs performance. Well, in short Spline offers somewhat limited options in that area, and we are rather speaking about future development. By design Spline can capture some execution metrics and store them in the execution event as an extra data - https://opensource.bigusdatus.com/spline/rest/docs/producer.html#operation/executionEventUsingPOST
That data can potentially be used for performance analysis. This part of the model isn't standardized however, so such analysis is only doable on data captured by a known agent, practically ad-hoc. For example, spark-spline-agent currently captures some very basic metrics - https://github.com/AbsaOSS/spline-spark-agent/blob/release/0.5.3/core/src/main/scala/za/co/absa/spline/harvester/LineageHarvester.scala#L192 See if it helps.
As a generic feature, it could be implemented in the future, but it requires more preparation.