Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Feature][Sort] Add new source metrics for sort-connector-pulsar-v1.15 #11340

Closed
2 tasks done
PeterZh6 opened this issue Oct 11, 2024 · 0 comments · Fixed by #11341
Closed
2 tasks done

[Feature][Sort] Add new source metrics for sort-connector-pulsar-v1.15 #11340

PeterZh6 opened this issue Oct 11, 2024 · 0 comments · Fixed by #11341
Assignees
Milestone

Comments

@PeterZh6
Copy link
Contributor

PeterZh6 commented Oct 11, 2024

Description

Parent Issue: #10961
Added source metrics in the same way as PostgreSQL-connector [Feature][Sort] Enhanced source metric instrumentation for InLong Sort Flink Connector #11129 for sort-connector-pulsar-v1.15

Key Metric Categories:

  1. Serialization/Deserialization Metrics:

    • Success/Error Counters: Track successful and failed deserialization attempts (numDeserializeSuccess, numDeserializeError).
    • Latency Gauges: Measure the time taken for both serialization and deserialization (deserializeTimeLag, serializeTimeLag).
  2. SnapshotState Metrics:

    • Creation/Error Counters: Monitor the number of snapshots created and errors encountered during snapshot operations (numSnapshotCreate, numSnapshotError).
  3. NotifyComplete Metrics:

    • Completed Snapshots Counter: Track the number of completed checkpoints (numCompletedSnapshots).
    • Snapshot-to-Checkpoint Latency: Record the time between snapshot creation and checkpoint completion (snapshotToCheckpointTimeLag).

Use case

Are you willing to submit PR?

  • Yes, I am willing to submit a PR!

Code of Conduct

PeterZh6 added a commit to PeterZh6/inlong that referenced this issue Oct 11, 2024
@dockerzhang dockerzhang added this to the 2.1.0 milestone Oct 13, 2024
@PeterZh6 PeterZh6 changed the title [Feature][Sort] Added new source metrics for sort-connector-pulsar-v1.15 [Feature][Sort] Add new source metrics for sort-connector-pulsar-v1.15 Oct 14, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants