Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Updating end of interval Flex metrics publishing #10210

Merged
merged 1 commit into from
Jun 6, 2024
Merged

Updating end of interval Flex metrics publishing #10210

merged 1 commit into from
Jun 6, 2024

Conversation

mathewc
Copy link
Member

@mathewc mathewc commented Jun 5, 2024

Currently, we only calculate and emit activity metrics during periods of function inactivity. This works well when relatively short lived invocations start stop often during metering windows. However, this doesn't work well in cases where functions are long running (e.g. longer than the default 5ms metering interval), or when invocations are very frequent such that they overlap and there's rarely a period of inactivity. In such cases, we can go many metering intervals without emitting metrics, continuing to accumulate in memory. Eventually the correct duration will be emitted, but it's all shoved into a single interval. In addition to making metrics reporting spiky, this also makes it more likely that metrics may be lost, e.g. if the process dies abruptly.

Instead, we want to ensure that during any interval, we calculate and emit metrics for any outstanding activity in that interval. In this way, even if there are no idle periods, metrics are progressively emitted.

Pull request checklist

IMPORTANT: Currently, changes must be backported to the in-proc branch to be included in Core Tools and non-Flex deployments.

  • Backporting to the in-proc branch is not required
    • Otherwise: Link to backporting PR
  • My changes do not require documentation changes
    • Otherwise: Documentation issue linked to PR
  • My changes should not be added to the release notes for the next release
    • Otherwise: I've added my notes to release_notes.md
  • My changes do not need to be backported to a previous version
    • Otherwise: Backport tracked by issue/PR #issue_or_pr
  • My changes do not require diagnostic events changes
    • Otherwise: I have added/updated all related diagnostic events and their documentation (Documentation issue linked to PR)
  • I have added all required tests (Unit tests, E2E tests)

Additional information

Additional PR information

@mathewc mathewc requested a review from a team as a code owner June 5, 2024 17:27
Copy link
Member

@paulbatum paulbatum left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This batch of changes looks good, but I did find one thing I'm not sure about in the existing logic.

@mathewc mathewc merged commit d1067c5 into dev Jun 6, 2024
14 checks passed
@mathewc mathewc deleted the flex-metrics branch June 6, 2024 15:48
@mathewc mathewc mentioned this pull request Jun 6, 2024
9 tasks
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants