Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Work_Item] Add support for coverage eligibility #406

Open
3 tasks
cnharris10 opened this issue Apr 18, 2024 · 9 comments · May be fixed by #407
Open
3 tasks

[Work_Item] Add support for coverage eligibility #406

cnharris10 opened this issue Apr 18, 2024 · 9 comments · May be fixed by #407
Assignees
Labels
csp Cloud service providers metrics Fields that can be aggregated work item Issues to be considered for spec development

Comments

@cnharris10
Copy link
Contributor

cnharris10 commented Apr 18, 2024

1. Problem Statement *

  • What is the problem?: Explain the context and why it needs resolution.
  • Impact: Describe how the problem affects users, systems, or the project.

Problem: It is a non-trivial task to determine whether an on-demand usage-based charge is eligible for a commitment (and which commitment) or not.

Impact: Commitment discount efficiency and rate optimization analysis is inaccessible or unachievable for practitioners with a provider-native dataset, without joining additional data or context. This adds undue overhead to computation of commitment coverage, and undermines decision-making capability, particularly around purchases and renewals.

Use-case: Analyze resource costs by SKU

For this use-case, additional information around coverage eligibility allows practitioners to determine whether a resource's cost can be reduced or not.


2. Objective *

  • Success Criteria: Define how success will be measured (e.g. metrics and KPIs).

Success Criteria:

  • Practitioners are able to discern commitment eligibility of a charge
  • Practitioners are able to identify type of commitment eligibility
  • Practitioners are able to systematically compute commitment coverage and usage

3. Supporting Documentation *

Example restrictions across AWS, Azure, GCP
  1. AWS
    a. Spot instances
    b. P5 instance family
    c. Mac instance families
    d. Ubuntu OS

  2. Azure
    a. Spot VM's
    b. VM series - A-series, Av2-series, or G-series

  3. GCP
    a. Preemptible VM's
    b. n1 shared-cost, or extended memory VM's


4. Proposed Solution / Approach

  • Initial Ideas: Describe potential solution paths, tools, or technologies.
  • Considerations: Include any constraints, dependencies, or risks.
  • Feasibility: Include any information that helps quantify feasibility, such as perceived level of effort to augment the spec, or existing fields in current data generator exports.
  • Benchmarks: Are there established best practices for solving this problem available to practitioners today (e.g. mappings from existing CSP exports that are widely used)?

Initial considerations:

  • Extend an existing column, or introduce a new column to capture the eligibility of a charge for commitment discounts.
  • SKU eligibility for discounts is a known property by providers
  • Some SKUs are eligible for multiple types of commitments

To date, CSP providers do not expose a coverage eligibility dimension in any cost and usage datasets but due expose coverage metrics within native cost management tools.


5. Epic or Theme Association

This section will be completed by the Maintainers.

  • Epic: [Epic Name]
  • Theme: [Theme Name, if applicable]

6. Stakeholders *

  • Primary Stakeholders: [Name/Role]
  • Other Involved Parties: [Names/Roles]

Companies expressing desire for this feature

Atlassian

@github-project-automation github-project-automation bot moved this to Triage in FOCUS WG Apr 18, 2024
@cnharris10 cnharris10 self-assigned this Apr 18, 2024
@cnharris10 cnharris10 added this to the v1.1 milestone Apr 18, 2024
@cnharris10 cnharris10 changed the title Add Commitment-Based Coverage columns to rows that are or potentially can be covered by a Commitment. Add a Commitment-Based Coverage column to rows that are or potentially can be covered by a Commitment. Apr 18, 2024
@mike-finopsorg
Copy link
Contributor

We just need to be clear the goal here isn't for the practitioner to use the value as a direct coverage calculation (e.g. Lines that are covered by a commitment-based discount over those lines that are eligible for a commitment-based discount).

However, that isn't the right approach to "coverage" calculations just because a single line charge could be covered by a commitment, does not mean it is financially sensible to have it covered. While I see the value in this field I think we need to be clear on how the practitioner should use the column correctly.

@jpradocueva jpradocueva added the P2 label May 21, 2024
@jpradocueva jpradocueva moved this from Triage to Parking Lot in FOCUS WG May 21, 2024
@cnharris10
Copy link
Contributor Author

cnharris10 commented May 21, 2024

We just need to be clear the goal here isn't for the practitioner to use the value as a direct coverage calculation (e.g. Lines that are covered by a commitment-based discount over those lines that are eligible for a commitment-based discount).

However, that isn't the right approach to "coverage" calculations just because a single line charge could be covered by a commitment, does not mean it is financially sensible to have it covered. While I see the value in this field I think we need to be clear on how the practitioner should use the column correctly.

@mike-finopsorg

100%. The goal is to expose what is eligible for coverage by commitments, because it's hard enough figuring out what is/is not eligible. We should definitely clarify that we're not giving an opinion on a coverage strategy based on aspects like target rates, on-demand fluctuation percentages, or similar.

I envision people doing roughly the following:

1. Determining their coverage percentage.

(SELECT SUM(ChargePeriodEnd - ChargePeriodStart) FROM <table> WHERE CommitmentDiscountCategory IS NOT NULL AND ChargeSubCategory = 'Used') 
/
(SELECT SUM(ChargePeriodEnd - ChargePeriodEnd) FROM <table> WHERE CommitmentDiscountCategory IS NOT NULL AND ChargeSubCategory IN ('Used', 'On-Demand'))
* 100

2. If above coverage rate target, query rows (or rollups), drill down further into coverage rates above where coverage is higher than expected.

3. If below coverage rate target, query rows (or rollups) with on-demand spend with a CommitmentDiscountCategory value to determine what to do next.

SELECT * FROM <table> WHERE ChargeCategory = 'Usage' AND ChargeSubCategory = 'On-Demand' AND CommitmentDiscountCategory IS NOT NULL

@jpradocueva jpradocueva removed the P2 label Oct 10, 2024
@cnharris10 cnharris10 added the 1.2 consideration To be considered for release 1.2 label Oct 11, 2024
@shawnalpay shawnalpay added the needs work item Needs an issue that adheres to the Work Item issue template, prior to consideration by stakeholders label Oct 16, 2024
@shawnalpay shawnalpay assigned aqu-erp and unassigned aqu-erp Oct 16, 2024
@jpradocueva
Copy link
Contributor

jpradocueva commented Oct 17, 2024

Summary TF-2 call on Oct 16:

#406 Add Commitment-Based Coverage Column
Key Discussion Items: Adding a column to identify rows that are, or could be, covered by a commitment-based discount.
Problem Identification: It is difficult to determine which specific SKUs are eligible for commitment-based discounts.
Divergent Views: Some participants mentioned this issue was of lower priority, while others believed it was important for commitment utilization.
Final Agreement: Chris will create a work item for this, with further refinement needed for different providers.
Action Items:

@cnharris10
Copy link
Contributor Author

@aqu-erp to produce the work item for this issue

@cnharris10 cnharris10 added work item Issues to be considered for spec development and removed needs work item Needs an issue that adheres to the Work Item issue template, prior to consideration by stakeholders labels Oct 17, 2024
@shawnalpay
Copy link
Contributor

@aqu-erp @cnharris10 As the first authors to have completed a Work Item, you have become our guinea pigs for the process! So please bear with me as we work through a few items that come to mind, now that we see the issue fully fleshed out. I'd love to use this issue as our main example for other Work Item authors to reference as we move forward. So, to that end:

  • Could you please rename the issue in a way that is more conceptual than solution-oriented?
    • For example, instead of saying we're going to add a column (which we may very well do), I believe it would be helpful to make clear that we will address a particular idea. Perhaps Add support for commitment eligibility or Add support for coverable spend or something along those lines. Maybe that seems like a small change, but I think it will be very helpful when it comes time to explain to our stakeholders.
  • Section 1: could you please add at least one use case?
    • Several come to mind, but perhaps the easiest and most elucidating for the average practitioner might be Report coverage % across all resources and services. Ideally following the format as listed in the use case library here.
  • Section 1: In the problem statement, perhaps we keep it more grounded in the world of FOCUS?
    • For example, perhaps we go farther than "non-trivial" and say something like coverage % is not possible to calculate in FOCUS because spend that is eligible for coverage is not tracked. If the practitioner is using FOCUS exclusively, it's straight-up impossible to calculate coverage % without external enrichment. I think making this statement helps clarify why this is important to do, rather than just let the practitioner deal with it downstream.
  • Section 4: is this at all a datapoint that you've seen any data generator provide? Some verbiage that touches on whether this is a known field will be helpful for future discussions. My understanding is that this is not an exposed field in any of the CSP exports; is this also your understanding?

I super appreciate you assembling this!

@shawnalpay shawnalpay added needs use case Needs a description of the why (use case or other problem to solve) metrics Fields that can be aggregated labels Oct 17, 2024
@cnharris10 cnharris10 changed the title Add a Commitment-Based Coverage column to rows that are or potentially can be covered by a Commitment. [Work_Item] Add a Commitment-Based Coverage column to rows that are or potentially can be covered by a Commitment. Oct 20, 2024
@cnharris10 cnharris10 changed the title [Work_Item] Add a Commitment-Based Coverage column to rows that are or potentially can be covered by a Commitment. [Work_Item] Add support for coverage eligibility Oct 24, 2024
@cnharris10
Copy link
Contributor Author

@aqu-erp @cnharris10 As the first authors to have completed a Work Item, you have become our guinea pigs for the process! So please bear with me as we work through a few items that come to mind, now that we see the issue fully fleshed out. I'd love to use this issue as our main example for other Work Item authors to reference as we move forward. So, to that end:

  • Could you please rename the issue in a way that is more conceptual than solution-oriented?

    • For example, instead of saying we're going to add a column (which we may very well do), I believe it would be helpful to make clear that we will address a particular idea. Perhaps Add support for commitment eligibility or Add support for coverable spend or something along those lines. Maybe that seems like a small change, but I think it will be very helpful when it comes time to explain to our stakeholders.
  • Section 1: could you please add at least one use case?

    • Several come to mind, but perhaps the easiest and most elucidating for the average practitioner might be Report coverage % across all resources and services. Ideally following the format as listed in the use case library here.
  • Section 1: In the problem statement, perhaps we keep it more grounded in the world of FOCUS?

    • For example, perhaps we go farther than "non-trivial" and say something like coverage % is not possible to calculate in FOCUS because spend that is eligible for coverage is not tracked. If the practitioner is using FOCUS exclusively, it's straight-up impossible to calculate coverage % without external enrichment. I think making this statement helps clarify why this is important to do, rather than just let the practitioner deal with it downstream.
  • Section 4: is this at all a datapoint that you've seen any data generator provide? Some verbiage that touches on whether this is a known field will be helpful for future discussions. My understanding is that this is not an exposed field in any of the CSP exports; is this also your understanding?

I super appreciate you assembling this!

Added additional information based on most points above including a title change, use-case, and section 4

@cnharris10 cnharris10 removed the needs use case Needs a description of the why (use case or other problem to solve) label Oct 24, 2024
@shawnalpay shawnalpay added the csp Cloud service providers label Oct 29, 2024
@smokemonster99
Copy link

Big support for this one as we have folks at Capital One whose primary duties are optimizing CUD and planning future purchases by analyzing coverage and utilization. I believe this feature will help out in that regard.

@jpradocueva
Copy link
Contributor

Notes from the Maintainers' call on November 4:

Context: This item seeks to track costs eligible for commitments (like Reserved Instances) to help practitioners accurately calculate coverage KPIs. Coverage metrics are essential in understanding and managing commitment utilization in cloud environments.
Level of Effort Required: Medium — Supporting commitment-eligible cost tracking is moderately complex, requiring integration across providers with different commitment models.
Level of Impact: High – Commitment-eligible cost tracking is critical for calculating coverage KPIs, which are core metrics in FinOps. This feature helps practitioners monitor commitment utilization, improving budget planning and resource allocation.

@jpradocueva jpradocueva removed this from the v1.1 milestone Nov 5, 2024
@flanakin
Copy link
Contributor

flanakin commented Nov 7, 2024

nit: Should we rename this to "commitment discount eligibility"?

@shawnalpay shawnalpay removed the 1.2 consideration To be considered for release 1.2 label Nov 27, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
csp Cloud service providers metrics Fields that can be aggregated work item Issues to be considered for spec development
Projects
Status: Parking Lot
7 participants