[Work_Item] Add Invoice ID

[kk09v]

Problem Statement

The "Procure to Pay" process (P2P) is pretty well defined:

1. A Purchase Order is issued for the purchase of goods or services
2. The goods are received or receipt of a service is confirmed
3. An invoice sent by the supplier is matched against the receipts to determine whether the invoice should be paid in full or in part
   The process by which these 3 documents are evaluated for alignment and determination is made on whether to pay is called the "3-way match."

Also see EDI (Electronic Data Interchange) the set of standards which include how data will be exchanged for this process as it relates to physical goods.

With Public Cloud, the Procure to Pay process still exists, but it is much less mature than in other industries. Cloud Service Provider invoices do not typically contain a complete list of all goods and services purchased; it is often summarized and usually refers to the the CSP's cost and usage data export process (including FOCUS, if available).

Inclusion of the invoice # in the FOCUS data will allow teams to positively identify which charges, credits, adjustments, etc. are applied to each invoice and improve the 3-way match process.

Use cases:

• Verify accuracy of provider invoices aka invoice reconciliation
  • In order to verify that the charges found in the FOCUS dataset match those of the providers invoices a FinOps Practitioner can aggregate cost data from providers for a billing period and compare it to invoices.
• Perform showback / chargeback activities
  • Quote from Narisco: "We need to distribute all cost back to each cost center and then group cost by Billing Account ID, Invoice Issuer and Invoice ID so that our finance department can charge back the business units and pay the invoices.
    The sum of the invoices for a given Billing Account ID and Invoice Issuer must match to the cent the sum of the chargeback per business unit for that same perimeter. Without Invoice ID there are situations like when an account is moved between different Billing Accounts, where you loose track and are unable to properly match those."
• Analyze credit memos
  • Quote from Narisco: "Another important use case are Credit Memos (Refund Invoices): AWS creates Credit Memos and they show up in the CUR at that point in time, however there is a manual process involved to request that Credit Memo to be applied to a given regular Invoice. The result is that you cannot usually include the credit memo in the same billing period. We set the applicable billing period for each Credit Memo to take them into account, and for that we need the Invoice ID to be able to discriminate the right rows of data in the input."

Objective

Success Criteria: A practitioner should be able to tie out their invoice from a CSP (to the penny or lowest denomination of billing currency) to the sum of BilledCost having the corresponding InvoiceId (presumptive column name) in their FOCUS dataset from that CSP.

Supporting Documentation

Discussion Documents: Proposal from 1.0 #302
Duplicate issue created in Oct 2024 #607

Proposed Solution / Approach

• Initial Ideas: An additional field be included in FOCUS to contain the invoice number on which that record is charged
• Considerations:
  • The majority of CSP contracts are post-paid, so the invoice will be issued at a later date than when the usage happens and thus will require the FOCUS data to be restated for the billing period once the invoice is issued.
  • BillingPeriodStart and BillingPeriodEnd are related columns which assumed providers have discrete periodic billing and invoices aligned to this. These columns will need to be reviewed for consistency/compatibility with this new column and provider invoicing practices.
• Feasibility: This or similar data is already provided in some way or another by multiple CSPs.
  • GCP invoice.month takes the form YYYYMM (e.g. 202406)
  • Azure has x_InvoiceId
• Benchmarks: EDI is a standard used for other industries

Epic or Theme Association

TBD

Stakeholders

Primary Stakeholder: TBD

Other Parties Involved:

• Narciso Cerezo (BBVA) @narciso-cerezo
  • "We need to distribute all cost back to each cost center and then group cost by Billing Account ID, Invoice Issuer and Invoice ID so that our finance department can charge back the business units and pay the invoices... Without Invoice ID there are situations like when an account is moved between different Billing Accounts, where you loose track and are unable to properly match those... Adding the Invoice ID in all three providers would help us implement the exact same process for all three, which would be great." - excerpts from [FEEDBACK]: Invoice ID #607
• Brett McCulloch (General Dynamics IT) @bmcculloch95
  • "A big use case gap we've been working to overcome is mapping out usage back to its invoice. While the framework makes it easy to match charge totals to each invoice type, not having the InvoiceID as an available field means that information is still needing to be pulled from the CSP outside of FOCUS. Having the InvoiceID added to a FOCUS release down the road would be a great addition in my opinion." Slack post (here)

[shawnalpay]

@kk09v Thank you for standing up this work item, Karl! Could you add at least one use case in the format of our use-case library? Also, data examples will be very helpful in telling this story -- could you add at least one example of how the data looks from one of the existing CSP exports?

[ljadvey]

+1 on this work item. I was thinking about this in our SaaS call - would be good for CSP and SaaS datasets.

[jpradocueva]

Notes from the Maintainers' call on November 4:

Context: Adding an Invoice ID column would improve traceability by enabling practitioners to link costs and usage to specific invoices, facilitating easier reconciliation and tracking.
Level of Effort Required: Low — Adding an additional column is a straightforward adjustment with minimal complexity.

[jpradocueva]

Summary from the Maintainers' call on Nov 25

Context:
The goal is to include a specific field for "Invoice ID" to improve traceability and reconciliation of billing data within the FOCUS dataset.
Maintainers Assigned:
Karl, Alex, Riley, Irena, someone from OCI?
This can be an easy PR. It is a great opportunity for someone else to write the PR.
Task Force Assigned:
Task Force 2 (TF2).

Next Steps from the TF-2 call on November 27:

• [#615] Riley @rileyjenk & Alex @ahullah Karl @kk09v : Begin drafting the initial specification for the Invoice ID field, including guidelines for when and how the field should be populated.
• [#615] Joaquin @jpradocueva : Identify and reach out to community members who might be interested in contributing to this work item, particularly those who have not yet submitted PRs.
• [#615] Irena @ijurica : Research provider capabilities to determine which account types can reliably provide Invoice ID data.
• [#615] Task Force: Add the draft specification to the agenda for review in the next TF2 meeting.
• [#615] All Members: Review the draft specification when shared and provide feedback before the next meeting.
• [#615] Michael @flanakin : Confirm with Microsoft team if Invoice ID data is available across all account types, especially for enterprise agreements.
• [#615] David: Provide GCP’s perspective on the feasibility of including Invoice ID data for different billing scenarios.

[jpradocueva]

Action Items from the Maintainers' call on December 16:

• [#615] Alex @ahullah : Post a message in the working group to request a volunteer for this work item.