Automate BM usage calculations

[joachimweyl]

Motivation

We will have more BM users as time goes by. We also want to be able to charge for lease time only.

Completion Criteria

CSV with BM usage by project with cost associated.

Description

• Create 3 new tables to pull from
  • Project Table - filled with info we need to gather when we create new ESI projects
    • ESI project - value from ESI project column
    • Invoice Project name = " BM Usage" created by combining the projects name with BM Usage
    • PI - we will have to ask when we create the ESI project or if owner column is filled we can use that
    • Institution
  • Server Types - or maybe this should be added to nerc-rates?
    • Server
      • BM FC430 - fc430 in ESI Resource Column
      • BM FC830 - fc830 in ESI Resource Column
      • BM GPUA100SXM4 - lenovo-sd650nv2-a100 or sd650nv2 in ESI Resource Column
      • BM GPUH100 - no ESI Resource Column yet but once we have it we want to add it here
    • Cost - per hour
      • $0.75
      • $3.97
      • $8.312
      • $24.16
  • BM Usage
    • Invoice Month
    • Project - Allocation
    • Project - Allocation ID
    • Manager (PI)
    • Invoice Email
    • Invoice Address
    • Institution
    • Institution - Specific Code
    • SU Hours (GBhr or SUhr)
    • SU Type
    • Rate
    • Cost
    • Credit
    • Credit Code
    • Balance
• Gather data from ESI at 12:01 am on the 1st of the month
  • OpenStack esi lease list --status any --long should give you a list to pull from
    • this will have data for each lease with a start, end and expire time, we need to SUM all lease time during the previous month for each node for each project.
      • If Start time is earlier than first of month use first of month
      • If end time is later than expire time use expire time otherwise use end time
        • If above is later than end of month use end of month
  • With that information, we can fill in BM Usage CSV when combined with the other tables we created
  • Many of the BM Usage columns will be empty but the ones below will be filled
  • Invoice Month is the previous month in this format "2024-10"
  • Project - Allocation
    • for example, ESI project "Noodles" becomes "Noodels BM Usage" links from
  • Project - Allocation ID becomes "ESI Bare Metal"
  • Manager (PI) comes from the Project Table above
  • Invoice Email is "[email protected]" for now
  • Institution comes from the Project Table above
  • SU Hours (GBhr or SUhr) comes from the summation we did above
  • Rate comes from Server Types or nerc-rates whichever we decided to hold this info
  • Cost Rate * SU Hours
  • Balance Cost-Credit (which is zero so basically just cost again)
• Setup a list of non-billable projects
  • Current list
    • admin
    • ajamias
    • anshul
    • cloudlab
    • d4n
    • danni
    • Edge-Manager
    • flightctl-dev
    • FlinkEnergy
    • hakan
    • hwbroken
    • hypershift-cluster
    • ibm_research
    • jappavoo
    • larsks
    • MOCPerf
    • naved
    • nerc-admins
    • nerc-test
    • okd
    • okdci
    • okdci_hosts
    • ope
    • orran_cloud_computing
    • racedoro
    • sina_ahmadi
    • skanthed
    • storage-optimization
    • storage-research
    • Triton
    • tzumainn
    • uitest
    • ukl
• Test that output looks similar to this to the right of column Invoice Month

Completion dates

Desired - 2025-01-26
Required - TBD

[joachimweyl]

@QuanMPhm This is worth starting to think about.