Data Rollup

Data Rollup

Data in rating needs to follow a flow to be efficient. As input data can be related to many elements in the system, there is a need to define which data will be used, and clean it so it can be used in rating. The method to provide clean data is called mediation.

A mediation engine will normally perform the following: Following list provides insight on mediation software activities

• Collection and Archive
• Decoding/Encoding
• Normalization (Common Format)
• Filtering
• Conversion
• Validation
• Record Enrichment (Using Complex Reference Data)
• Duplicate Record Detection
• Aggregation or Correlation
• Buffering
• Cloning
• Sorting
• Downstream Format Mapping
• Error Messaging and Alarms
• Reconciliation
• etc

In ManageIQ, mediation will use the information from the database to generate data rollups, that have all the necessary data for the rating to be done.

Data Rollup: is a data rollup of information in the database, both from the inventory and the C&U database. It is unique per resource (VM, pod, service, etc), and represents all the data needed to be able to apply a rate to the measure. It is similar to a metric rollup per resource, being the main difference that is not generic, and can thus include preprocessed and derived data. Once the Data Rollup is updated, we don't need to go back to C&U or inventory data for the next steps.

Data Rollups are created based on the inventory, and updated depending on inventory + C&U data. They reflect the lifecycle of the resource they are measuring. Although they are similar to metrics rollup, there are some difference between both elements: metrics rollups are generic, and are used all through the product, and for that reason hold all the information available. Data rollups are specific to chargeback, and thus only hold that information needed for chargeback. On the other hand, data rollups in chargeback can be stored longer than the metrics rollup datatables, as the amount of content is lower. A customer will normally want to compare quarters between them, and with the same quarter in the previous year, making it necessary to store data for longer time.

Data Rollups are updated independently of rating, so they should always reflect up to date information.

Getting inventory allows us to charge for resources that have no c&u data, including those that don't exist in the database (i.e. set up fees).