Skip to content

Commit

Permalink
Include QoS ID standard extension
Browse files Browse the repository at this point in the history
  • Loading branch information
ved-rivos committed Sep 10, 2024
1 parent 095c674 commit ba735d7
Show file tree
Hide file tree
Showing 4 changed files with 159 additions and 6 deletions.
40 changes: 40 additions & 0 deletions src/iommu.bib
Original file line number Diff line number Diff line change
Expand Up @@ -25,3 +25,43 @@ @electronic{PR243
title = {Clarification updates to IOMMU v1.0.0},
url = {https://github.com/riscv-non-isa/riscv-iommu/pull/243/commits}
}
@electronic{CBQRI,
title = {RISC-V Capacity and Bandwidth QoS Register Interface},
url = {https://github.com/riscv-non-isa/riscv-cbqri}
}
@article{PTCAMP,
author = {Du Bois, Kristof and Eyerman, Stijn and Eeckhout, Lieven},
title = {Per-Thread Cycle Accounting in Multicore Processors},
year = {2013},
issue_date = {January 2013},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
volume = {9},
number = {4},
issn = {1544-3566},
url = {https://doi.org/10.1145/2400682.2400688},
doi = {10.1145/2400682.2400688},
journal = {ACM Trans. Archit. Code Optim.},
month = {jan},
articleno = {29},
numpages = {22},
}
@inproceedings{HERACLES,
author = {Lo, David and Cheng, Liqun and Govindaraju, Rama and Ranganathan, Parthasarathy and Kozyrakis, Christos},
title = {Heracles: Improving Resource Efficiency at Scale},
year = {2015},
isbn = {9781450334020},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/2749469.2749475},
doi = {10.1145/2749469.2749475},
booktitle = {Proceedings of the 42nd Annual International Symposium on Computer Architecture},
pages = {450–462},
numpages = {13},
location = {Portland, Oregon},
series = {ISCA '15}
}
@electronic{SSQOSID,
title = {RISC-V Quality-of-Service (QoS) Identifiers},
url = {https://github.com/riscv/riscv-ssqosid}
}
22 changes: 21 additions & 1 deletion src/iommu_data_structures.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -484,6 +484,7 @@ the PTEs from the first page table or the second page table. These are the only
expected behaviors.
====
[[DC_TA]]
===== Translation attributes (`ta`)
.Translation attributes (`ta`) field
Expand All @@ -492,7 +493,9 @@ expected behaviors.
{reg: [
{bits: 12, name: 'reserved'},
{bits: 20, name: 'PSCID'},
{bits: 32, name: 'reserved'},
{bits: 8, name: 'reserved'},
{bits: 12, name: 'RCID'},
{bits: 12, name: 'MCID'},
], config:{lanes: 2, hspace: 1024, fontsize: 16}}
....
Expand All @@ -502,6 +505,21 @@ fences on a per-address-space basis. The `PSCID` field in `ta` is used as the
address-space ID if `DC.tc.PDTV` is 0 and the `iosatp.MODE` field is not `Bare`.
When `DC.tc.PDTV` is 1, the `PSCID` field in `ta` is ignored.
The `RCID` and `MCID` fields are added by the QoS ID extension. If
`capabilities.QOSID` is 0, these bits are reserved and must be set to 0.
IOMMU-initiated requests for accessing the following data structures use the
value configured in the `RCID` and `MCID` fields of `DC.ta`.
* Process directory table (`PDT`)
* Second-stage page table
* First-stage page table
* MSI page table
* Memory-resident interrupt file (`MRIF`)
The `RCID` and `MCID` configured in `DC.ta` are provided to the IO bridge on
successful address translations. The IO bridge should associate these QoS IDs
with device-initiated requests.
===== First-Stage context (`fsc`)
If `DC.tc.PDTV` is 0, the `DC.fsc` field holds the `iosatp` that provides
the controls for first-stage address translation and protection.
Expand Down Expand Up @@ -716,6 +734,8 @@ misconfigured" (cause = 259).
. `DC.tc.SBE` value is not a legal value. If `fctl.BE` is writable
then `DC.tc.SBE` may be 0 or 1. If `fctl.BE` is not writable then
`DC.tc.SBE` must be the same as `fctl.BE`.
. `capabilities.QOSID` is 1 and `DC.ta.RCID` or `DC.ta.MCID` values
are wider than that supported by the IOMMU.
[NOTE]
====
Expand Down
92 changes: 91 additions & 1 deletion src/iommu_extensions.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,4 +2,94 @@

== IOMMU Extensions

This chapter specifies the standard extensions to the IOMMU Base Architecture.
This chapter specifies the following standard extensions to the IOMMU Base
Architecture:

[%autowidth,float="center",align="center",cols="^,^,^",options="header",]
|===
| Specification |Version |Status
| <<QOSID, *Quality-of-Service (QoS) Identifiers Extension*>>
|*1.0*
|*Ratified*
|===

[[QOSID]]
=== Quality-of-Service (QoS) Identifiers Extension, Version 1.0

Quality of Service (QoS) is defined as the minimal end-to-end performance
guaranteed in advance by a service level agreement (SLA) to a workload.
Performance metrics might include measures such as instructions per cycle (IPC),
latency of service, etc.

When multiple workloads execute concurrently on modern processors -- equipped
with large core counts, multiple cache hierarchies, and multiple memory
controllers -- the performance of any given workload becomes less
deterministic, or even non-deterministic, due to shared resource contention
cite:[PTCAMP].

To manage performance variability, system software needs resource allocation
and monitoring capabilities. These capabilities allow for the reservation of
resources like cache and bandwidth, thus meeting individual performance targets
while minimizing interference cite:[HERACLES]. For resource management, hardware
should provide monitoring features that allow system software to profile
workload resource consumption and allocate resources accordingly.

To facilitate this, the QoS Identifiers ISA extension (Ssqosid) cite:[SSQOSID]
introduces the `srmcfg` register, which configures a hart with two identifiers:
a Resource Control ID (`RCID`) and a Monitoring Counter ID (`MCID`). These
identifiers accompany each request issued by the hart to shared resource
controllers.

These identifiers are crucial for the RISC-V Capacity and Bandwidth Controller
QoS Register Interface cite:[CBQRI], which provides methods for setting resource
usage limits and monitoring resource consumption. The `RCID` controls resource
allocations, while the `MCID` is used for tracking resource usage.

The IOMMU QoS ID extension provides a method to associate QoS IDs with requests
to access resources by the IOMMU, as well as with devices governed by it. This
complements the Ssqosid extension that provides a method to associate QoS IDs
with requests originated by the RISC-V harts. Assocating QoS IDs with device
and IOMMU originated requests is required for effective monitoring and
allocation of shared resources.

The IOMMU `capabilities` register (<<CAP>>) is extended with a `QOSID` field
which enumerates support for associating QoS IDs with requests made through the
IOMMU. When `capabilities.QOSID` is 1, the memory-mapped register layout is
extended to add a register named `iommu_qosid` (<<IOQOSID>>). This register is
used to configure the Quality of Service (QoS) IDs associated with
IOMMU-originated requests. The `ta` field of the device context (<<DC_TA>>) is
extended with two fields, `RCID` and `MCID`, to configure the QoS IDs to
associate with requests originated by the devices.

==== Reset Behavior

If the reset value for `ddtp.iommu_mode` field is `Bare`, then the
`iommu_qosid.RCID` field must have a reset value of 0.

[NOTE]
====
At reset, it is required that the `RCID` field of `iommu_qosid` is set to 0 if
the IOMMU is in `Bare` mode, as typically the resource controllers in the
SoC default to a reset behavior of associating all capacity or bandwidth to the
`RCID` value of 0. When the reset value of the `ddtp.iommu_mode` is not `Bare`,
the `iommu_qosid` register should be initialized by software before changing
the mode to allow DMA.
====

==== Sizing QoS Identifiers

The size (or width) of `RCID` and `MCID`, as fields in registers or in data
structures, supported by the IOMMU must be at least as large as that supported
by any RISC-V application processor hart in the system.

==== IOMMU ATC Capacity Allocation and Monitoring

Some IOMMUs might support capacity allocation and usage monitoring in the IOMMU
address translation cache (IOATC) by implementing the capacity controller
register interface.

Additionally, some IOMMUs might support multiple IOATCs, each potentially having
different capacities. In scenarios where multiple IOATCs are implemented, such
as an IOATC for each supported page size, the IOMMU can implement a capacity
controller register interface for each IOATC to facilitate individual capacity
allocation.
11 changes: 7 additions & 4 deletions src/iommu_registers.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,7 +81,8 @@ the register returns 0 and writes to that offset are ignored.
control>> | if `capabilities.DBG==0`
|616 |`tr_response` |8 |<<TRR_RSP,Translation-request
response>> | if `capabilities.DBG==0`
|624 |Reserved |64 |Reserved for future use
|624 |`iommu_qosid` |4 |<<IOQOSID,IOMMU QoS ID >> | if `capabilities.QOSID==0`
|628 |Reserved |60 |Reserved for future use
(`WPRI`) |
|688 |custom |72 |Designated for custom use
(`WARL`) |
Expand Down Expand Up @@ -153,7 +154,8 @@ the IOMMU. At reset, the register shall contain the IOMMU supported features.
{bits: 1, name: 'PD8'},
{bits: 1, name: 'PD17'},
{bits: 1, name: 'PD20'},
{bits: 15, name: 'reserved'},
{bits: 1, name: 'QOSID'},
{bits: 14, name: 'reserved'},
{bits: 8, name: 'custom'},
], config:{lanes: 8, hspace:1024}}
....
Expand Down Expand Up @@ -223,7 +225,8 @@ the IOMMU. At reset, the register shall contain the IOMMU supported features.
|38 |`PD8` |RO | One level PDT with 8-bit process_id supported.
|39 |`PD17` |RO | Two level PDT with 17-bit process_id supported.
|40 |`PD20` |RO | Three level PDT with 20-bit process_id supported.
|55:41 | reserved |RO | Reserved for standard use.
|41 |`QOSID` |RO | Associating QoS IDs with requests is supported.
|55:42 | reserved |RO | Reserved for standard use.
|63:56 |custom |RO | Designated for custom use.
|===

Expand Down Expand Up @@ -1647,7 +1650,7 @@ The `iommu_qosid` register fields are defined as follows:
[width=100%]
[%header, cols="^1,2,^1,5"]
|===
|Bits ^|Field |Attribute ^| Description
|Bits |Field |Attribute | Description
|11:0 |`RCID` |WARL | `RCID` for IOMMU-initiated requests.
|15:12 |reserved |WPRI | Reserved for standard use.
|27:16 |`MCID` |WARL | `MCID` for IOMMU-initiated requests.
Expand Down

0 comments on commit ba735d7

Please sign in to comment.