Add support for flexible device hierarchy model

[jandres742]

Resolves: #1

[jandres742]

@wdamon-intel , @MichalMrozek: please review

[wdamon-intel]

LGTM, mostly, just a few minor comments to address.

[wdamon-intel]

[...] 0 device handles and and [...]
"and and"

[MichalMrozek]

in mode 0 DeviceGetRootDevice is not allowed.

[jandres742]

@MichalMrozek

You mean in mode 0, which is what we have currently, the application cannot query for the root device? could you elaborate on why would that be? this has been a request from some customers some times, to query for the parent device for a sub-device. From point of view of implementation, this doesnt change too much, as the Level Zero Driver implementation already has that info, it would be just a matter of returning it.

[MichalMrozek]

The API was only meant for mode 2, when you start to flatten the hierarchy.
If hierarchy is not flattened (as in mode 0), then getroot is not supported.

That's the whole difference between mode 2 and other modes, it allows this new API.

[jandres742]

@MichalMrozek : thanks. yes, the API was originally meant for mode 2, but there's no functional or performance reason to not have it available in mode 0, would that be correct? By having it in mode 0 as well, we allow users to have more flexibility and broader set of APIs to use.

[MichalMrozek]

let's say you are in mode 0
app did ZE_AFFINITY_MASK=0.0
app calls getRoot on such device, what is returned?

[jandres742]

@MichalMrozek it is return nullptr, but if you are in mode 0, and use ZE_AFFINITY_MASK=0, then you can call getRootDevice on any handle obtained with getSubDevices and get the parent device.

It is the same for getSubDevices. With MASK=0, getSubDevices for a root device returns a list of sub-devices, and with MASK=0.0 getSubDevices returns a count of 0. Spec doesnt forbid the use of getSubDevices with MASK=0.0, it just returns the appropriate value depending on the situation.

[MichalMrozek]

but in mode 1 , we will return all sub devices as root device and getRoot will return nullptr?

[jandres742]

@MichalMrozek Correct. in mode 1, getRootDevice returns nullptr, and also getSubDevices will return count of 0.

[MichalMrozek]

there are no sub devices when FLAT_DEVICE_HIERACHY is "1"

[jandres742]

thanks @MichalMrozek . I have expanded this section to make it clearer and added some tables showing what is being exposed. It would look something like this when rendered:

[wdamon-intel]

LGTM

[jandres742]

@MichalMrozek : do you have further comments or think that PR is good to go?

[MichalMrozek]

The initial plan was that new api getRoot is only available in mode 2.
Right now it will be enabled in mode 0, so this would be addition to existing model.
We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc.
Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

[jandres742]

The initial plan was that new api getRoot is only available in mode 2. Right now it will be enabled in mode 0, so this would be addition to existing model. We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc. Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

@MichalMrozek : currently, we allow for sub-devices to share the resources from the root device, that's correct, and you are correct, the same cannot be assumed when you have created a resource in a sub-device, and then getRootDevice is called.

So we could add something like this to getRootDevice:

"To allow the device handle returned by getRootDevice access the resources created by the sub-device handle, these resources need to be created with a context containing explicitly both handles.".

[jandres742]

The initial plan was that new api getRoot is only available in mode 2. Right now it will be enabled in mode 0, so this would be addition to existing model. We need clear explanation that resources created for sub devices will not work on extracted root devices, unless root device is a part of the context, same with programs, kernels etc. Right now we allow that resources created on root devices are available in sub devices , even if they are not in the context, so we need to be careful to not have the same level of expectations with new getRoot API.

@MichalMrozek : currently, we allow for sub-devices to share the resources from the root device, that's correct, and you are correct, the same cannot be assumed when you have created a resource in a sub-device, and then getRootDevice is called.

So we could add something like this to getRootDevice:

"To allow the device handle returned by getRootDevice access the resources created by the sub-device handle, these resources need to be created with a context containing explicitly both handles.".

Added clarification.