RFC: Sensor YAML

[teburd]

Introduction

Sensors need structured metadata associated with them to be able to be able to do things
statically at build time and optionally at run time.

Problem description

Statically defining things like buffers that allow for direct reading of a subset of enabled sensor channels with varying sample bitwidths along with metadata requires an understanding at build time of these attributes.

Some sensors may have a large number of possible channels and channel configurations.

Proposed change

Akin to board.yml have each sensor driver provide a sensor.yml with metadata about available channels and sample sizings allowing for C defines and macros to be generated that provide the
metadata at build time for reading subsets of channels.

Detailed RFC

An earlier attempt at doing this in devicetree failed to find a foothold as it..

• Required usage of subtree includes
• Still would have required custom device tree macros to obtain the desired information.

The goal is to be able to have a macro akin to

SENSOR_BUF_SIZE(sensor_model, ...) which takes a set of tokens representing a channel and configuration and provide an exact buffer size needed to hold the data for that channel subset and configuration.

E.g.

#define BUF_SIZE SENSOR_BUF_SIZE(SENSOR_ICM42688, SENSOR_CHANNEL(SENSOR_CHAN_ACCEL_XYZ, 0 ICM42688_ACCEL_20BIT), SENSOR_CHANNEL(SENSOR_CHAN_GYRO_XYZ, 0, ICM42688_GYRO_20BIT), SENSOR_CHANNEL(SENSOR_CHAN_TEMP, 0, ICM42688_TEMP_16BIT))

static uint8_t icm42688_buf[BUF_SIZE]

sensor_read(icm42688_iodev, r, icm42688_buf, ARRAY_SIZE(icm42688_buf));

Proposed change (Detailed)

A sensor.yml in each sensor driver directory, with a yet to be defined schema associated with it, containing all the necessary metadata to successfully provide, at compile time and optionally run time, buffer sizes for specific channel configurations. Along with a new python build script which takes said yaml file and generates appropriate C defines which a common set of macros to inspect the sensor metadata.

Alternatives

Devicetree was initially tried for this task and had some unfortunately downsides due to limitations around both the ability to inject subtrees in dts as well as an inability to reuse C defines in dts schema yaml files.

#65830
#71198

[teburd]

Created this RFC after a discussion in the sensor wg today, April 8 2024 with below attendees in agreement.
@ubieda @laurenmurphyx64 @yperess @MaureenHelm

[nordicjm]

Would an expansion of/plugin for #68127 help for this? How would this work with sensors that support different levels of precision? E.g. an adc might support 8-bit, 10-bit and 12-bit output, maybe that is set at compile time with a Kconfig, but maybe it set at runtime (and an application might switch dynamically depending upon power/system state), or different precisions for the same adc device but on a per-channel basis

[jukkar]

Hi @teburd, in #68127 I created a way to describe system network interfaces in a yaml file, then have some Cmake tooling to use that yaml file to generate a C header file to be used in network configuration library to setup network interfaces when device boots.

If you are interested, the tooling part in that PR could be used to generate the sensor configuration data from a yaml file. You would need to create a python script that could convert the yaml file to some other format usable in runtime.

[teburd]

Would an expansion of/plugin for #68127 help for this? How would this work with sensors that support different levels of precision?

I definitely think that would be worth a look, @yperess hacked around and already got something working in less than a day doing most of what I think we want.

E.g. an adc might support 8-bit, 10-bit and 12-bit output, maybe that is set at compile time with a Kconfig, but maybe it set at runtime (and an application might switch dynamically depending upon power/system state), or different precisions for the same adc device but on a per-channel basis

In that scenario you could allocate to the largest buffer size needed for the largest configuration you want at compile time and be done. Or allocate dynamically on the stack. Or allocate dynamically from a heap. Or dynamically allocate using the rtio mempool allocator. That's a feature of the new API, how memory is allocated for reading these things comes from the application now and not buried in the driver.

The most common use case I've had myself, and I'd guess many others have is likely an "on" mode and perhaps a low power/off mode. E.g. I already know my sample size and rate and the channels I want to read from at build time because the design of the firmware requires it. So being able to statically allocate the buffers for that to be the exact right size is very useful.

The above macro includes a device specific sample size format. I believe having a SENSOR_SAMPLE_SIZE() type macro here that takes the device type, channel, and a device specific configuration TOKEN.

The size of the buffer can then be sorted out at build time. There's other neat things I think we had in mind with this data as well.

The sensor shell/samples/tests ideally would know what channels are available for which sensors. That's not really possible today, we instead have a sensor shell sample that attempts to read all the channels, samples per sensor type, etc.

[teburd]

Hi @teburd, in #68127 I created a way to describe system network interfaces in a yaml file, then have some Cmake tooling to use that yaml file to generate a C header file to be used in network configuration library to setup network interfaces when device boots.

If you are interested, the tooling part in that PR could be used to generate the sensor configuration data from a yaml file. You would need to create a python script that could convert the yaml file to some other format usable in runtime.

I definitely think its worth looking at, @yperess has volunteered to tackle this so its up to Yuval in the end how to approach it.